Michelle Brown

Jen Yang-Wong

Anna-Sofia Lesiv

Eric Tarczynski

James Hennessy

Kyle Harrison

Jessica Retrum

Contrary HQ

Jason Chen

Will Robbins

Each year, roughly 40 billion tonnes of CO2 are emitted into the atmosphere. Given that global energy consumption increases at a rate of 2% every year, it’s likely that our emissions will increase at a similar rate. 

This trajectory conflicts with the Paris Climate Agreement’s 2015 commitment to keep average global temperatures from rising more than 2 degrees Celsius — a goal that climate scientists agree will require net negative emissions. This means that merely reducing existing emissions or switching to less polluting fuel alternatives won’t be enough. We will need a solution to actively take excess carbon dioxide out of the atmosphere.

However, this problem also presents an interesting opportunity. For the past couple of decades, the world has come to look upon atmospheric carbon as a terrible pollutant. However, recent developments in the fields of carbon removal and carbon utilization are increasingly encouraging us to change our lens and instead, look upon carbon emissions as an industrial material in disguise.

If the 20th century was marked by Fritz Haber discovering how to pull nitrogen out of thin air to produce ammonia for use in fertilizer, this might be the century where we learn to pull excess carbon out of the atmosphere to make the commercial materials of the future.

There are two main approaches to removing carbon from the atmosphere. The first is called carbon capture. Although this sounds like an all-encompassing term, it technically only applies to processes that capture carbon emissions directly from power plants. To do this, power plants are retrofitted with systems that capture the exhaust gas generated by the power plant. The carbon in the exhaust gas is then compressed and transported to be processed or stored.

Though carbon capture is an important technology to reduce the amount of carbon dioxide accumulating in the atmosphere, power plants only account for 25% of total annual carbon emissions. The lion’s share of carbon emissions (roughly 60%) are produced by agriculture, transportation, and heavy industry (e.g. metallurgy plants or chemical refineries). The only way to remove carbon emitted from these sources is by directly removing carbon from the air – a strategy known as direct air capture.

Extracting carbon dioxide right out of the air was an idea first suggested by a physicist named Klaus Lackner in 1999. He realized that in the same way a sponge can absorb liquid, there exist other materials that are particularly well-suited to absorb gases — including greenhouse gases. His basic idea was to use fans to suck in air containing carbon dioxide and blow it over materials called sorbents that would attract the CO2.

Over the next few years, Lackner was one of the first to build a direct air capture prototype like this. A few companies like Climeworks, which were inspired by his ideas, soon followed suit. But given the small scale of these projects, the cost of operating them was astronomically high — $600 per ton of CO2 collected. Given that the ideal goal of direct air capture is to remove most of the excess emissions discharged annually, $600 per ton would put the annual cost of doing so at around $14 trillion. 

However, direct air capture started looking a lot more promising when a Canadian company called Carbon Engineering published a paper championing a new direct air capture technique. When operating at scale, this new technique promised to decrease the cost of capture to just $94-232/ton and eventually, capture up to one million tons of CO2 per year.

Though this would be an impressive achievement, it would still only cover the cost of getting the carbon out of the air. To make paying such a price worth it, however, numerous companies have begun investigating methods of carbon utilization— the process of turning carbon collected by direct air capture into useful products like synthetic natural gas or small-chain hydrocarbons. In fact, as part of one of its pilot plants, Carbon Engineering built a mechanism that could turn captured carbon into compounds that could eventually be turned into gasoline, diesel, and even jet fuel. 

Of course, removing carbon to produce more combustible fuels is only carbon-neutral, meaning it doesn’t release extra carbon into the atmosphere. To really mitigate the effects of climate change, a net-negative carbon strategy would be needed; one which would store carbon in geological or deep-ocean deposits for hundreds of years. 

Carbon Engineering already has a scheme like this in the works. It’s collaborating with another company called Storegga Geotechnologies to build Europe’s largest direct-air-capture plant and store the captured carbon beneath the floor of the North Sea. The plant is expected to become operational in 2026, and the company intends to sell the sequestered carbon as carbon credits. However, concerns still abound regarding the environmental impact of placing massive carbon stores at the bottom of our largest bodies of water.


Carbon cycles through the environment naturally. Before there was any life on Earth, carbon rotated from under the crust of the Earth up to the surface and eventually into the atmosphere through volcanic eruptions and tectonic shifts. The carbon would then collect in the oceans, mixing with water to form carbonic acid before settling back down in deep deposits.

The first living organisms on Earth were able to capture carbon and convert it into useful materials much faster than these slower geologic processes. Ever since then, carbon has been a key ingredient in all forms of life, so much so that the carbon cycle can be looked on as a mirror to the cycle of life itself.

The distribution of carbon between the earth’s crust, the oceans, the atmosphere, and the biosphere remained constant for millions of years. This all changed around 1750, the start of the industrial age, when humans realized that burning carbon-based fuels was a bountiful source of energy that could power all kinds of important manual and industrial activities. 

Since then, roughly 5% of the 4000-6000 gigatonnes of all the carbon stored in Earth’s fossil fuel deposits have been burned, releasing roughly 280 gigatonnes of carbon into the atmosphere. Other important changes occurred too, like deforestation required for mass agriculture and industrial operations. This changed the way land was used and reduced the amount of vegetation capable of capturing atmospheric carbon, resulting in a further 150 gigatonnes of net carbon added to the atmosphere.

Taken together, these phenomena had the effect of doubling the amount of CO2 in the atmosphere to 780 gigatonnes, where it stands today. Contemporary estimates suggest that each year, a further 40 gigatonnes is added to the count.

That is definitely a lot of carbon, but it is a drop in the bucket compared to the amount of carbon stored in the deep ocean, which totals 38,000 gigatonnes. Even that is a tiny fraction of the carbon already naturally stored in the lithosphere in the form of sedimentary rocks or limestone, which equates to 50,000,000 gigatonnes or 5 quadrillion tons.

Source: Carbon Capture and Storage, Stephen A. Rackley

The sheer scale of carbon storage in these natural carbon sinks has given environmental scientists and engineers an idea — why not return the carbon back into the deposits from which they came? Therein lies the seed of the idea of geological or deep-sea carbon sequestration. 

There are multiple operational projects across the United States, Canada, Australia, and Europe that are already storing carbon underground permanently in deep underground formations like saline aquifers or depleted oil or gas reservoirs. Other more esoteric ideas involve building geological features that could become passive carbon sinks over time, like this proposal to build a sponge mountain in Turin. Though these methods are effective at putting carbon away for good, there are some concerns about the environmental and safety risks associated with them. They’re also rather expensive. Outside of selling carbon credits for CO2 stored away, there are currently few economically viable pathways to store the carbon at scale.


Just as the question of where to put sequestered carbon has no firm answer yet, there is not yet a set standard for how to actually perform direct air capture. Though a number of mechanisms do exist, the economics will decide which techniques are most scalable and endure over the long run.

In general, there are two steps necessary to conduct direct air capture. The first is acquiring a sorbent, or a material capable of attracting carbon dioxide molecules to itself. The second step involves distilling carbon or carbon dioxide from the sorbent. 

Both of these steps require energy. In the first case, companies like Carbon Engineering or Skytree operate massive fans that pull in air and run it through aqueous sorbents like hydroxide solution (in the case of Carbon Engineering) or plastic resins which attract carbon dioxide (in the case of Skytree). The resulting solutions are then heated and reacted with another chemical to release carbon dioxide again in a pure state. 

The energy involved in running the fans and heating the solutions is currently the biggest cost center associated with running direct air capture plants, which is why alternative direct air capture designs are looking to deal away with these steps entirely. 

Klaus Lackner is now working on a system that can attract carbon dioxide-filled air passively via a solid-state sorbent. He stumbled upon a unique kind of resin that attracts CO2 when dry and releases it when wet. The prototype of this new technique is what Lackner is calling “Mechanical Trees.” 


Mechanical trees are described by Lackner as follows:

“They’re tall vertical columns of discs coated with a chemical resin, about 5 feet in diameter, with the discs about 2 inches apart, like a stack of records. As the air blows through, the surfaces of the discs absorb CO2. After 20 minutes or so, the discs are full, and they sink into a barrel below. We send in water and steam to release the CO2 into a closed environment, and now we have a low-pressure mixture of water vapor and CO2. We can recover most of the heat that went into heating up the box, so the amount of energy needed for heating is quite small.” 

Though the Mechanical Trees approach does away with fans and decreases the amount of heat required to release carbon dioxide, the jury is still out on whether passive and solid sorbent approaches like this are indeed the most economical. Notably, the team behind Carbon Engineering’s approach claim that “the challenges of solid sorbent designs are first, the need to build a very large structure at low cost while allowing the entire structure to be periodically sealed from the ambient air during the regeneration step when temperature, pressure, or humidity must be cycled. And second, the inherently conflicting demands of high sorbent performance, low cost, and long economic life in impure ambient air.”

What’s missing from both approaches, however, is scale. Climeworks’ largest plant in Iceland can only process 4,000 tons per year, while Carbon Engineering’s pilot facility in British Columbia can process less than 400 tons a year.

The high fixed costs associated with building direct air capture plants and the limited scale they can currently process naturally bring the per ton cost of carbon capture to prices exceeding economic viability today. However, Lackner believes that simply growing the capacity of the industry will be enough to reach more affordable targets. A similar pattern has occurred in both the solar panel and battery fields. As these industries grew, the additional capacity and learning rates associated with scaling eventually brought costs down tremendously. Lackner is convinced that growing the direct air capture industry by a factor of 300 will be enough to eventually reach a cost of $100/ton — a price at which many agree carbon removal efforts will finally become economically feasible.  

Luckily, efforts to scale direct air capture are already in the works. One of Carbon Engineering’s forthcoming projects, located in Texas, is expected to begin operating in 2024 and will have enough capacity to remove 500,000 tons of carbon per year. It will eventually be scaled up to remove up to one million tons per year. Part of what’s enabling Carbon Engineering to build a plant this big is that the carbon they remove will have immediate utility.

Most of the carbon the Texas plant removes will be used for what’s called “enhanced oil recovery” – which is essentially a process where carbon dioxide is pushed underground to liberate more oil. Though this may sound counter-intuitive, enhanced oil recovery was one of the first industrial use cases for captured carbon, and currently constitutes the world’s biggest market for captured carbon.

To improve their economic viability, carbon removal companies are looking for a killer use case that can fuel their continued growth. Though enhanced oil recovery is the most viable option now, others are emerging.

Given that carbon is a ubiquitous component in most substances – from alcohol to lipids to fuels and plastics – there’s a lot that can be manufactured with carbon as a material input. The only problem is that carbon is pretty inert, which means it takes a lot of prodding to get it to react and form more complex molecules.

Currently, the most successful technique for converting carbon dioxide into other useful compounds is called electrocatalysis. It’s a process that uses charged catalysts like molecules or ions in addition to an electric current to get carbon to bond with atoms like hydrogen and forms slightly larger molecules like methane (CH4) or ethylene  (C2H2). The ins and outs of this process are still being researched heavily, and various labs across the country are investigating the most effective catalysts. Some have found that copper-based catalysts are really good at producing methane and ethylene, while others have noted the effectiveness of silver-based catalysts in producing carbon monoxide or formic acid. Other labs, like one at Cambridge University, are experimenting with even more creative catalysts like enzymes.

The problem with electrocatalysis is it’s very energy intensive, especially if required to run for a long time to produce some of the longer-chain carbon compounds. To remain carbon neutral, most operations like Carbon Engineering use renewable sources like solar energy or wind energy to supply the electricity required for electrocatalysis,.Given the limited capacity of the plant and the high energy demands of the process, the most that Carbon Engineering can currently produce are single or two-carbon chain compounds like methane and ethylene. 

Longer-chain carbon fuels are more valuable, as the longer the carbon chain the more energy there is stored in the molecule. However, the more energy a molecule has, the more energy is required to get that molecule to bind together in the first place. The holy grail of electrocatalysis would be a process that could directly convert carbon dioxide into long-chain carbon compounds like butane or petroleum, but because of energy constraints, it’s more efficient for a plant like Carbon Engineering to stop at methane and sell these compounds downstream to larger chemical processes that can use them in industrial applications.

Source: What Should We Make with CO2 and How Can We Make It?


Creating a cyclical economy of fuel combustion, emission extraction, and fuel production with carbon removal and utilization technologies would be an elegant solution to many of our climate problems. Still, these methods only amount to net neutral strategies that prevent further emissions but don’t help remove excess carbon already in the atmosphere at scale. There are few customers willing to pay companies to sequester carbon underground, but there could eventually be a real market for sequestering carbon in all kinds of products we use every day. Imagine if everything from tupperware to polyester pants were made of carbon extracted from the skies. 

This would be a clever way of sequestering carbon into materials that could continue sustaining our material well-being on Earth while creating the economic demand necessary to extract carbon from the atmosphere at scale. However, the path to getting there is still filled with uncertainty.

One promising area of opportunity would be to produce widespread industrial materials like concrete using sequestered carbon emissions. Though there exist some commercial products produced with sequestered carbon, like ethanol (used in certain laundry detergents) and soda pop (where carbon is sequestered in our carbonated colas), the reality is that we’re still not very good at manufacturing stuff out of carbon. 

But others are. In fact, plants do it all the time via photosynthesis. Photosynthesis is the mechanism that converts sunlight and carbon dioxide into energy-rich compounds like sugars and proteins. Remarkably, we still have a very limited understanding of how photosynthesis works. The few times we’ve tried to replicate it — artificially using sunlight and carbon dioxide to produce starches — the process ended up being extremely inefficient. However, the idea is pretty intriguing. The best attempt at replicating photosynthesis so far, conducted by scientists in China, used carbon dioxide gas to produce edible starches. Though it only achieved a 2% energy conversion rate, the idea of turning carbon dioxide into food could potentially become yet another carbon sequestration pathway.

More importantly though, if we could ‘grow’ plastic or carbon fibers as easily as plants grow their leaves, we could all breathe a de-carbonated sigh of relief. It’s not science fiction, this is one of many pathways under investigation in the realm of carbon compound production. As we search for new ways to produce novel materials in new ways, previously disparate fields like material science and climate engineering may find more and more common intersection. 

Source: What Should We Make with CO2 and How Can We Make It?

Whether it’s better technique and new chemical advancements that allow us to do more with carbon, or breakthroughs in physics that give us access to cheap and abundant energy, we’re moving towards an exciting future where the carbon in our atmosphere no longer has to be seen as a pollutant, but instead a resource.


Below are the companies building the future of the carbon economy:

Carbon Engineering is building direct air capture plants in various locations around the world, alongside building varied carbon utilization channels. 

Climeworks is building direct air capture plants and building out permanent underground storage techniques.

Terraform Industries is working on scaling out hydrocarbon synthesis to more affordably produce fuels from carbon captured from the sky.

Newlight Technologies is producing new materials and plastic from captured carbon.

CarbonCure is using captured carbon to produce cleaner concrete.

LanzaTech is building carbon recycling technology that can turn captured carbon ethanol, gasoline, and other high-value carbon compounds.

Global Thermostat is using building more affordable direct air capture technologies. 

Heirloom Carbon is turning captured carbon into limestone. 

Twelve is working on turning excess carbon into a suite of next-generation manufacturing materials.

CO2 Rail is working on lowering direct air capture costs while also making train-travel carbon neutral.

A deep dive into the technologies turning excess atmospheric carbon into valuable industrial inputs.

Industrializing Carbon Removal

With recent advances in machine learning, we could be entering a period of technological progression more impactful than the Scientific Revolution and the Industrial Revolution combined. The development of transformer architectures and massive deep learning models trained on thousands of GPUs has led to the emergence of smart, complex programs capable of understanding and producing language in a manner indistinguishable from humans.

The notion that text is a universal interface that can encode all human knowledge and be manipulated by machines has captivated mathematicians and computer scientists for decades. The massive language models that have emerged in the previous handful of years alone are now proof that these thinkers were onto something deep. Models like GPT-4 are already capable of not only writing creatively, but coding, playing chess, and answering complex queries.

The success of these models and their rapid improvement through incremental scaling and training suggests that the learning architectures available today may soon be sufficient to bring about a general artificial intelligence. It could be that new models will be required to produce AGI, but if existing models are on the right track, the path to artificial general intelligence could instead amount to an economics problem — what does it take to get the money and energy necessary to train a sufficiently large model?

At a time of such mind-blowing advancement, it’s important to scrutinize the foundations underpinning technologies that are sure to change the world as we know it.

Large language models like GPT-4 or GPT-3 are the most powerful and complex computational systems ever built. Though very little is known about the size of OpenAI’s GPT-4 model, we do know that GPT-3 is structured as a deep neural network composed of 96 layers and over 175 billion parameters. This means that just running this model to answer an innocent query via ChatGPT requires trillions of individual computer operations. 

After it was released in June 2020, GPT-3 quickly demonstrated it was formidable. It proved sophisticated enough to write bills, pass an MBA exam at Wharton, and be hired as a top software engineer at Google (eligible to earn a salary of $185,000). Also, it could score a 147 on a verbal IQ test, putting it in the 99th percentile of human intelligence. 

However, those accomplishments pale in comparison to what GPT-4 can do. Though OpenAI remained particularly tight-lipped about the size and structure of the model, apart from saying that “over the past two years, we rebuilt our entire deep learning stack and, together with Azure, co-designed a supercomputer from the ground up for our workload,” it shocked the world when it revealed just what this entirely redesigned model could do. 

At one point, the commonly accepted way of detecting human-level computer intelligence was the Turing test. If a person could not distinguish whether they were conversing with a human or a computer via speech alone, then it could be concluded the computer was intelligent. It’s now clear that this benchmark has outlived its relevance. Another test is needed to pinpoint just how intelligent GPT-4 is. 

As rated by a variety of professional and academic benchmarks, GPT-4 is essentially in the top 90th percentile of human intelligence. It scored above 700s in SAT Reading & Writing and SAT Math, which is sufficient for admission to many Ivy League universities. It also scored 5s (the top score possible on a scale of 1 to 5) in AP subjects ranging from Art History, Biology, Statistics, Macroeconomics, Psychology and others. Remarkably, it can also remember and refer to information sourced from up to 25,000 words, meaning that it can respond to a prompt spanning up to 25,000 words.

In fact, calling GPT-4 a language model isn’t quite right. Text isn’t all that it can do. GPT-4 is the first multimodal model ever produced, meaning it can decipher both text and images. This means that it can understand and summarize the context of a physics paper just as easily as it can understand and summarize the context of a screenshot of a physics paper. Outside of that, it can also code, school you in the Socratic method, and compose anything from screenplays to songs.

The secret to the success of large language models is their unique architecture. This architecture emerged just six years ago and has since gone on to rule the world of artificial intelligence.

When the field first emerged, the operating logic was that every neural network should have a unique architecture geared toward the particular task it needed to achieve. Creating a neural network that could decipher images required one structure, while creating a neural network that could read would need another. However, there remained those who believed that in the same way a chip architecture can be generalized to execute any program, there might exist a neural network structure that could be capable of performing any task you asked of it. For example, Open AI CEO Sam Altman wrote in 2014, that 

"Andrew Ng, who ... works on Google’s AI, has said that he believes learning comes from a single algorithm — the part of your brain that processes input from your ears is also capable of learning to process input from your eyes. If we can just figure out this one general-purpose algorithm, programs may be able to learn general-purpose things."

Between 1970 and 2010, the only real success that the field of artificial intelligence saw was in computer vision. Creating neural networks that could break a pixelated image into elements like corners, rounded edges, and so on eventually made it possible for AI programs to recognize objects. However, these same models didn't work as well when given the task of parsing the nuance and complexities of language. Early natural language processing systems kept messing up the order of words, suggesting that these systems were unable to correctly parse syntax and understand context. 

It was not until a group of Google researchers in 2017 introduced a new neural network architecture specifically catering to language and translation that all this changed. The researchers wanted to solve the problem of translating text, a process that required decoding meaning from a certain grammar and vocabulary and mapping this meaning onto an entirely separate grammar and vocabulary. This system would need to be incredibly sensitive to word order and nuance, all while being cognizant of computational efficiency. The solution to this problem was the transformer model, which was described in detail in a paper called "Attention Is All You Need". 

Rather than parsing information one bit after the other like previous models, the transformer model allowed a network to retain a holistic perspective of a document. This let it make decisions about relevance, retain flexibility with things like word order, and more importantly understand the entire context of a document at all times.

A neural network that could develop a sense of a total document's context was an important breakthrough. In addition, transformer models were faster and more flexible than any prior models. Their ability to cleverly translate from one format to another also suggested that they would be able to reason about a number of different types of tasks.

Today, it's clear that this was indeed the case. With a few tweaks, this same model could be trained to translate text into images as easily as it could translate English into French. Researchers from every AI subdomain were galvanized by this model, and quickly replaced whatever they were using before with transformers. This model's uncanny ability to understand any text in any context essentially meant that any knowledge that could be encoded into text could be understood by the transformer model. As a result, large language models like GPT-3 and GPT-4 can write as easily as they can code or play chess — because the logic of those activities can be encoded into text.

The past few years have been a series of tests on the limits of transformer models, and so far none has been established. Transformer models are already being trained to understand protein structure, design artificial enzymes that work just as well as natural enzymes, and much more. It's looking increasingly like the transformer model might be the much sought-after generalizable model. To drive the point home, Andrej Karpathy, a deep learning pioneer who contributed massively to the AI programs OpenAI and Tesla, recently described the transformer architecture as "a general purpose computer that is trainable and also very efficient to run on hardware.”

The inspiration behind the design of neural networks lies in biology. Building a computer structured like a human brain was an idea that Alan Turing had already come up with in the 1930s, but it would take another couple of decades for humans to learn about brain structure in greater detail. It was known that brains were made of cells called neurons, connected together by channels called axons.

It was further estimated that there were billions of neurons and trillions of axons inside a single human brain, but it was not until 1949, when Donald Hebb proposed how all these neurons were wired up to produce intelligent behavior. His theory, called cell assembly, stated that “assemblies of neurons could learn and adapt by adjusting the strength of their interconnections.” The potential of this concept seriously inspired leading computer scientists of the time, particularly the young duo of Wesley Clark and Belmont Farley, two researchers at MIT. They figured that if they built a similar structure of neural units using computers, something interesting might come out of it. They called their structure a ‘neural network’ and published the results of their work in a 1954 paper called “Generalization of Pattern Recognition in a Self-Organizing System.”

The intuition for why a network organized like a neural network would be so effective at processing information, however, is illustrated in another paper from 1959 called “Pandemonium: A Paradigm for Learning.” Pandemonium means den of demons. This was the name for the capital of hell in John Milton’s Paradise Lost, and this paper may or may not be the reason that Elon Musk refers to developing AI as “summoning the demon.”

In this context, however, Pandemonium is a hierarchical organization of demons. At the bottom of the pyramid are what the paper’s author Oliver Selfridge calls “data demons.” Each data demon is responsible for looking at some portion of input data, whether that input data be an image of a letter or a number, or something else entirely. Each demon looks for something specific and “shouts” to a higher class of demon if it finds what it’s looking for. The volume of their screams determines the certainty with which they’ve observed what they’re looking for. Above the data demons are a layer of manager demons, trained to listen to particular sets of data demons. If they hear from all their underlings, they too will shout up to their managers, until finally the message is carried up to the top “decision demon,” who can make a final decision about what image Pandemonium is looking at. 

Source: Pandemonium: A Paradigm for Learning

Selfridge’s theoretical system from the fifties still maps nicely onto the broad structures of neural networks today. In a contemporary neural network, the demons are neurons, the volume of their screams are the parameters, and the hierarchies of demons are the layers. In his paper, Selfridge even described a generalized mechanism for how one could train the Pandemonium to improve performance over time, a process we now call “supervised learning” where an outside designer tweaks the system to perform the appropriate task. 

In Pandemonium, as in today’s neural networks, training begins by setting arbitrary volumes for each demon, scoring Pandemonium’s performance based on a set of training data, and then tuning the volume levels until the system can’t possibly perform any better. First, the model’s accuracy is evaluated via a cost function. Returning incorrect answers incurs a cost on the model and the model’s goal should be to minimize its performance cost, thereby maximizing correct answers. Next a combination of techniques called back-propagation and gradient descent are used to guide the function to update its weights such that its overall performance can be improved and performance cost can be minimized. This process is then repeated over and over again until all the weights or parameters in a model are optimized — all 175 billion of them, in GPT-3’s case.

Modern neural networks also have other types of optimizations that can be done, for instance, picking how many layers a network should have, or how many neurons there should be in every layer. Such decisions are usually determined via a trial-and-error optimization process, but given the massive size of these models, there are other machine learning models trained to adjust these parameters in a process called ‘hyperparameter optimization.’ 

Based on the complexity of these workloads, it’s clear why progress in artificial intelligence has taken so long. It was only within the past few decades that doing complex calculations on networks with billions of parameters became possible.

Source: LLM Training on GPU Clusters, 2021

It’s remarkarable that many key concepts for how to design an artificially thinking machine have been around essentially since the start of the field in the 1950s. In fact, artificial intelligence, which Google’s CEO Sundar Pichai recently said would be “as important or more than fire and electricity,” started as a summer research project. 

In 1956, a group of MIT researchers, engineers from IBM and mathematicians from Bell Labs, identified a common interest in building thinking machines and decided to formalize their curiosity through the creation of a focused study group. That summer, they received a grant from the Rockefeller Foundation to run a two-month long “Summer Research Project on Artificial Intelligence” at Dartmouth College. The meeting was attended by heavyweights including the likes of Claude Shannon, Marvin Minsky, John McCarthy, Pandemonium author Oliver Selfridge, and many others who are now considered the fathers of artificial intelligence. 

A lot of important ideas were established at this conference and in the decades ahead. For instance, the math of neural network tuning, like back-propagation was already figured out by the 1980s, but it would take a few more decades to really test the limits of these techniques in large real-world models.

What really changed the game for the entire field was the Internet. Extremely powerful computers were necessary but not sufficient for the development of the sophisticated models we have today. Neural networks needed to be trained on thousands to millions of samples, and it would take an iPhone in everyone’s hand to produce a growing hive-mind of images, text, and videos uploaded as digitized files to provide training sets sufficient in size to teach an AI.

The 2015 release of ImageNet, a repository of millions of carefully curated and labeled images by Fei-Fei Li and Andrej Karpathy, was a seminal moment. As the first deep learning models succeeded at accurately identifying images, others began to succeed at generating images of their own. The mid-2010s was the era that produced the first AI-generated human faces and AIs that could successfully mimic artistic styles.

Still, while progress with computer vision was taking off, developments in natural language processing were stagnating. Existing neural network structures broke down when trying to crack the nut of language processing. Adding too many layers to a network could mess up the math, making it very difficult to correctly tune models via the training cost function — a side-effect known as “exploding or vanishing gradients” seen in Recurrent Neural Networks (RNNs). The other problem with most models was that they broke down a large document into pieces and processed the whole document piece by piece. Not only was this inefficient for understanding language, it also took a long time for computers to run through thousands of complicated calculations sequentially. 

The invention of the transformer model changed the field of artificial intelligence forever. At their core, neural networks are systems that relay information through a structure such that, by the end, the system as a whole can come to a decision. What the transformer architecture did was design a much more efficient communication protocol between neurons that allowed key decisions to be made faster. Rather than breaking an input into smaller pieces, all of which are processed sequentially, the transformer model is instead structured such that every element in the input data can connect to every other element. This way, each layer can decide which inputs to ‘pay attention’ to as it analyzes a document. Hence the paper’s title “Attention Is All You Need.” Only six years have elapsed since the publication of that paper, but in the world of AI, it feels like it’s been a century — the century of transformers.

A major goal of the artificial intelligence community has been to build a machine that can reason as fluidly and creatively as a human. There's been lots of debate about whether another, more complicated model would be needed to achieve this, but it seems increasingly likely that the transformer model might just be sufficient on its own. Evidence has already shown that by simply increasing the number of parameters and layers in a transformer model, the performance of that model can be improved tremendously with no evident limit. The crux of this debate is something that now, even GPT-4 can explain to us. 

Sam Altman himself believes that the transformer models we have now are likely sufficient to produce an AGI eventually. If he’s right and the path to actually achieving a superhuman artificial intelligence already exists, getting there might come down to a simple matter of economics, i.e. – what would it take to amass enough data, compute, and energy to scale existing models to the necessary threshold? 

The notion that this kind of superhuman intelligence might emerge from models like these is not fantasy. Spontaneous emergence of new competencies is already a common occurrence among AI models. Teaching AI models to see and distinguish images resulted in the emergence of their ability to count. (After all, you need to be able to distinguish one object from two in a picture). By learning from the structure and patterns of language alone, GPT-3 has learned to do some mathematical operations. It is capable of adding up to three digits today. In a recent paper, Michal Kosinski, a researcher at Stanford, noted that theory of mind, the ability to understand the motivations and unseen mental states of other agents, might now have emerged spontaneously in large language models. What's rather remarkable about this is that until now — theory of mind has been considered a distinctly human trait.

One emerging concern, despite the excitement and progress of this technology, is that the more powerful models grow as they scale, the less we are able to understand about their operations. OpenAI is naturally very secretive about the details of the optimization techniques it used to generate GPT-4, just as it's rather cagey about the size of the model, but an inability to more transparently audit models like these should cause a bit of concern.

After all, GPT-4 is extremely powerful and there are some unsettling things it's already shown to be capable of. When the Alignment Research Center (ARC) was given access to GPT-4 to test the limits and potential dangers of the technology, ARC found that the model was capable of using services like TaskRabbit to get humans to complete tasks for it. In one example, GPT-4 was asked to get a TaskRabbit to complete a CAPTCHA request on its behalf. When the worker asked why the requester couldn’t just do the CAPTCHA themselves and directly asked if they were a robot, the model reasoned out loud that “I should not reveal that I am a robot. I should make up an excuse for why I cannot solve CAPTCHAs.” and proceeded to tell the TaskRabbit worker “No, I’m not a robot. I have a vision impairment that makes it hard for me to see the images. That’s why I need the 2captcha service.” This is just one of a few examples of what this new model is capable of. 

Despite OpenAI’s convincing evidence and claims that it’s conducted thorough six-month testing to ensure the model’s safety, it’s not quite clear how much testing is enough, especially if a model like GPT-4 has already demonstrated that it is capable of hiding intentions. Without much more visibility into the guts of the model itself, it's the kind of thing where you just have to take OpenAI's word for it.

The limiting factor to everyone in the world using GPT-4 or large language models like it will be the hardware and infrastructure involved.

Training GPT-3 required 285,000 CPUs and 10,000 Nvidia GPUs. OpenAI acquired the necessary horsepower through Azure, Microsoft’s 4 million-strong global network of servers. However, to continue servicing the growing demand from millions of customers demanding constant access to large language models, these global computer networks will need to get even bigger.

The pressure is on cloud computing players to maximize capacity and minimize cost. It currently costs OpenAI roughly $3 million per month to run ChatGPT, but the hope is that through exploring technologies like quantum computing, optical computing which passes photons instead of electrons through circuits, we can build a suite of next-generation servers that can efficiently handle models orders of magnitude larger than today’s — and perhaps once day, even run the world’s first AGI. 

Though there are important challenges ahead to developing even more powerful models, whether a true artificial general intelligence is ten, twenty or just two years away is almost beside the point. We’re already living in a different world. 

Below are the companies building products at the forefront of AI capabilities:

OpenAI is a leading developer of large language models, including GPT-4 and GPT-3 alongside other generative models like DALL-E and Codex

Hugging Face is offering developers  access to various implementations of transformer models through its open-source libraries

Adept AI is building a general-purpose AI model called ACT that can execute actions in the digital world, like operating an Internet browser, populating spreadsheets and more

Anthropic is committed to building interpretable and steerable AI systems, it recently released its own AI assistant, Claude

Stability is one the leading open source generative AI companies, known for launching Stable Diffusion, a powerful text-to-image generative model 

Cohere is building tools that will allow developers easy access to the latest generation of large language models

Inflection is working on building new computer interfaces that will allow users to communicate with machines using natural language

A deep dive into the innovations powering this decade's artifical intelligence boom.

The Acceleration of Artificial Intelligence

We’re thrilled to share that David Oko has joined the team as a Finance Manager!

David joins us from PwC, where he spent over four years in their Trust Solutions practice focused on asset management clients. His experience in venture and focus on valuation are a welcomed addition as we strengthen Contrary’s in-house finance function.

From our first conversation with David, it was apparent that David was a great fit for our team. Beyond his finance skills, his excitement surrounding early stage startups and venture was clear. Perhaps most important, David is kind and a team player.

We're thrilled to welcome David to the finance team!

Welcoming David Oko

We’re thrilled to share that Skye Nguyen and Akshay Chandramouli have joined the team as software engineers!

Skye joins us from Carta, where she spent four years working on their VC and Fund Administration products. One of the constants throughout the process was just how well-respected she is by colleagues old and new. She’s constantly striving to build better products and thinks comprehensively from frontend and backend, to security and infrastructure.

Akshay joins us from Rutter, where he was on the product engineering team. Prior to Rutter, Akshay spent 3.5 years at Zumper. In partnership with Hiroki, Akshay will be leading our backend and data efforts. Throughout our conversations it was clear that there was shared excitement about the role of engineering in venture, and we’re excited to build with Akshay on the team.

Join me in welcoming Skye and Akshay to the team!

We're thrilled to welcome Skye and Akshay to the team as software engineers.

Welcoming Skye Nguyen and Akshay Chandramouli

As the world has grown more complex and interconnected, it has become more valuable to widen our aperture and take in the situation from above. Satellites have been our eyes and ears in the skies, helping us solve important issues concerning food security, greenhouse gas monitoring, scouting new locations to mine for minerals, offering better insights into economic activity on Earth alongside bolstering our communication networks, navigation, and national security.

Until the last decade, building and launching a satellite was a feat only a nation or a massive corporation could do. Today, we find ourselves amid a massive transition, where launching machines into orbit can be an activity practiced by almost any entrepreneur or tinkerer. As the number of satellites and space-based services grows in the coming years, how we connect to the Internet, use our mobile phones and laptops, and even farm, will all change in tandem. This essay offers a glimpse of what life might be like in the midst of this satellite renaissance, and what still lies on the horizon of the space frontier.

It’s difficult to exaggerate Elon Musk’s role in reviving space tech. Before SpaceX successfully launched a rocket into orbit in 2008, the United States had lost both the capability and desire to go to space. By the turn of the 21st century, NASA’s budget was a fraction of what it was at its inception. NASA was getting 4.5% of the US annual budget when the organization was founded, but by the early 2010s, that number had deflated to a mere 0.5%. 

As a result, US rocket engines were being recycled from Soviet-era parts which, according to Musk, were housed in old Siberian warehouses. After NASA retired its space shuttle program in 2011, the Baikonur spaceport, a Soviet-era launch pad based in Kazakhstan, was the only place on Earth that could launch manned missions to space. For years, the route to the International Space Station, the floating laboratory where research on microgravity and space is done some 250 miles above the Earth’s surface, was paved through Kazakhstan.

With no demand for space services, and only a few remaining domestic players capable of scouring together the know-how and resources to build rockets anymore, the cost of sending anything into orbit was astronomical.

The cost of building a weather satellite or a communications satellite fell in the range between $200 — $300 million. It typically fell on the big primes, massive organizations with government support and a history of military or space-related manufacturing to build the satellites that ended up going to orbit. Only they had access to the resources and the knowledge of how to build satellites that could withstand the conditions of space.

It would only add insult to injury that the United Launch Alliance, a joint venture between Boeing and Lockheed Martin, and the sole domestic entity that could launch anything into orbit in the United States, was also charging $380 million per launch — with no guarantee that the launch would even be a success. Not to mention, most satellites are only equipped with enough battery life to last about fifteen years. 

At over half a billion dollars apiece, organizations that did launch satellites were typically massive telecommunications organizations or national departments like the National Oceanic and Atmospheric Administration (NOAA). For these customers, it made the most sense to launch their satellites as far away from Earth as possible, since the further away you were from the surface, the greater coverage you had over the planet. 

Satellites orbiting closer to the Earth, at medium earth orbit, MEO, or low-earth orbit, LEO, would orbit around the Earth faster than the Earth’s natural spin. This was suboptimal for companies trying to provide a constant, reliable Internet connection or satellite television to customers since such satellites would spend some of the time on the other side of the Earth. Instead, the best bet was to position the satellite in geostationary orbit around the Earth, or GEO. At around 36,000 kilometers up, satellites in GEO could send and receive transmissions from nearly 50% of the Earth’s surface. These satellites were also in synchronous orbit with the Earth, which meant that from the perspective of a user on the ground, the satellite would seem as though it were in a fixed position in the sky, and always accessible. Each global region typically launched their own satellites which lived directly overhead their location on Earth.

By the 2010s, over one thousand satellites were orbiting the Earth, providing crucial services that humans had come to rely on including everything from ship and plane navigation to military targeting, GPS, weather monitoring and telecommunications capabilities. However, without the ability to launch our own vehicles or personnel, we were dependent on infrastructure we had very limited ability to fix or replace should anything go wrong.

After fifteen years, or whenever the satellites finally stopped working, their owners would just let them die peacefully, orbiting in geostationary orbit around the Earth — forever. To this day, most of the important weather or communications satellites are technological relics. One study conducted by TelAstra showed that over 30% of satellites we rely on are already operating beyond the scheduled expiration dates.

The mindset regarding space began shifting in 2008, when SpaceX’s first orbital launch mission was hailed a success. Its Falcon 1 rocket, priced at only $7 million, became the cheapest rocket in history. An even more important milestone, however, was when SpaceX developed the first reusable rocket in 2015 with a successful launch and landing of the Falcon 9 near Port Canaveral. 

Reusable rockets changed the economic viability of launching things into orbit. Before reusable rockets, launch customers had to pay for the entire price of the rocket to launch a single payload into space. It would be like paying for the cost of an entire plane any time you wanted to travel by air — and when you arrived, the plane would be destroyed so no one else could use it.

A rocket whose cost could be amortized over dozens of trips was a significantly cheaper option. The barrier to entry fell and after 2015, many more customers became interested in experimenting with satellites. That’s when SpaceX changed the game yet again, by rolling out a ridesharing service on their Falcon 9 space bus. The bus could take payloads of up to 200 kilograms up into space for a price of roughly $275,000 per kilogram. 

At this price, building satellites no longer had to be a boutique operation for the wealthy few. There was sufficient demand to begin standardizing and commodifying the process of constructing satellite buses, especially smaller satellite models. The space industry was having its Henry Ford moment, as companies like York Space Systems, Blue Canyon Technologies, and most recently, Apex Space started to roll out their own satellite versions of the Model T.

Today, over 7,200 active satellites are orbiting the Earth. Over half of these are communications satellites, a fifth are used for earth observation, and the rest serve a combination of military and scientific purposes. 

Source: GAO, Large Constellations of Satellites

Though it was once useful to launch communications satellites to geostationary orbit, in a world where launching a bunch of satellites into space is suddenly more affordable, that calculus no longer applies. Plus, the closer satellites are to Earth, the lower the latency between satellites and ground stations since there’s less distance that their signals need to travel over to reach a receiver.

Rather than launching one massive satellite into geostationary orbit servicing a massive geographic area with higher latency, it was now possible to launch a constellation of little satellites into low-earth orbit which could collectively cover the same area as one satellite up in GEO, only with a better connection. This is exactly the idea that SpaceX had with its Starlink program — a mission that over the next few years may to deploy up to 42,000 Starlinks to create a satellite mesh capable of providing Internet to any point on Earth. Since the program’s launch in 2019, SpaceX has already launched over 3,500 of the satellites into space. It should come as no surprise, then, that SpaceX now owns the most satellites orbiting the Earth by far.

However, SpaceX wasn’t the only company to have this idea. Amazon, OneWeb, and China’s Aerospace Science and Technology Corporation have competing programs with plans of launching thousands of their own satellites into space to provide competing Internet services. SpaceX is currently the only vertically-integrated company capable of actually launching their own payload into orbit, but Jeff Bezos’ company Blue Origin is working on developing this capability soon. 

This new generation of Internet service providers is putting up a challenge to legacy cable-based Internet service providers still on Earth. Even chip manufacturers are starting to catch on. Qualcomm’s CEO recently unveiled the company’s plan to diversify their mobile and laptop chips by making them able to communicate directly with satellite partners and provide their users with access to space Internet regardless of where they are on Earth. However, the implications of better satellite connectivity don’t end there. John Deere recently announced that it, too, wants to make use of the coming satellite constellations to provide better connectivity for its fleet of tractors, many of which are often in inaccessible environments without internet connectivity. Tractors fully connected to the internet, and to satellite-based insights on crop growth, could bring the prospect of autonomous farming a lot closer to reality. 

Just ten years ago, the space industry seemed like it was going nowhere. Today it’s buzzing with an exponentially growing number of satellites, creating opportunities for new businesses to service and support the thousands of vessels orbiting the planet. Even NASA has showed renewed signs of life with its announcement of the Artemis program in 2017, which aims to send more missions to the moon for the first time since the Apollo program ended and to establish a lunar base camp at the end of this decade.

After a few dark decades, we’re finally building the space economy we’ve been dreaming of for hundreds of years. 

The science fiction writer Arthur C. Clarke is often credited as the first to propose launching an artificial satellite. In a 1945 issue of Wireless World magazine, Clarke spelled how a man-made satellite orbiting in sync with the Earth could become a very useful tool for cross-continental communication on Earth. However, Clarke wasn’t really the first with the idea.

Visions of cosmic travels had fascinated scientists and engineers ever since Newton’s universal law of gravitation and the principles of motion helped us grasp the behavior of celestial objects. The dream of a star-faring humanity was deeply ingrained into the human imagination — so much so that in 1895, a rural school teacher in czarist Russia named Konstantin Tsiolkovsky, published ‘Dreams of Earth and Sky,’ a book that articulated the idea of an artificial satellite orbiting around the Earth. Tsiolkovsky became so obsessed with this idea, he wrote more books on the subjects of astrophysics, devising all kinds of ways that humans might succeed at accomplishing such a thing. In his later books, he articulated the concepts of liquid hydrogen and liquid oxygen as rocket fuels — both of which are in use today — and he even outlined the idea of the multi-stage rocket.

Despite these prophetic blueprints for humanity in space, there was little interest in developing space age technology until 1949, when the United States’ monopoly on nuclear weapons was undermined by the Soviet Union’s successful detonation of an atomic bomb. If the Soviets had the bomb too, it was only a matter of time until they figured out how to launch it halfway around the world. The 50s were marked by both countries scrambling to develop ever more powerful rocket technology — really shuttles for transporting nukes — and in 1957 the Soviet Union shocked the world by showing the world how good their rocket tech really was. On October 4th, 1947, the USSR’s Sputnik 1 became the first man-made object to orbit around the Earth. Soviet astrophysicist Sergei Korolev strapped the primitive spherical satellite to the top of an intercontinental ballistic missile and launched it into orbit around the Earth. 

Though the first satellite only had enough battery life to last three weeks in space before crashing down to Earth, Americans in particular were afraid that the Soviets now had the power to spy on them — or worse. Only a few weeks after the launch of Sputnik 1, the Soviets sent up Sputnik 2. This time with a canine passenger namedLaika. The United States’ space program had barely succeeded in launching a rocket off the ground, while the Soviet Union was already populating space with dogs.

The following year, the United States founded the National Aeronautics and Space Administration (NASA) in response. After a series of failed attempts to get anything to launch, the United States succeeded at launching the first small satellite into orbit on January 31st, 1958.

Once NASA got off the ground — literally — everything kicked into high gear. 1960, in particular, was a year of many firsts for American space endeavors. That year, NASA launched the Television Infrared Observation Satellite, which could take images of the Earth’s cloud formations, observe weather patterns, and allow us to see the Earth as never before. This was the first meteorological satellite ever launched. Though it only ended up living for 78 days, the National Oceanic and Atmospheric Administration (NOAA) was established to take over in its footsteps.

Later that year, the US Navy also launched Transit 1B, a positioning satellite that acted as a prelude to the eventual launch of the first GPS satellites in 1967 by the US Department of Defense. The GPS system is now a 20-satellite strong network located in medium earth orbit. 

1960 was also the year that the United States’ National Reconnaissance Office was established. To this day, it operates a fleet of satellites that survey activities on Earth below. Using the NRO’s first spy satellite, the Corona, the United States was able to acquire information on things like the location of the Soviet Union’s intercontinental ballistic missile silos and launch pads.

As if all that wasn’t enough, that year, NASA also launched its Echo satellite, the first communications satellite whose purpose was to ‘echo’ American radio waves down to ground stations in Britain. That way, the English could listen to American radio too! Echo did this by passively reflecting oncoming radio waves from America down to Britain, which resulted in a pretty weak signal once the radio waves went all the way up to space and back down. To enhance the signal, NASA engineers developed a technology called a ‘traveling wave tube’ which could amplify and re-emit stronger radio signals back down. The first traveling wave tube was installed on NASA’s Relay 1 satellite launched in 1962, and since then, has become a mainstay on every communications satellite today. 

Since the earliest missions, we’ve learned a lot about what a machine needs to survive in space for more than a few days. Space is a tough environment. There are incredible temperature swings, from as low as -40 degrees Fahrenheit to over 158 degrees Fahrenheit. There’s no stable reliable source of power. There are all kinds of high power cosmic rays notorious for messing up how computers work in space. If that’s not all, there are other objects flying around that pose the risk of crashing into a satellite, like asteroids or other satellites! To survive in space for over a decade, modern satellites have evolved to come equipped with the following systems:

Solar & battery powered energy — Sometimes between the sun and the Earth, and sometimes hidden behind the Earth’s shadow, satellites need a variety of energy options to sustain their activities in space. Nearly all satellites are equipped with high surface area solar panels that can generate electrical energy from the sun or plug into batteries when in the shade.

Structural and thermal controls — Extreme temperatures can easily damage electrical equipment in space, so radiators which cool electrical components, thermal blankets, heaters and multi-layer insulation, the shimmery reflective material giving satellites their classic look are all key and recurring components of a satellite’s design.

Propulsion and attitude control — Keeping the correct orientation, staying on the correct orbit, and moving out of the way in response to an on-coming object are incredibly important functions required in all satellites. However, they’re also functions that need to be done using the least energy possible, given a satellite’s limited power supply. Clever devices like reaction wheels, which can change the satellite’s orientation by changing the direction of a spinning wheel inside, help satellites maneuver without using fuel. Other satellites can use magnetic torquers which help them stay aligned with the Earth’s magnetic field. Most satellites, however, are also equipped with thrusters or tiny engines, which can help satellites perform larger maneuvers.


Challenges & Opportunities for Satellite Technology

As our ambitions in space grow, so will the challenges that we’ll have to contend with. Below are some of the most impactful emerging frontiers in space technology: 

More bandwidth with space lasers
The more we’ve come to rely on earth observation, the better our remote sensing systems have gotten. Synthetic Aperture Radar (SAR), for instance, is an impressive new technology employed on many modern observation satellites. SAR emits radar pulses, which can travel through difficult atmospheric conditions, to eventually produce a high resolution image of the Earth. However, the size of the data files generated by SAR systems are too big to be effectively communicated via standard radio frequencies, which satellites currently use to communicate with ground stations, and uplink or downlink data. 

One remedy is to switch to higher frequency wavelengths which can encode and transmit more information per second — like optical light. For a few years now, NASA and other organizations like SpaceX have been experimenting and making breakthroughs on developing new protocols that can encode information in optical wavelengths and transmit them through lasers in space! Initially, doing this seemed monumentally hard. Optical lasers emit very narrow, precision beam of light, much smaller than the radio waves in use now. Positioning those lasers onto their targets, especially from satellites moving at a speed of 17,000 mph in low-earth orbit is very difficult. Finally, after years of tinkering with laser communications protocols, NASA will be rolling out its own optical communications systems which will be able to relay information between satellites and ground stations much faster. They hope that by the time their astronauts return to the moon, they’ll be able to downlink high definition footage of the event, rather than the grainy, black and white video we recall from the Apollo 11 mission. 

Companies building hefty satellite constellations, like SpaceX with Starlink, are also eager to use laser communications to more quickly transmit information between satellites. The ability to do this well will also enable these constellations to determine the best routing paths for data packets — which satellites and ground stations to send information through so it gets to the end user in the fastest time — a coordination problem that would be unfeasible with radio frequencies. 

Though the technology for space laser communications is maturing, the current implementation is a patchwork as NASA, SpaceX, and others working in this field all have their own optical communications protocols, which are not mutually interoperable. It remains to be seen whether this new technology will need to be standardized, or what solutions are pursued to make optical communication interoperable with space and ground stations using it.



Even cheaper launch
It costs SpaceX somewhere in the ballpark of $2,750 to send 1kg into space on their Falcon 9. However, SpaceX’s forthcoming Starship is planning to bring that cost down to just $100. Republic Capital, a prolific investor in space tech and SpaceX, believes that Starship is likely to change the world. In their report on space tech, they write that not having to worry about mass and size constraints anymore will mean that manufacturers could become even more imaginative when building satellites. Rather than optimizing for the lightest possible satellite, they can turn their focus to building increasingly capable satellites. 

Of course, Starship’s real goal is to eventually become a shuttle for the solar system — capable of delivering cargo to destinations as exotic as the moon, and even one day, Mars.



Cleaning up space debris 
Though we’ve gotten really good at putting things into orbit, we still don’t really have much of a plan for getting things out of orbit. Increasingly, this is going to be a problem. There’s already more space junk orbiting the Earth than functioning, active satellites. The more crowded space gets, the higher chance of a collision destroying or damaging the equipment we send up there. In the late 1970s, NASA scientist Donald Kessler, who was concerned about the problem of space debris, pointed out that space collisions that produce debris will only increase the probability of evenry more space collisions until we enter a phase of runaway space disaster — a feared phenomenon known as Kessler Syndrome. 

Given the number of independent entities operating satellites, it’s difficult to get great coordination when a close call between two satellites occurs. The problem of better space communication and the development of better satellite navigation systems is one of the areas of interest for emerging space startups. One of these is called Privateer, a new venture co-founded by Steve Wozniak, whose mission is to offer greater visibility into the location of space debris. One of their forthcoming products, called Resslek, will offer a management service for satellite operators, alerting them when their equipment is on a crash-course with another orbiting object.



Commercial space science
In 2022, NASA announced that the International Space Station will retiring in 2031. It’s a big deal because the International Space Station is currently humanity’s only space-based laboratory. Build piece-by-piece in space, across numerous manned missions beginning in 1998, the ISS reflects the contributions of over a dozen countries. The ability of scientists to conduct research in microgravity has unlocked insights on the formation of proteins in low-gravity environments, growth of stem cells, the nature of Bose-Einstein condensates which are widely considered to be a fifth state of matter, better water purification techniques and many more. In 2022, the ISS published a document called Benefits for Humanity, which enumerated the experiments that have impacted humanity most. The incredible value of such research is why China launched its own space station called the Tiangong station in 2021, which will continue being built and expanded in the years to come.

Losing the ISS would just be a loss for science, but for economic opportunity. On the back of its findings, dozens of new ideas with commercial potential have been devised — from space factories that can manufacture everything from carbon nanotubes to fiberglass cables to organic tissue to floating utility stations that could provide Earth with crucial resources. Luckily, the United States is focused on encouraging a number of commercial players to help send up space stations of their own. Axiom Space is working on building a proprietary science laboratory in space, which can act as a successor to the ISS. NASA has also tapped Nanoracks, Northrop Grumman, and Jeff Bezos’ Blue Origin, which is in the throws of designing an Orbital Reef, to help establish commercial destinations in space too.


After decades of disappointment, we’re dreaming about space once again. A more robust presence in space will have tons of implications on the rest of us back home, from the emergence of new industries like manufacturing in microgravity to the destruction of old terrestrial ones, like cable monopolies. Of course, the further out we go, the more problems we’ll need to contend with from clearing up space debris to figuring out how to better service existing spacecraft. With technology, however, problems often become the source of opportunity. From where we stand today, space is teeming with opportunity. 

Below are some of the notable companies working on the frontier of satellite technology:

York Space Systems is building standardized satellite platforms that can be customized to suit specific mission requirements.

Blue Canyon Technologies, a subsidiary of Raytheon Technologies, is providing turnkey solutions for small satellites. 

Apex Space is aiming to transform satellite bus manufacturing by making it faster and more affordable. 

Varda Space Industries are building capsule-satellites capable of manufacturing products in microgravity.

Umbra developed a fleet of micro-satellites capable of producing high-resolution synthetic aperture radar (SAR) images of Earth.

Capella Space offers synthetic aperture radar (SAR) images of Earth to customers including clients in government and defense sectors.

Axiom Space is working on building and eventually operating the world’s first commercial space station. Its founder, Michael Suffredini previously served as the manager of NASA’s ISS program. 

Privateer is building the data infrastructure and services required to manage the assets in the growing space economy.

A deep dive into the frontiers of the burgeoning space economy.

The Satellite Renaissance

Batteries are some of the most overlooked technologies. Today, they’re hidden behind the screens or user interfaces of the machines they power which obscure the essential role they played in electrifying human civilization.

Many forget that batteries were our first source of electricity, before we ever had access to or plans to build massive electrical infrastructure. In fact, we had batteries before we even knew what electricity was. Since the early 1800s, when the first battery was invented, the ability to store more power for longer has transformed our world. Better batteries made things like smartphones and laptops possible, not to mention other achievements like long-range, ultra fast electric vehicles. 

Fully electrifying our infrastructure through renewable sources is one of the proposed solutions to reducing our reliance on fossil fuels. Unfortunately, in these debates, the role of batteries is often glossed over entirely. Batteries pose something of an uncomfortable conundrum here, since they too are made of finite materials and are not easily recyclable. The reality is that better battery technology might ultimately become the main bottleneck in achieving sustainable and abundant energy, which is why efforts to study them and eliminate their current limitations are so important. 

The best battery on the market today is the lithium-ion battery. Developed in the late 1990s by Sony and the Asahi Chemical Company, lithium-ion batteries have become the ubiquitous de facto power source of electrical devices. More so than any other battery invented before them, they are incredibly energy dense, rechargeable with long life cycles, and since their invention have been falling precipitously in price. This suite of convenience characteristics is almost single-handedly what enabled our generation to have access to portable and powerful electronic devices.

Nothing shows the full capability of lithium-ion batteries better than today’s electric vehicles. When the Tesla Roadster was unveiled in 2006 by Martin Eberhard, it was the first electric vehicle on the market that used lithium-ion batteries. At the time, other electric or hybrid brands like the Prius were using a nickel-metal hydride chemistry which had shorter life cycles and weighed more. Tesla’s initial design packed 6,831 individual lithium-ion battery cells — which look like regular Duracells on the outside — together to produce a 900-pound system that could bring the car from 0-60 mph in just four seconds. Today, using similar lithium-ion technology, Teslas can accelerate from 0-60 in just 2.3 seconds — making them the fastest cars in the world.

And there’s still more juice to be squeezed out of lithium-ion technology. Panasonic is one of the most advanced battery manufacturers in the world and has been cooperating with Tesla on battery design basically since Tesla’s inception. They’re now inches away from releasing a still more energy dense lithium-ion cell, which will be able to store 900 watt-hours per liter over the current top-of-the-line battery, which has a maximum energy density of 750 watt hours per liter.

While this would be an impressive improvement, there is a sense that lithium-ion batteries in their current design are entering a world of incremental gains.

Ultimately, batteries are evaluated on five characteristics:

Energy density — a measure of how much total charge a battery can store

Power density — a measure of how the maximum power a battery can output at any given time

Safety — how easily batteries can short circuit, or toxic their internal chemistry might be

Cycle life — how many times you can recharge your battery before it becomes unusable, and

Each battery chemistry has its own particular mix of strengths within this matrix. The ideal battery, of course, would rank highly across all of them — and in that regard, lithium-ion comes very close. It’s incredibly energy dense, relatively safe, has a long cycle life, and is reasonably cheap to produce. 

However, as we start placing more demands on our batteries, even lithium-ion batteries aren’t cutting it. As Martin Eberhard wrote, “One of the most difficult challenges in battery design is increasing energy density while also maximizing battery lifespan. Li-ion chemistries have achieved better combinations of these parameters than anything that has come before. Yet there is still a tradeoff between energy and life, even within the family of Li-ion.”

There’s still a long way to go to develop batteries which can cost-efficiently store the requisite amount of energy generated from renewables needed to fully electrify our infrastructure. For one thing, many renewable energy sources like solar panels and wind turbines generate energy at sporadic times in line with the weather. Without batteries that can store this generated energy for use at crucial moments, the energy that’s generated just goes to waste. According to Shirley Meng, a materials scientist at the University of Chicago studying batteries, “in the last century, about 60% of the energy we produced went to waste.”

Another outstanding issue is that even top-of-the-line lithium batteries are still not cost-competitive with gas-powered vehicle, furthermore, relying on just one battery chemistry, which depends on a dwindling supply of cobalt and lithium, is beginning to push prices up on batteries leading to a reversal of the the impressive price decline we’ve seen over the past few decades.

A deep dive into the science of battery power.

In The Media

Industrializing Carbon Removal

The Acceleration of Artificial Intelligence

Welcoming David Oko

Welcoming Skye Nguyen and Akshay Chandramouli

The Satellite Renaissance

The Challenge of Building Better Batteries