Today, Cerebras Systems is a public company that sells AI chips for inference to major companies such as OpenAI and AWS. The company held a massive IPO on Thursday, and both co-founders are billionaires, ending the week with a valuation of about $60 billion.
But in 2019, three years after its founding, it came dangerously close to bankruptcy and burned through a shocking amount of money. It was trying to solve a technical problem that no one in the semiconductor industry thought was possible.
“We were spending about $8 million a month,” founder and CEO Andrew Feldman told TechCrunch at the time. “At this point, we had spent nearly $200 million solving one technical problem.”
Every few weeks, Mr. Feldman was forced to make the painful walk of shame to the board of directors to report new failures and further losses.
But he had no choice. Without a solution, Cerebras would be dead anyway.
It was founded on an idea that was simple on paper. The microprocessor industry has spent more than 50 years making CPUs faster and cheaper by cramming more transistors onto silicon wafers and breaking the wafers into smaller pieces. But AI requires so much computing power that many chips have to be chained together and communicate with each other. The founders of Cerebras believed that turning an entire even larger wafer into one huge, powerful chip would work faster.
The problem is, no one has ever successfully done this before, AI or not. Integrating so many microscopic electronic components onto a larger but still thin surface created complex engineering problems.
Once Cerebras crossed the first hurdle of designing a megachip and manufacturing it with TSMC, the team hit a real roadblock.
They couldn't solve “packaging”. This includes all operations after the silicon itself has been manufactured. These include bonding to the motherboard, providing power, heating and cooling, and treating the pipes that transmit and receive data.
Cerebras' chip was “58 times larger. We were using 40 times more power than we've ever used before,” he said. There were no off-the-shelf heat sinks. There are no vendors. We have no manufacturing partners. The brightest minds in microprocessor engineering have been trying and failing for decades to build chips this big and even denser.
The Cerebras team was left with a lot of trial and error and a huge amount of cash, as they “destroyed a huge number of chips.” But without functional packaging, the chip was useless.
After thoroughly analyzing each failure, the team finally solved enough problems: how to cool it and how to move the data. In one case, we had to invent a unique machine that could bolt 40 screws at the same time to secure the wafer to the substrate without breaking it.
Feldman still remembers the day in July 2019 when, miraculously, everything went well.
Once the packaged chip was installed in a computer and turned on, the entire founding team (pictured below) “just stood there in the lab staring at it,” he said. “Watching a computer work is as exciting as watching paint dry. But then we looked at the flashing lights on the computer and were amazed that we had solved this.”
“That was one of the greatest moments of my life,” he said. This is significant, given that the same founding team had previously built the pioneering cloud server startup SeaMicro, which they sold to AMD in 2012 for $334 million.
Cerebras Systems founding team in 2015: Andrew Feldman, Gary Lauterbach, Michael James, Sean Lie, Jean-Philippe Fricker Image credit: Cerebras Systems
The day the chip finally worked also came nearly two years after OpenAI negotiated with Cerebras to acquire the chip, and Feldman confirmed to TechCrunch that it happened exactly as described in the published email.
Those negotiations broke down amid a growing dispute between OpenAI's founders. Several of them are angel investors in Cerebras.
Today, OpenAI is a customer and partner that has loaned Cerebras $1 billion, secured by warrants. The S-1 revealed that these stock acquisition rights conditionally grant OpenAI approximately 33 million Cerebras shares. (At Friday's closing price of $279, the 33 million shares are worth more than $9 billion.)
Interestingly, as part of its financing agreement, Cerebras also agreed not to sell products to certain competitors of OpenAI. Feldman declined to acknowledge that Anthropic was clearly involved. But he said the restrictions were temporary.
“Time is limited and it was designed to ensure that OpenAI has the capacity,” he said.
The truth was that Cerebras wasn't yet big enough to handle multiple burgeoning model manufacturers anyway. He likened the sale of AI computing power to an all-you-can-eat buffet. Rather than trying to cram in every potential customer, “we're just going to treat one part of the buffet, and we're going to get used to it before we attack the rest,” he said.
If you buy through links in our articles, we may earn a small commission. This does not affect editorial independence.