Chipmaker Nvidia generated $30 billion in revenue in the last fiscal quarter, driven largely by the AI industry's insatiable demand for GPUs. GPUs are essential for training and running AI models. These contain thousands of cores working in parallel to rapidly execute the linear algebraic equations that scaffold the model.
Interest in AI remains high, and Nvidia's GPUs have become the chips of choice among AI players of all sizes. But TensorWave, a company founded late last year, is going against the grain by launching a cloud that only provides access to Nvidia rival AMD's hardware for AI workloads.
“We realized that there was an unhealthy monopoly at work that was starving end users of computing access and stifling innovation in the AI space,” said TensorWave CEO and Co-Founder. Darrick Horton told TechCrunch. “Driven by our desire to democratize AI, we set out to provide viable alternatives and restore competition and choice.”
winding road
Pickleball originally brought together Horton and two other TensorWave co-founders, Jeff Tatarchuk and Piotr Tomasik. At least, that's what got the ball rolling (excuse the pun).
After a game one day, Tomasik and Tatarczyk, close friends and longtime pickleball doubles partners, invited Horton, Tatarczyk's former colleague, to their favorite Las Vegas watering hole.
“As the conversation progressed, we talked about how GPUs' monopolistic control of computing power is leading to supply constraints,” Horton said. “This realization led to the creation of TensorWave.”
The three partners didn't just know each other through pickleball.
Tatarchuk co-founded cloud vendor VMAccel with Horton before selling another startup, CRM developer Lets Rolo, to digital identity company LifeKey. Horton, who holds bachelor's degrees in mechanical engineering and physics, previously worked at Lockheed Martin's Skunk Works R&D division before co-founding VaultMiner Technologies, a cryptocurrency mining company and parent company of VMAccel.
As for Mr. Tomasik, he co-founded Lets Rolo with Mr. Tatarcuk. (Tomasic is also the co-founder of influencer marketing site Influencial, which was acquired by French PR firm Publicis in July for $500 million.)
So how did three entrepreneurs with little knowledge of the hyperscaler landscape think they could compete with the giants of the AI industry? Basically, tenacity.
“We believed we could solve the GPU supply problem,” Horton says.
Vegas Co., Ltd.
TensorWave is headquartered in Las Vegas, an unusual city choice for a cloud infrastructure startup. But Horton said the team likes the odds.
“We saw the potential for a thriving technology and startup ecosystem in Las Vegas,” he said.
That prediction isn't completely off the mark. According to data from Dealroom.co, Las Vegas is home to more than 600 startups, employing more than 11,000 people, and attracted more than $4 billion in investment in 2022.
Energy costs and overhead costs are also lower in Las Vegas than in many major U.S. cities. And Tomasik and Tatarcuk both have close ties to the city's VC community.
Mr. Tomasik previously served as GP at Vegas-based seed fund 1864 Fund and currently works at nonprofit accelerators StartUp Vegas and Vegas Tech Ventures. (Oddly, the Vegas Tech Ventures site had a 404 error on the page listing its partners and portfolio companies. A spokesperson said this was a technical error and would be fixed.) ) Tatarchuk is an angel investor at Las Vegas incubator Fruition Lab. It began as an unusual Christian religious group.
These connections, along with Horton's, helped bootstrap TensorWave as one of the first clouds to market with AMD Instinct MI300X instances for AI workloads. TensorWave provides setups with dedicated storage and high-speed interconnects upon request, and rents GPU capacity by the hour. Additionally, a minimum 6-month contract is required.
“We have a good relationship in the whole cloud space,” Horton said. “We see ourselves as complementary, offering additional AI-specific computing at a competitive price/performance.”
AMD forward
The market for startups building low-cost, on-demand GPU-powered clouds for AI is booming.
CoreWeave, a GPU infrastructure provider that started as a crypto mining business, recently raised $1.1 billion in new capital (and $7.5 billion in debt) and signed a multibillion-dollar capacity deal with Microsoft. Lambda Institute secured up to $500 million in special purpose funding in early April and is reportedly seeking to raise an additional $800 million. Voltage Park, a nonprofit backed by cryptocurrency billionaire Jed McCaleb, announced last October that it would invest $500 million in GPU-powered data centers. And Together AI, a cloud GPU host that also does generative AI research, raised $106 million in a Salesforce-led round in March.
So how does TensorWave hope to compete?
First, about the price. Horton points out that the MI300X is significantly cheaper than the H100, currently Nvidia's most popular GPU for AI workloads, allowing TensorWave to pass cost savings on to customers. He did not reveal the exact instance price of TensorWave. But to beat the competitive H100 plan, you'll need to spend less than about $2.50 per hour. Although this is difficult, it is not inconceivable.
“Prices range from approximately $1 to $10 per hour, depending on the bespoke requirements of the workload and the GPU configuration selected,” Horton said. “Due to nondisclosure agreements, we are unable to share details about the per-instance costs that TensorWave incurs.”
Second, it's about performance. Horton cites benchmarks that show the MI300X outperforms the H100 when it comes to running (not training) AI models, particularly text generation models like Meta's Llama 2 (other reviews show that It has been suggested that the benefits may be workload dependent).
Horton's claims seem to have some credence, given the tech industry's proponents and interest in MI300X. Meta announced in December that it would use MI300X chips for use cases such as running the Meta AI assistant, and OpenAI, the maker of ChatGPT, says it plans to support MI300X in its developer tools.
competition
Companies betting on AMD's AI chips range from startups like Lamini and Nscale to larger, more established cloud providers like Azure and Oracle. (Google Cloud and AWS remain unconvinced by AMD's competitiveness.)
Currently working in favor of all of these vendors is the continued Nvidia GPU shortage and delays in Nvidia's upcoming Blackwell chips. But the shortage could soon ease as production of critical chip components, particularly memory, ramps up. And this could allow NVIDIA to expand shipments of H100's successor, H200, which boasts dramatically improved performance.
Another existential dilemma for cloud startups betting on AMD hardware is filling the competitive moat that Nvidia has built around its AI chips. Nvidia's development software is perceived to be more mature and easier to use than AMD's, and is widely deployed. AMD CEO Lisa Su herself admits that adopting AMD “takes effort.”
In the distant future, price competition may become more difficult as hyperscalers increase their investments in custom hardware to run and train their models. Google offers TPU. Microsoft recently announced two custom chips: Azure Maia and Azure Cobalt. AWS has Trainium, Inferentia, and Graviton.
“Developers are looking for alternatives that can effectively handle AI workloads, especially with increasing memory and performance demands and ongoing production issues that cause delays, and AMD We will maintain our dominance even longer and play a key role in the democratization of computing in the age of AI,” Horton said.
early demand
TensorWave began onboarding customers in preview later this spring. But Horton says it's already generating $3 million in annual recurring revenue. He expects that amount to reach $25 million by the end of the year, an eight-fold jump, as TensorWave's capacity increases to 20,000 MI300X.
Assuming $15,000 per GPU, 20,000 MI300Xs equates to a $300 million investment. Still, Horton claims that TensorWave's burn rate is “well within sustainable levels.” TensorWave previously told The Register that it plans to use its GPUs as collateral for a large debt financing round. This is the approach taken by other data center operators, including CoreWeave. Horton says that's still the plan.
“This reflects the strength of our financial health,” he continued. “We are strategically positioned to weather potential headwinds by delivering value where it is needed most.”
I asked Horton how many customers TensorWave currently has. He declined to answer, citing “confidentiality,” but highlighted TensorWave's announced partnerships with networking backbone provider Edgecore Networks and AI inference startup MK1, founded by former Neuralink engineers.
“We are rapidly expanding capacity with multiple nodes available and are continually expanding capacity to meet the growing demand in our pipeline,” Horton said. , added that TensorWave plans to introduce AMD's next-generation MI325X GPU. It will be released in Q4 2024 and online as early as November or December.
Investors seem satisfied with TensorWave's growth trajectory so far. Nexus vice president revealed on Wednesday that he led a $43 million round in the company, with participation also from Maverick Capital, StartupNV, Translink Capital, and AMD Ventures.
TensorWave's first tranche values the startup at $100 million post-money.
“AMD Ventures shares TensorWave's vision to transform AI computing infrastructure,” AMD Ventures SVP Matthew Hein said in a statement. “The company's AMD Instinct MI300X deployment and ability to offer public instances to AI customers and developers has positioned the company as an early competitor in the AI space, and we are excited about this latest round of funding. We are excited to support their growth throughout the round.”