Groq, a startup developing chips that run generative AI models faster than traditional processors, said Monday it had raised $640 million in a new funding round led by BlackRock, with participation from Neuberger Berman, Type One Ventures, Cisco, KDDI and Samsung Catalyst Fund.
The raise brings Groq's total funding to over $1 billion and values the company at $2.8 billion. This is a major win for Groq, which was reportedly initially seeking to raise $300 million at a slightly lower valuation ($2.5 billion). This is more than double the valuation (about $1 billion) the company was valued at when it raised $300 million in a round led by Tiger Global Management and D1 Capital Partners in April 2021.
Groq announced today that Yann LeCun, Meta's chief AI scientist, will serve as a technical advisor to Groq, while Stuart Pang, former head of Intel's foundry business and former CIO of HP, will join the startup as chief operating officer. LeCun's appointment is a bit surprising given Meta's investment in its own AI chips, but it certainly gives Groq a powerful ally in a fierce competitive arena.
Groq, which emerged from stealth in 2016, is developing what it calls an LPU (Language Processing Unit) inference engine. The company claims that its LPU can run existing generative AI models, which are similar in architecture to OpenAI's ChatGPT and GPT-4o, at 10 times faster and with 1/10th the energy.
Groq CEO Jonathan Ross became famous for helping to invent the Tensor Processing Unit (TPU), Google's custom AI accelerator chip used to train and run models. Ross is an entrepreneur who co-founded Groq about a decade ago, teaming up with Douglas Wightman, a former engineer at Google parent Alphabet's X Moonshot Lab.
Groq offers “open” models like Meta's Llama 3.1 family, Google's Gemma, OpenAI's Whisper, and Mistral's Mixtral, as well as an LPU-powered developer platform called GroqCloud that offers APIs that allow customers to use the company's chips in their cloud instances. (Groq also hosts GroqChat, a playground for AI-powered chatbots that launched late last year.) As of July, GroqCloud had more than 356,000 developers. Groq says it will use part of the proceeds from this round to expand capacity and add new models and features.
“Many of these developers are from large companies,” Groq COO Stuart Pann told TechCrunch. “We estimate that over 75% of Fortune 100 companies are participating.”
A close-up of Groq's LPU, designed to accelerate specific AI workloads. Image credit: Groq
As the generative AI boom continues, Groq faces increasing competition from both rival AI chip startups and Nvidia, a powerful incumbent in the AI hardware field.
Nvidia controls roughly 70% to 95% of the market for AI chips used to train and deploy generative AI models, and it is taking aggressive steps to maintain its dominance.
Nvidia has committed to releasing a new AI chip architecture every year, rather than every two years as in the past, and is reportedly creating a new business unit focused on designing custom chips for cloud computing companies and other businesses, including AI hardware.
Besides Nvidia, Groq is competing with Amazon, Google and Microsoft, all of which offer or soon will offer custom chips for AI workloads in the cloud. Amazon offers its Trainium, Inferentia and Graviton processors available through AWS. Google Cloud customers can use the aforementioned TPUs and, in the future, Google's Axion chips. Microsoft recently released preview Azure instances of its Cobalt 100 CPUs, with Maia 100 AI Accelerator instances expected to arrive in the coming months.
Some analysts say the AI chip market could hit $400 billion in annual sales over the next five years. Groq may also be looking at rivals from Arm, Intel, AMD and a growing number of new startups. Arm and AMD in particular are seeing booming AI chip businesses thanks to capital spending surges from cloud vendors to meet the capacity demands of generative AI.
D-Matrix raised $110 million late last year to commercialize what it describes as a first-of-its-kind inference computing platform. In June, Etched emerged from stealth, spending $120 million on custom-built processors to speed up Transformer, the current mainstream AI generative model architecture. SoftBank's Masayoshi Son is reportedly looking to raise $100 billion for a chip venture to rival Nvidia, and OpenAI is said to be in talks with investment firms to launch an AI chip manufacturing initiative.
Groq is investing heavily in corporate and government outreach to carve out a niche market.
In March, Groq acquired Palo Alto-based Definitive Intelligence, which offers a range of AI solutions for business, and created a new business unit called Groq Systems, whose responsibilities include serving U.S. government agencies, independent nations and other organizations that want to add Groq chips to existing data centers or build new ones with Groq processors.
More recently, Groq partnered with government IT contractor Carahsoft to sell its solutions to public sector customers through Carahsoft's reseller partners, and the startup has a letter of intent to install tens of thousands of LPUs in European company Earth Wind & Power's Norwegian data centers.
Groq is also working with Saudi Arabian consulting firm Aramco Digital to install LPUs in future data centers in the Middle East.
Mountain View, California-based Groq is simultaneously building customer relationships and developing its next-generation chips. Last August, the company announced it had contracted with semiconductor company Global Foundries to manufacture its 4nm LPU, which is expected to deliver better performance and efficiency than Groq's first-generation 13nm chips.
Groq said it plans to deploy more than 108,000 LPUs by the end of the first quarter of 2025.