Meta is desperate to catch up with rivals in the generative AI space, spending billions of dollars on its own AI efforts. Some of these billions of dollars will go toward hiring AI researchers. But an even bigger chunk is going into developing the hardware, specifically the chips to run and train Meta's AI models.
A day after Intel announced its latest AI accelerator hardware, Meta today announced the latest in its chip development efforts. Dubbed the “next generation” Meta Training and Inference Accelerator (MTIA), the successor to last year's MTIA v1, the chip runs models such as ranking and recommending display ads on meta properties (like Facebook). Masu.
The next generation MTIA is 5nm compared to MTIA v1 which is built on 7nm process. (In chip manufacturing, a “process” refers to the size of the smallest component that can be built on a chip.) Next-generation MTIAs are physically larger designs, packing more processing cores than previous generations. I am. It also consumes 90W more power than 25W, but also has more internal memory (128MB compared to 64MB) and runs at a higher average clock speed (800MHz to 1.35GHz).
According to Meta, next-generation MTIA is currently live in 16 datacenter regions and delivers up to 3x better overall performance compared to MTIA v1. While the “3x” claim may sound a little vague, it's not wrong. We thought so too. But Mehta only voluntarily claims that this figure is the result of testing the performance of “four major models” across both chips.
“Because we control the entire stack, we can achieve higher efficiency compared to commercially available GPUs,” Meta wrote in a blog post shared with TechCrunch.
Meta's hardware showcase comes just 24 hours after the company held a press conference about various generative AI initiatives currently underway, which is unusual for several reasons.
First, Meta clarifies in a blog post that it is not currently using next-generation MTIA for generative AI training workloads, but the company has “several programs underway” to consider it. It claims to be. Second, Meta acknowledges that next-generation MTIA will not replace GPUs for model execution and training, but will complement them.
Reading between the lines, the meta moves slowly – perhaps slower than you think.
Meta's AI team is almost certainly under pressure to cut costs. The company plans to spend an estimated $18 billion by the end of 2024 on GPUs for training and running generative AI models, and the cost of training state-of-the-art generative models runs into the tens of millions of dollars, so the company expects to clothing is an attractive alternative.
And while Meta's hardware is lagging behind, I suspect its rivals are moving forward, much to the surprise of Meta's leadership.
Google this week made TPU v5p, its fifth generation custom chip for training AI models, generally available to Google Cloud customers and announced Axion, its first purpose-built chip for running models. Amazon has several custom AI chip families under its umbrella. And Microsoft entered the fray last year with the Azure Maia AI Accelerator and Azure Cobalt 100 CPU.
Meta said in a blog post that it took less than nine months to “move from first silicon to production model” for the next generation MTIA. To be fair, this is shorter than the typical period between Google TPUs. But Meta has a lot of catching up to do before it can establish some independence from third-party GPUs and compete against stiff competition.