Meta announced Llama 3.3 70B, the latest addition to the Llama family of generative AI models.
Ahmad Al-Dahle, VP of Generated AI at Meta, said in a post on Then he said.
“By leveraging the latest advances in post-training techniques… this model improves core performance at significantly lower cost,” Aldahl wrote.
Al-Dahle said Llama 3.3 70B outperformed Google's Gemini 1.5 Pro, OpenAI's GPT-4o, and Amazon's newly released Nova Pro on a number of industry benchmarks, including MMLU, which evaluates a model's language understanding ability. A graph showing this has been published. A Meta spokesperson said in an email that the model should result in improvements in areas such as math, general knowledge, following directions and using apps.
Introducing Llama 3.3 – a new 70B model that offers the performance of the 405B model, but is easier and more cost effective to run. This model improves core performance by leveraging the latest advances in post-training techniques, including optimizing online settings… pic.twitter.com/6oQ7b3Yuzc
— Ahmad Al Dahle (@Ahmad_Al_Dahle) December 6, 2024
Llama 3.3 70B can be downloaded from the AI development platform Hugging Face and other sources including the official Llama website. This is Meta's latest effort to dominate the AI field with an “open” model that can be used and commercialized in a variety of applications. application.
Meta conventions restrict how certain developers can use Llama models. Platforms with more than 700 million monthly users must request a special license. But for many people, it doesn't matter that the Llama model is not “open” in the strict sense of the word. According to Meta, Llama has been downloaded more than 650 million times.
Meta also uses Llama internally. According to Meta CEO Mark Zuckerberg, the company's AI assistant, Meta AI, is fully powered by the Llama model and currently has nearly 600 million monthly active users. Zuckerberg claims MetaAI is on track to become the world's most used AI assistant.
For Mehta, Rama's open nature was both a blessing and a curse. In November, a report was published saying that Chinese military researchers had developed a defense chatbot using the Llama model. Meta responded by offering its Llama model to U.S. defense contractors.
Meta also expressed concerns about its ability to comply with the AI Act, the EU law that sets the regulatory framework for AI, saying the implementation of the law would be “too unpredictable” for its open release strategy. A relevant issue for the company is the AI training provisions of the EU's privacy law, GDPR. Meta trains AI models based on public data from Instagram and Facebook users who have not opted out (data that is covered by GDPR guarantees in Europe).
Earlier this year, EU regulators asked Meta to halt training on European user data while they assessed the company's GDPR compliance. Mr Mehta did not relent, but at the same time supported an open letter calling for a “modern interpretation” of the GDPR that does not “reject progress”.
Meta is not immune to the technical challenges facing other AI labs, ramping up its computing infrastructure to train and serve future generations of llamas. The company announced Wednesday that it will build a $10 billion AI data center in Louisiana, the largest ever built by Meta.
During Meta's Q4 earnings call in August, Zuckerberg said that training Llama 4, the next major set of Llama models, requires 10 times the amount of compute needed to train Llama 3. He said it would be. Meta is a GPU for model development that rivals competing resources such as xAI, which has raised over 100,000 Nvidia clusters.
Training generative AI models is expensive. Meta's capital expenditures increased nearly 33% to $8.5 billion in the second quarter of 2024 from $6.4 billion in the same period last year, driven by investments in servers, data centers and network infrastructure.