Executives and researchers leading Meta's AI efforts said they were using OpenAI while developing Llama 3, according to internal messages made public Tuesday by a court in one of the company's ongoing AI copyright cases, Kadri v. Mehta. He was said to be obsessed with breaking the GPT-4 model.
“Let's be honest… our goal has to be GPT-4,” Ahmad Aldar, vice president of generative AI at Meta, told Meta researcher Hugo Touvron in October 2023. I said this in a message to. “64,000 GPUs are coming! We need to build the frontier and learn how to win this race.”
Although Meta is releasing open AI models, the company's AI leaders are far more likely to beat out competitors such as Anthropic and OpenAI, which typically don't expose model weights and instead gate them behind APIs. was focused on. Meta executives and researchers held up Anthropic's Claude and OpenAI's GPT-4 as the gold standard to aspire to.
One of Meta's biggest competitors, French AI startup Mistral, was mentioned several times in internal messages, but the tone was negative.
“Mistral is peanuts to us,” Aldar said in a message. “We can do better,” he said later.
These days, technology companies are vying to compete with cutting-edge AI models, but these court filings raise questions about just how competitive Meta's AI leaders really were, and still are. It reveals what you can see. At several points in the message exchange, Meta's AI lead talked about how he was “very proactive” about getting the right data to train the llamas. At one point, one executive even said in a message to a colleague, “Literally all I care about is Rama 3.”
Prosecutors in the case allege that Meta executives occasionally cut corners in the frantic race to ship AI models, using copyrighted books to train them in the process. .
In his message, Touvron noted that the combination of datasets used for Llama 2 was “bad” and talked about how Meta could improve Llama 3 by better combining data sources. Touvron and Al-Dahle then discussed clarifying the paths to use. LibGen dataset. This includes copyrighted works of Cengage Learning, Macmillan Learning, McGraw Hill, and Pearson Education.
“Are there good datasets out there?[?]'' said Al Dar. “Is there anything you wanted to use but couldn't for some stupid reason?”
Meta CEO Mark Zuckerberg has previously said that the company aims to close the performance gap between Llama's AI models and closed models such as OpenAI and Google. Internal messages reveal that there is strong pressure within the company to do so.
“This year's Llama 3 competes with cutting-edge models and leads in some areas,” Zuckerberg said in a July 2024 letter, adding, “Starting next year, future Llama models will We expect it to be the most advanced model in the industry.”
When Meta finally released Llama 3 in April 2024, the open AI model competed with leading closed models from Google, OpenAI, and Anthropic, and outperformed Mistral's open option. But the data that Meta used to train its models – data that Zuckerberg reportedly gave the green light for use despite its copyright status – has come under scrutiny in several ongoing lawsuits. facing.