Nvidia reported Wednesday that it made more than $19 billion in net income last quarter, but that did little to reassure investors that the company's rapid growth will continue. Analysts asked CEO Jensen Huang on the earnings call about what would happen to Nvidia if technology companies started using new methods to improve their AI models.
The methodology behind OpenAI's o1 model, or “test-time scaling,” has been much talked about. The idea is that if you give an AI model more time and computational power to “think” about a question, it will give you a better answer. Specifically, more compute is added to the AI inference phase (everything that happens after the user presses Enter at the prompt).
Nvidia's CEO was asked if he sees AI model developers moving to these new techniques, and how Nvidia's older chips will work for AI inference.
Huang told investors that o1 and broader test time scaling could play a bigger role in Nvidia's business going forward, calling it “one of our most exciting developments” and ” “New Scaling Laws”. Huang did his best to assure investors that Nvidia is well-positioned for change.
The Nvidia CEO's remarks echo what Microsoft CEO Satya Nadella said on stage at a Microsoft event on Tuesday. o1 represents a new way for the AI industry to improve its models.
This is a big problem because the chip industry is focused on AI inference. Nvidia's chips are the gold standard for training AI models, but there are also a number of well-funded startups developing ultra-fast AI inference chips, including Groq and Cerebras. It could be a more competitive field for Nvidia.
Despite recent reports that improvements in generative models are slowing, Huang told analysts that AI model developers are still improving their models by adding more compute and data in the pre-training phase. He said he is improving.
Anthropic CEO Dario Amodei also said in an on-stage interview Wednesday at the Cerebral Valley Summit in San Francisco that he doesn't see a slowdown in model development.
“The scaling of the underlying model pre-training remains intact and continues,” Huang said Wednesday. “You know, this is a law of thumb, not a fundamental law of physics, but there is evidence that it continues to grow. But what we're learning is that it's not enough.”
That's certainly what NVIDIA investors wanted to hear, as the chipmaker's stock soared more than 180% in 2024 on sales of AI chips used by OpenAI, Google, and Meta to train their models. But Andreessen Horowitz partners and several other AI executives have previously said these techniques are already starting to become less profitable.
Huang pointed out that most of Nvidia's computing workloads today are about pre-training AI models rather than inference, but he thinks that's rather where the AI world is today. He said that one day there will simply be more people running AI models, and more AI inference will be made. Huang said Nvidia is currently the world's largest inference platform, and the company's scale and reliability give it a significant advantage compared to startups.
“Our hope and dream is that one day the world will be doing a lot of inference, and that's when AI will really be successful,” Huang said. “We all know that if we innovate on CUDA and Nvidia architecture, we can innovate faster, and we know that everything should work.”