OpenAI, the developer of ChatGPT, has announced its next major product release: a generative AI model codenamed Strawberry, but officially called OpenAI o1.
To be more precise, o1 is actually a family of models. There are two available now via ChatGPT and OpenAI's API: o1-preview and a smaller, cheaper model, o1 mini. You must be a ChatGPT Plus or Team subscriber to select them in ChatGPT. Enterprise and Edu users will have access early next week.
It's important to note that the o1 chatbot has fairly basic capabilities at the moment: unlike ChatGPT, o1 can't browse the web or analyze files (yet). And the model isn't cheap: in the API, o1-preview costs $15 per million tokens (roughly 750,000 words) input to the model, and $60 per million tokens generated by the model. By comparison, GPT-4o costs $5 per million input tokens and $15 per million output tokens.
OpenAI says it plans to provide access to o1-mini to all free ChatGPT users, but has not yet set a release date.
According to OpenAI, at least, o1 avoids some of the reasoning pitfalls that generative AI models typically fall into because it spends more time considering all parts of a command or question, effectively fact-checking it.
OpenAI says o1, which emerged from an internal project called Q*, is particularly good at solving math and programming-related challenges. But what makes the text-only o1 “feel” qualitatively different from other generative AI models is its ability to “think” before responding to a query.
When O1 is given additional time to “think,” it is able to reason holistically about a task – planning ahead and taking a series of actions over an extended period of time to arrive at an answer. This makes O1 well suited to tasks that require synthesizing the results of multiple subtasks, like finding confidential emails in a lawyer's inbox or brainstorming a product marketing strategy.
“O1 is trained through reinforcement learning to 'think' before reacting via a private thought chain,” Noam Brown, a research scientist at OpenAI, said in a series of tweets. “The longer it thinks, the better it performs on reasoning tasks. This opens up a new dimension of scaling.”
TechCrunch wasn't given a chance to test o1 before its debut, but the company aims to get hold of it as soon as possible, but according to someone who had access to it (Thomson Reuters VP Pablo Arredondo), o1 is better than OpenAI's previous models (such as GPT-4o) at things like analyzing court documents and identifying solutions to LSAT logic game problems.
“We saw that we were doing more substantive, multifaceted analysis,” Arredondo told TechCrunch. “Our automated tests were paying off for a wide range of simple tasks.”
In the qualifying exams for the International Mathematical Olympiad, a math competition for high school students, o1 solved 83% of the problems correctly, while GPT-4o only solved 13%, according to OpenAI. The company also said o1 should perform better on science and coding problems.
But there are drawbacks: o1 can be slower than other models for some queries; Arredondo says the model can take more than 10 seconds to answer a question. (Helpfully, the chatbot version of o1 lets you know its progress by displaying a label for the subtask that's currently being performed.)
Given the unpredictable nature of generative AI models, o1 will likely have other flaws and limitations (Brown acknowledges that it also fails at the game of tic-tac-toe, for example). These will no doubt become clear in time, and once we have a chance to test the model ourselves.
It would be disingenuous to fail to point out that OpenAI is not the only AI vendor investigating these types of inference methods to improve the facticity of their models: Google DeepMind researchers recently published research showing that by giving models more compute time and guidance on how to fulfill requests as they arise, they can significantly improve their performance without any additional tuning.
OpenAI may be first in with the o1, but assuming rivals follow suit quickly with comparable models, the real test for the company will be making the o1 affordable and widely available.