A startup called Letta has emerged from stealth with technology that helps AI models remember users and their conversations. Founded at UC Berkeley's famed Labs startup factory, the company also announced a $10 million seed round led by Felicis' Astasia Myers, bringing its post-raise valuation to $70 million.
Letta is also backed by some notable angel investors in the AI industry, including Jeff Dean from Google, Clem Delangue from Hugging Face, Cristóbal Valenzuela from Runway, and Robert Nishihara from Anyscale.
Founded by Berkeley PhD students Sarah Woodards and Charles Packer, the company is a highly anticipated AI startup, as it is a subsidiary of Berkeley's Sky Computing Lab and the commercial arm of the popular MemGPT open source project.
Led by distinguished professor and Databricks co-founder Ion Stoica, Berkeley's Sky Computing Lab is the successor to RISELab and AMPLab, which spawned companies like Anyscale, Databricks, and SiFive. Sky Lab in particular has spawned a number of popular open source large-scale language model (LLM) projects, including Gorilla LLM, vLLM, and SGLang, a structured language for LLMs.
“In a short space of a year, so many projects came out of the lab, and we just had people sitting next to us,” Woodards explains, “so it was a really great time.”
MemGPT is one such project, and it’s so popular that it’s actually been making waves even before it’s been released.
“Someone else got there first,” Packer told TechCrunch. The founders had planned to post the white paper on Thursday, October 12, 2023, and then release a more detailed paper and code on GitHub the following Monday. However, someone found the paper and posted it to Hacker News on Sunday, and “it was on Hacker News before we had a chance to properly publish the code, publish the paper, or make a tweet thread,” Packer said.
The reason for the excitement is that MemGPT mitigates a pernicious problem with LLM: In their native form, models like ChatGPT are stateless and don't store historical data in long-term memory. This is a problem for AI apps that rely on getting to know and learn from their users over time, from customer support bots to healthcare symptom tracker apps. MemGPT manages data and memory, enabling AI agents and chatbots to remember previous users and conversations.
Packer said the paper's posting was on the front page of Hacker News, a popular programmer site run by Y Combinator, for 48 hours. He spent the weekend and the next few days answering questions on the site and preparing the code for release. Once the project was available on GitHub, the link was trending again on Hacker News. It quickly led to interviews and tutorials on YouTube, a Medium post, and 11,000 stars and 1,200 forks on GitHub.
VC Felicis’ Myers also discovered Woodards and Packer after reading about MemGPT and quickly recognized the technology’s commercial potential.
“I saw the paper when it came out,” she told TechCrunch, and quickly reached out to the founders. “We had an investment thesis around AI agent infrastructure, and we recognized that data and memory management was a really key component of making conversational chatbots and AI agents effective.”
The founders virtually walked Sand Hill Road and had Zoom calls with venture capitalists before choosing the one that liked them best.
Stoica, in turn, introduced him to Dean, Nishihara and other prominent Silicon Valley angel investors. “A lot of the faculty at Berkeley have very extensive contacts because of their time at Berkeley,” Packer said, recalling how easy the angel investment process was. “They’re interested in projects that come out of this lab and are commercialized.”
Competition and the Threat of OpenAI o1
While MemGPT is already publicly available and in use, Letta's commercial version, Letta Cloud, isn't yet live. As of Monday, Letta was accepting beta user requests. Letta offers a hosted agent service, allowing developers to deploy and run stateful agents in the cloud. The agents are accessible through a REST API, a programming interface that allows state to be maintained. Letta Cloud stores the long-term data needed to do so. Letta also provides developer tools for building AI agents.
Woodards believes MemGPT has a wide range of applications. “I think the number one use case we see is essentially a highly personalized, highly engaging chatbot,” she says. But there are also cutting-edge applications like a chatbot for cancer patients, where patients can upload their medical history and share ongoing symptoms so the bot can learn over time and offer guidance.
It's worth noting that MemGPT isn't the only one working on this: LangChain is perhaps the best-known competitor and already offers commercial options, and the biggest model makers also offer AI agent creation tools, such as OpenAI's Assistants API.
Also, OpenAI's new o1 model may allow users to ignore the need for frozen state: it's a multi-step model, so it essentially needs to maintain some state to “think” and check facts before responding.
But Woodards, Packer and Myers see some key differences between what Letta is offering and what the £800 market leader, OpenAI, is doing. Letta claims to work with any AI model, and expects users to use many of them: OpenAI, Anthropic, Minstrel, and home-grown models. OpenAI's technology currently only works with OpenAI.
More importantly, Letta uses the open-source MemGPT, firmly entrenching itself on the open-source side of the FOSS vs. black-box LLM debate, arguing that open source is the better choice for AI application programmers.
“We're positioning ourselves as an open alternative to OpenAI,” Packer said, “and we think it's very hard to build really good AI applications if you can't see what's going on under the hood, especially when you're dealing with problems like hallucinations.”