Since its release Technology leaders are rushing to follow the ChatGPT conversation. Everywhere you go, another company is touting its pioneering AI capabilities. But real business value comes not just from using hot technology, but from delivering product features that matter to users.
By starting with fundamental principles of what users want from a product, building AI capabilities that support that vision, and then measuring adoption to see if it's hitting the mark, we Achieved a 10x increase in profitability for engineering work using .
Our first AI product feature did not align with this idea and unfortunately took a month to reach 0.5% adoption among repeat customers. After refocusing on the core principles of what users need from our product, we developed an “AI as an agent” approach, shipped new AI features, and in our first week His adoption rate skyrocketed to 5%. This formula for success with AI can be applied to almost any software product.
Waste of hype
Many startups like ours are often seduced by the lure of integrating the latest technology without a clear strategy. So after the groundbreaking release of various incarnations of Generative Pre-Trained Transformer (GPT) models from OpenAI, we started looking for ways to use Large-Scale Language Model (LLM) AI technology in our products. Soon, we secured a position to jump on the hype train with new AI-driven elements in production.
This first AI feature was a small summary feature that used GPT to write a short paragraph describing each file a user uploaded to the product. It gave us a buzz and created some marketing content, but it had no meaningful impact on the user experience.
Many startups are often seduced by the lure of integrating the latest technology without a clear strategy.
We knew this because none of our key metrics showed any noticeable changes. Only 0.5% of repeat customers manipulated the description in his first month. Furthermore, there was no improvement in user activation and no change in the pace of user sign-ups.
Looking at it from a broader perspective, it was clear that this feature would never move those metrics. The core value proposition of our products is big data analytics and using data to understand the world.
Generating a few words about an uploaded file does not provide any significant analytical insight. That means you can't do much to help the user. In our rush to deliver something AI-related, we missed out on delivering real value.
Success with AI as an agent: 10x more revenue
The AI approach that has brought us success is the “AI as an agent” principle, which allows users to interact with data in our products through natural language. This recipe can be applied to almost any software product built on API calls.
After the first AI feature, we were checking the boxes, but we weren't satisfied because we knew we could do more for our users. So we did what software engineers have been doing since the invention of programming languages: gather together at hackathons. Starting with this hackathon, we implemented an AI agent that operates on behalf of users.
Agents use their own products by making API calls to the same API endpoints that the web front end calls. It builds API calls based on natural language conversations with the user and attempts to do what the user has requested. The agent's actions are manifested in her web user interface as a result of the API call, as if the user had performed the action herself.