Distribution, an AI testing platform founded by former Intel AI software director Scott Clark, has closed a $19 million Series A funding round led by Two Sigma Ventures.
Clark said Distribution was inspired by AI testing problems he encountered while applying AI at Intel and before that while working at Yelp as the software lead for the company's ad targeting division. It was something.
“As the value of AI applications continues to grow, so too does the operational risk,” he told TechCrunch. “AI product teams use our platform to proactively and continuously detect, understand, and address AI risks before they pose a risk to production environments.”
Clark joined Intel through an acquisition.
In 2020, Intel acquired SigOpt, a model experimentation and management platform co-founded by Clark. Clark will remain in the position and will be appointed vice president and GM of Intel's AI and Supercomputing Software Group in 2022.
At Intel, Clark and his team say they were frequently held back by AI monitoring and observability issues.
Clark pointed out that AI is non-deterministic, meaning it produces different outputs given the same data. Additionally, AI models have many dependencies (such as software infrastructure and training data), and identifying bugs in AI systems can be like looking for a needle in a haystack.
According to a 2024 Rand Corporation study, more than 80% of AI projects fail. Gartner research finds that generative AI is particularly challenging for enterprises, with one-third of deployments predicted to be abandoned by 2026.
“We need to create statistical tests on the distribution of many data properties,” Clark says. “AI must be continuously and adaptively tested throughout its lifecycle to capture changes in behavior.”
Clark created Distributional to take some of the abstraction out of this AI audit work, leveraging technology he and his team at SigOpt developed while working with enterprise customers. Distributional can automatically create statistical tests for your AI models and apps to your specifications and organize the results of these tests into dashboards.
From that dashboard, distribution users can collaborate on test “repositories”, prioritize failed tests, and retune tests as needed. You can deploy your entire environment on-premises (although Distribution also offers management plans) and integrate with popular alerting and database tools.
“We provide visibility across the organization into when, what, and how AI applications are tested and how they change over time,” Clark said. “Also shareable templates, configurations, filters and tags.”
AI is certainly an untamable beast. Even the best AI labs have poor risk management. Platforms like Distributional can reduce the burden of testing and perhaps even help companies achieve ROI.
At least, that's what Clark says.
“Identifying AI risks can be difficult when there are many instability, inaccuracies, or other potential challenges,” he said. “If teams are unable to properly conduct AI testing, there is a risk that AI applications will not go into production. There is a sex.”
Distributional is not the first company to bring technology to market to investigate and analyze the trustworthiness of AI. There are many AI experimentation solutions, including Kolena, Prolific, Giskard, and Patronus. Big technology companies like Google Cloud, AWS, and Azure also offer model evaluation tools.
So why do customers choose distribution?
Clark claims that Distributional, which is on the verge of commercializing a suite of products, offers a “better” experience than other companies. Distributional is responsible for client installation, implementation, and integration, and provides AI testing troubleshooting (for a fee).
“Monitoring tools often focus on higher-level metrics or specific instances of outliers, providing a limited sense of consistency but not providing insight into broader application behavior. No,” Clark said. “The goal of testing at Distribution is to help teams arrive at a definition of the desired behavior of an AI application, ensure it behaves as expected throughout production and development, detect when this behavior changes, and identify what evolves. or to be able to figure out what needs to evolve and be modified to reach a steady state again. ”
With new Series A funding, Distributional plans to expand its technology team with a focus on the UI and AI research engineering aspects. Clark said he expects the company's workforce to grow to 35 by the end of the year as Distribution embarks on its first wave of enterprise deployments.
“We have secured significant funding after just one year of existence and are well positioned to take advantage of this significant opportunity over the next few years, even as our team grows,” Clark added. .
Andreessen Horowitz, Operator Collective, Oregon Venture Fund, Essence VC, and Alumni Ventures also participated in Distributional's Series A. To date, the San Francisco-based startup has raised $30 million.