Mistral, a Microsoft-backed French AI startup valued at $6 billion, has released the first generative AI model for coding, called Codestral.
Codestral, like other code generation models, is designed to help developers write and work with code. According to a Mistral blog post, the model has been trained on more than 80 programming languages, including Python, Java, C++, and JavaScript. Codestral can complete coding features, write tests, and “fill in” partial code, as well as answer English questions about your codebase.
Mistral describes its model as “open,” but that's debatable. The startup's license prohibits the use of Codestral and its deliverables for any commercial activity. There is an exception for “development,” but even that comes with conditions. The license further explicitly prohibits “internal use by employees in the context of the company's business activities.”
The reason could be that Codestral was partially trained on copyrighted content — something Mistral didn't confirm or deny in its blog post, but it's not surprising, since there's evidence that the startup's previous training data sets contained copyrighted data.
Either way, Codestral may not be worth the trouble: You'll need a powerful PC to run its model, which has 22 billion parameters (parameters essentially define an AI model's skill for a problem like analyzing or generating text), and it beats the competition in some benchmarks (which, as you know, are unreliable), but not by a huge margin.
Image credit: Mistral
While Codestral is impractical for most developers and only incremental in terms of performance improvements, it is sure to stimulate debate about the wisdom of relying on code-generation models as programming assistants.
Developers are no doubt adopting generative AI tools for at least some coding tasks—44% of developers in a June 2023 Stack Overflow survey said they currently use AI tools in their development process, and 26% said they plan to do so in the near future—but these tools have clear flaws.
GitClear's analysis of more than 150 million lines of code committed to project repositories over the past few years has found that generative AI development tools are pushing more erroneous code into codebases. Security researchers have also warned that such tools can amplify existing bugs and security issues in software projects. A Purdue University study found that more than half of the answers OpenAI's ChatGPT returns to programming questions are incorrect.
That hasn't stopped companies like Mistral from trying to monetize (and capture mindshare) the model. This morning, Mistral released a hosted version of Codestral on its Le Chat conversational AI platform, in addition to a paid API. Mistral said it's also working to integrate Codestral into app frameworks and development environments, including LlamaIndex, LangChain, Continue.dev and Tabnine.