AI video creation platform D-ID is the latest company to release a tool that uses AI technology to translate videos into other languages, except in this case, D-ID replicates the speaker's voice and changes their lip movements to match the translated words as part of the AI editing process.
The technology grew out of D-ID's previous efforts. You may remember the viral trend a few years ago where users animated old family photos that later started talking. Following this success, the startup is looking to raise $25 million in Series B funding in 2022 to cater to the growing number of enterprise clients in the US who are using its technology to create AI-powered videos.
The company's currently launched AI video translation technology, which is currently available for free to D-ID subscribers, allows creators to expand their reach by automatically translating their videos into other languages. A total of 30 languages are currently available, including Arabic, Chinese, Japanese, Hindi, Spanish, and French. D-ID subscriptions start at $56 per year for the cheapest plan, which offers the fewest number of credits for AI features, and then go up to $1,293 per year, followed by enterprise pricing.
D-ID suggests that its new AI video technology could help its clients save on localization costs when expanding campaigns to global audiences in areas such as marketing, entertainment and social media. The technology will compete with other solutions in both dubbing and AI video.
For years, dubbing technology has made it easy for video viewers to hear audio in their own language, but it was often unavailable to smaller creators. That's changing as companies improve access to the technology. For example, YouTube launched a multilingual voice feature designed to help creators translate their videos into other languages to connect with a wider audience. Well-known creator MrBeast (Jimmy Donaldson) was one of the early adopters of the technology, using it to translate some of his most popular videos into over 11 languages.
AI is also expanding the capabilities of voice creation, translation, and duplication. Microsoft announced this year that it would use AI to translate and dub videos as you watch them on YouTube and other platforms. In July, creator platform Vimeo unveiled a tool to translate audio and subtitles by using AI technology to replicate the speaker's voice. A number of companies offer voice duplication and/or AI translation tools, including Descript, ElevenLabs, Speechify, Veed, Camb.ai, Captions.ai, and Akool. There are also tools from HeyGen, Deepbrain AI, and others that allow you to create videos with AI avatars that can speak dozens of languages.
Voice-over and lip-syncing AI libraries like Wav2lip are making it easier for startups to build these kinds of tools and pitch AI technology to creators as easier, and perhaps more affordable, to use.
D-ID says its new video translation technology will be available through D-ID Studio and its API. A one-month trial is being offered, with further demos posted on the company's website.