Creating a compelling video isn't just about the visuals. Much of the appeal of great video content is in the audio, but finding (or sometimes creating) the right audio effects can be a time-consuming process. Adobe is showing off Project Super Sonic at its annual Max conference. This is an experimental prototype demo that shows how you can one day quickly generate background audio and audio effects using Text-to-Audio, object recognition, and even your own voice. video project.
Being able to generate audio effects from text prompts is fun, but it might not be all that groundbreaking considering that companies like ElevenLab already offer this commercially.
What's even more interesting here is that Adobe is taking all of this a step further by adding two additional modes for creating these soundtracks. The first is to use an object recognition model that allows you to create a prompt and generate a sound by clicking on any part of the video frame. This is a clever way to combine multiple models into one workflow.
However, the real wow moment comes with the third mode, where you record yourself (along with a video) mimicking the sound you're looking for, and Project Super Sonic automatically generates the appropriate audio. I will.
Justin Salamon, head of sound design AI at Adobe, told me that the team started with a text-to-audio model. It also said that, as with all of Adobe's generative AI projects, the team used only licensed data.
“What we really wanted was to give users control over the process. We wanted this to be a tool for creators, sound designers, and anyone who wants to enhance their videos with sound. So we wanted to go beyond the initial text-to-sound workflow, so we worked on things like vocal controls that give you precise control over energy and timing, making it a truly expressive tool. ,” Salamon explained.
For vocal control, this tool actually analyzes various characteristics of your voice and the spectrum of the sound you're creating, and uses that to guide the production process. Salamon noted that while the demo uses audio, users can also clap their hands and play instruments.
It's worth noting that Adobe Max has always had a lot of things called “sneaks” built into it. These, like Project Super Sonic, are meant to showcase some of the experimental features the company is currently working on. Many of these projects will end up in Adobe's Creative Suite, but there is no guarantee that they will. Project Super Sonic is certainly a useful addition to something like Adobe Premiere, but you may never see it again.
One of the reasons I believe this project will make it into production is that the same group is also working on the audio portion of Generative Extend, a feature of the Firefly generative AI model that extends short video clips containing an audio track by a few seconds. It's about being. However, for now, Project Super Sonic remains a demo.