You may not know Viggle AI, but you've probably seen the viral memes it's created. The Canadian AI startup has made dozens of remix videos of rapper Lil Yachty bopping around onstage at summer music festivals. In one, Lil Yachty is replaced by Joaquin Phoenix's Joker. In another, Jesus appears to be pumping up the crowd. Users created countless versions of the video, but it was one AI startup that made the meme go viral. Viggle's CEO says YouTube videos power the company's AI model.
Viggle trained its 3D video foundation model, JST-1, to have a “true understanding of physics,” as the company claims in a press release. Viggle CEO Hang Chu says the main difference between Viggle and other AI video models is that Viggle allows users to specify the movements they want their characters to make. While other AI video models often create unrealistic character movements that don't follow the laws of physics, Chu claims Viggle's model is different.
“We're essentially building a new type of graphics engine that uses only neural networks,” Chu said in an interview. “The model itself is quite different from existing video generators, which are primarily pixel-based and don't have much understanding of physical structures or properties. Our model is designed to have that understanding, which gives us much more control and efficiency in generation.”
For example, to create a video of the Joker impersonating Lil Yachty, all you need to do is upload an image of the original video (of Lil Yachty dancing on stage) and a character (the Joker) who will recreate those movements. Alternatively, you can upload an image of your character and some text prompts that explain how to animate it. As a third option, Viggle lets you create an animated character from scratch using only text prompts.
But memes are only a small part of Viggle's users. Chu says the models have been widely adopted as a visualization tool for creative people. Though the videos are far from perfect — shaky, with expressionless faces — Chu says they've proven effective for filmmakers, animators and video game designers to visually express their ideas. Currently, Viggle's models only create characters, but Chu hopes to be able to create more complex videos in the future.
Viggle currently offers a limited free version of its AI models on Discord and in its web app. The company also offers a $9.99 subscription for more capacity and gives select creators special access through its Creators Program. The CEO said Viggle is in talks with film and video game studios to license the technology, but is also seeing adoption among independent animators and content creators.
On Monday, Viggle announced it had raised $19 million in Series A funding led by Andreessen Horowitz with participation from Two Small Fish. The startup said the round will help Viggle scale, accelerate product development and expand its team. Viggle told TechCrunch that it partners with cloud providers such as Google Cloud to train and run its AI models. These partnerships with Google Cloud often include access to GPU and TPU clusters, but don't typically include using YouTube videos to train AI models.
Training Data
In TechCrunch's interview with Chu, we asked what kind of data Viggle's AI video model was trained on.
“Until now we've relied on publicly available data,” Chu said, echoing a response given by OpenAI CTO Mira Murati about Sora's training data.
When asked if Viggle’s training data set includes YouTube videos, Chu answered with a resounding “yes.”
That could be a problem: In April, YouTube CEO Neal Mohan told Bloomberg that using YouTube videos to train an AI text-video generator would be a “clear violation” of the platform's terms of service, in the context of OpenAI potentially using YouTube videos to train Sora.
Mohan revealed that Google, which owns YouTube, may have agreements with certain creators to use their videos in Google DeepMind's Gemini training dataset, but according to Mohan and YouTube's terms of service, the company is not allowed to scrape videos from the platform without first getting permission from the company.
After TechCrunch interviewed Viggle's CEO, a Viggle spokesperson emailed TechCrunch to retract Chu's comments, saying, “It was premature for the CEO to speak about whether Viggle uses YouTube data for training. In fact, Hang/Viggle cannot share details about training data.”
However, we pointed out that Chu has already done so publicly and asked for a clarification on the matter. In a response, a Viggle spokesperson confirmed that the AI startup trains on YouTube videos.
Viggle generates AI content by leveraging a variety of public sources, including YouTube. Our training data is carefully curated and refined, adhering to all terms of service throughout the process. We prioritize maintaining strong relationships with platforms like YouTube, and are committed to respecting their terms of service by avoiding bulk downloads and other actions that lead to unauthorized video downloads.
This compliance effort seems to contradict comments made by Mohan in April that YouTube's video corpus is not a public source. We have reached out to YouTube and Google spokespeople but have not yet heard back.
The startup is in a gray area like others when it comes to using YouTube as training data. Many AI model developers, including OpenAI, Nvidia, Apple, and Anthropic, have been reported to use transcripts and clips from YouTube videos for training. It's a Silicon Valley dirty secret, but it's not that secret. Everyone probably does it. What's actually unusual is saying it out loud.