Close Menu
TechBrunchTechBrunch
  • Home
  • AI
  • Apps
  • Crypto
  • Security
  • Startups
  • TechCrunch
  • Venture

Subscribe to Updates

Subscribe to our newsletter and never miss our latest news

Subscribe my Newsletter for New Posts & tips Let's stay updated!

What's Hot

The founders of Digg explain how they are building sites for humans in the age of AI

June 2, 2025

Elon Musk's Neuralink closes the $650 million series

June 2, 2025

Finally, the $25 billion worth of chimes aims to reach $11 billion in future IPOs

June 2, 2025
Facebook X (Twitter) Instagram
TechBrunchTechBrunch
  • Home
  • AI

    OpenAI seeks to extend human lifespans with the help of longevity startups

    January 17, 2025

    Farewell to the $200 million woolly mammoth and TikTok

    January 17, 2025

    Nord Security founder launches Nexos.ai to help enterprises move AI projects from pilot to production

    January 17, 2025

    Data proves it remains difficult for startups to raise capital, even though VCs invested $75 billion in the fourth quarter

    January 16, 2025

    Apple suspends AI notification summaries for news after generating false alerts

    January 16, 2025
  • Apps

    The founders of Digg explain how they are building sites for humans in the age of AI

    June 2, 2025

    Elon Musk says Xchat is rolling out to everything, but questions remain about its suspected security

    June 2, 2025

    Google quietly released an app that allows you to download and run AI models locally

    May 31, 2025

    A guide to using editing, Meta's new Capcut Rival for Short-Form video editing

    May 31, 2025

    Automattic says it will start contributing to WordPress again after pause

    May 30, 2025
  • Crypto

    GameStop bought $500 million in Bitcoin

    May 28, 2025

    Vote for the session you want to watch in 2025

    May 26, 2025

    Save $900 + 90% from 2 tickets to destroy 2025 in the last 24 hours

    May 25, 2025

    Only 3 days left to save up to $900 to destroy the 2025 pass

    May 23, 2025

    Starting from up to $900 from Ticep, 90% off +1 in 2025

    May 22, 2025
  • Security

    Vanta Bug has made its customer data public to other customers

    June 2, 2025

    NSO Group calls the judge for a new trial, calling $167 million in damages “outrageous”

    June 2, 2025

    8 things we learned from WhatsApp vs. NSO Group Spyware Litigation

    May 30, 2025

    White House investigates how Trump's chief staff's phone was hacked

    May 30, 2025

    US government sanctions technology company involved in cyber fraud

    May 29, 2025
  • Startups

    7 days left: Founders and VCs save over $300 on all stage passes

    March 24, 2025

    AI chip startup Furiosaai reportedly rejecting $800 million acquisition offer from Meta

    March 24, 2025

    20 Hottest Open Source Startups of 2024

    March 22, 2025

    Andrill may build a weapons factory in the UK

    March 21, 2025

    Startup Weekly: Wiz bets paid off at M&A Rich Week

    March 21, 2025
  • TechCrunch

    OpenSea takes a long-term view with a focus on UX despite NFT sales remaining low

    February 8, 2024

    AI will save software companies' growth dreams

    February 8, 2024

    B2B and B2C are not about who buys, but how you sell

    February 5, 2024

    It's time for venture capital to break away from fast fashion

    February 3, 2024

    a16z's Chris Dixon believes it's time to focus on blockchain use cases rather than speculation

    February 2, 2024
  • Venture

    Elon Musk's Neuralink closes the $650 million series

    June 2, 2025

    Finally, the $25 billion worth of chimes aims to reach $11 billion in future IPOs

    June 2, 2025

    Elon Musk's Xai is reportedly seeking a $300 million tender offer

    June 2, 2025

    Request Exhibitor Tables for TC All Stages for 5 days remaining | TechCrunch

    June 2, 2025

    TC Session: AI Trivi Account Down – Earn big tickets

    June 2, 2025
TechBrunchTechBrunch

Meta's Movie Gen model outputs realistic videos with sound, so you can finally achieve infinite Moo Deng.

TechBrunchBy TechBrunchOctober 4, 20245 Mins Read
Facebook Twitter Pinterest Telegram LinkedIn Tumblr WhatsApp Email
Share
Facebook Twitter LinkedIn Pinterest Telegram Email


No one knows yet what generative video models are good for, but that doesn't stop companies like Runway, OpenAI, and Meta from pouring millions of dollars into their development. The latest version of Meta is called Movie Gen, and as the name suggests, it turns text prompts into relatively realistic videos with audio. But thankfully there is no audio yet. And wisely, they don't release this to the public.

Movie Gen is actually a collection of basic models (or “casts” as they call them), the largest of which is the text-to-video bit. Meta claims to outperform the likes of Runway's Gen3, LumaLabs' latest work, and Kling1.5, but as always, this kind of thing is less about Movie Gen winning and more about them. This is to show that they are playing the same game. Technical details can be found in the documentation published by Meta describing all components.

The audio is generated to match the content of the video, such as the sound of an engine as the car moves, the force of a waterfall in the background, or thunder added as needed during the video. Also add music if you think it's relevant.

It was trained on a “combination of licensed and publicly available datasets,” which they call “proprietary/commercially sensitive,” without providing further details. It was. Our best guess is that in addition to the large amount of Instagram and Facebook videos, there are also a large number of partner videos and other videos that are not well protected from scrapers, i.e. “public” videos. .

But Meta's clear goal here is not just to earn the “state-of-the-art'' accolade for a month or two, but to create a practical product that can produce a solid end product from a very simple process. , a soup-to-nuts approach. , natural language prompts. Something like, “Imagine me as a baker making a shiny hippo cake in a thunderstorm.”

For example, one of the problems with these video generators is that they are usually very difficult to edit. If you request a video of someone walking across the street and then realize you want them to walk right to left instead of left to right, repeating the prompt with that additional instruction will make the whole shot look different. There is a high possibility that it will look like this. Meta adds a simple text-based editing method. Just say “change the background to a busy intersection” or “change her outfit to a red dress” and it will attempt that change, but only that change will happen.

Image credit: Meta

Camera movement is also commonly understood, and things like “tracking shots” and “left pans” are taken into account when generating video. This is still pretty clunky compared to actual camera control, but it's much better than doing nothing.

The model limitations are a bit strange. Generates a video that is 768 pixels wide. This is the size most people are familiar with in the famous but outdated 1024×768, but it's also three times as large as 256, so it plays well in other HD formats as well. The Movie Gen system upscales this to 1080p. This is the basis for the claim to produce that resolution. Not really, but upscaling is surprisingly effective, so I'll give it a pass.

Oddly enough, it produces up to 16 seconds of video…16 frames per second, a frame rate that no one in history has ever wanted or demanded. However, you can also run 10 second videos at 24 FPS. Please lead with that!

As for why there is no audio…well, there are probably two reasons. First of all, it's super difficult. Generating speech is now easy, but matching it to lip movements, and those lip and facial movements, is a much more complex proposition. This was just a momentary failure, so I don't blame them for leaving it until later. Someone might say, “I'm going to generate a clown that rides around in circles on a small bicycle and delivers the Gettysburg Address.'' This is nightmare fuel and can spread quickly.

The second reason is probably political. Releasing what amounts to a deepfake generator a month before a major election is…not in the best interest of optics. A practical precaution is to limit its functionality a bit so that if a malicious attacker tries to use it, it requires some real work on their part. Sure, you can combine this generative model with a voice generator or an open lip sync model, but you can't just have it generate candidates who make outlandish claims.

“Movie Gen is purely an AI research concept at this point, and even at this early stage, safety is our top priority, as with all of our generative AI technology,” a Meta representative told TechCrunch when asked. He answered and spoke.

For example, unlike Llama's large language model, Movie Gen is not publicly available. Although the technique can be reproduced to some extent by following the research paper, the code is not made public except for the “underlying assessment prompt dataset”, i.e. the recording of the prompts used to generate the test videos.



Source link

Share. Facebook Twitter Pinterest LinkedIn Tumblr Email

Related Posts

OpenAI seeks to extend human lifespans with the help of longevity startups

January 17, 2025

Farewell to the $200 million woolly mammoth and TikTok

January 17, 2025

Nord Security founder launches Nexos.ai to help enterprises move AI projects from pilot to production

January 17, 2025

Data proves it remains difficult for startups to raise capital, even though VCs invested $75 billion in the fourth quarter

January 16, 2025

Apple suspends AI notification summaries for news after generating false alerts

January 16, 2025

Nvidia releases more tools and guardrails to help enterprises adopt AI agents

January 16, 2025

Leave A Reply Cancel Reply

Top Reviews
Editors Picks

7 days left: Founders and VCs save over $300 on all stage passes

March 24, 2025

AI chip startup Furiosaai reportedly rejecting $800 million acquisition offer from Meta

March 24, 2025

20 Hottest Open Source Startups of 2024

March 22, 2025

Andrill may build a weapons factory in the UK

March 21, 2025
About Us
About Us

Welcome to Tech Brunch, your go-to destination for cutting-edge insights, news, and analysis in the fields of Artificial Intelligence (AI), Cryptocurrency, Technology, and Startups. At Tech Brunch, we are passionate about exploring the latest trends, innovations, and developments shaping the future of these dynamic industries.

Our Picks

The founders of Digg explain how they are building sites for humans in the age of AI

June 2, 2025

Elon Musk's Neuralink closes the $650 million series

June 2, 2025

Finally, the $25 billion worth of chimes aims to reach $11 billion in future IPOs

June 2, 2025

Subscribe to Updates

Subscribe to our newsletter and never miss our latest news

Subscribe my Newsletter for New Posts & tips Let's stay updated!

© 2025 TechBrunch. Designed by TechBrunch.
  • Home
  • About Tech Brunch
  • Advertise with Tech Brunch
  • Contact us
  • DMCA Notice
  • Privacy Policy
  • Terms of Use

Type above and press Enter to search. Press Esc to cancel.