Gemini Live is Google's answer to the recently released (limited alpha) Advanced Voice Mode for OpenAI's ChatGPT, and it launches on Tuesday, a few months after it was announced at the Google I/O 2024 developer conference. It was announced at Google's Made by Google 2024 event.
Gemini Live allows users to have “in-depth” voice chats with Google's generative AI-powered chatbot, Gemini, on their smartphones. Thanks to an enhanced speech engine that Google claims delivers more coherent, emotive, and lifelike multi-turn interactions, users can interrupt Gemini to ask follow-up questions while the chatbot is talking, and Gemini will adapt to the user's speech patterns in real time.
A Google blog post explains: “Gemini Live [via the Gemini app]You can talk to your Gemini and choose [10 new] You can respond in a natural voice, speak at your own pace and even interrupt a response to ask a question, just like in any other conversation.”
Gemini Live can be used hands-free if you prefer: conversations can continue in the Gemini app even when it's in the background or your phone is locked, and conversations can be paused and resumed at any time.
So how does this help? Google cites the example of rehearsing for a job interview. A slightly tongue-in-cheek scenario, but that's OK. Google says that Gemini Live can practice alongside you, offering speaking tips and suggesting skills to highlight when speaking with a recruiter (or, in some cases, an AI).
One advantage Gemini Live has over ChatGPT's advanced voice mode is better memory: the architecture of the generative AI model underlying Live, Gemini 1.5 Pro, and Gemini 1.5 Flash has a longer-than-average “context window,” meaning it can ingest and reason about large amounts of data (theoretically hours of conversation) before formulating a response.
“Live uses an improved Gemini Advanced model that makes it more conversational,” a Google spokesperson told TechCrunch in an email, “and its larger contextual window is leveraged when users have longer conversations with Live.”
Of course, it remains to be seen how well this works in practice, and if OpenAI's failure with Advanced Voice Mode is any indication, demos rarely translate seamlessly to the real world.
Image credit: Google
For that matter, Gemini Live still doesn't have multimodal input, one of the features Google showed off at I/O. In May, Google showed off a pre-recorded video of Gemini Live recognizing and responding to the user's surroundings through photos and footage taken with a phone camera, like saying the name of a broken bike part or describing what a piece of code on a computer screen does.
Multimodal input will be available “later this year,” Google said, without providing details. Live will also expand to other languages and iOS via the Google app later this year, but for now it will only be available in English.
Gemini Live isn't free, as is Advanced Voice Mode, which is exclusive to Gemini Advanced, the more advanced version of Gemini available with the Google One AI premium plan, which costs $20 per month.
However, other upcoming new Gemini features will be free.
Soon (within a few weeks), Android users will be able to press the power button on their phone or say “Hey Google” to have a Gemini overlay appear on top of the app they're using and ask questions about what's on the screen (a YouTube video, for example). Gemini will then be able to generate images (not of people yet, unfortunately) directly from the overlay, which can then be dragged and dropped into apps like Gmail or Google Messages.
Gemini is also gaining new integrations with Google services (or “extensions,” as the company prefers to call them) on both mobile and web. Over the coming weeks, Gemini will be able to perform more actions in Google Calendar, Keep, Tasks, YouTube Music, and Utilities, the apps that control on-device features like timers and alarms, media controls, flashlight, volume, Wi-Fi, Bluetooth, and more.
In a blog post, Google laid out some ideas for how people can take advantage of it: Assuming it all works reliably, it sounds nifty.
Ask Gemini to make you a playlist of songs that remind you of the late 90s. Snap a photo of a concert flyer and ask Gemini if you're free on that day and set a reminder to buy tickets. Ask Gemini to find a recipe in Gmail and add the ingredients to your shopping list in Keep.
Finally, starting later this week, Gemini will also be available for Android tablets.