On Thursday, Google announced that it is rolling out new AI and accessibility features for Android and Chrome. Most notably, Talkback's Android screen reader allows Gemini to ask what the image is and what it contains on the screen.
Last year, Google brought Gemini's capabilities to those who are blind or provide low visual access to AI-generated descriptions of images, even when Alt text is unavailable. Now people can ask questions and get answers about their images.
Image credit: Google
For example, if a friend texts you a new guitar, you can get a description of it and ask questions about the brand and color. Additionally, you can get a description and ask questions about the entire phone screen. So if you're shopping on the app, you can ask Gemini about the material of the items you're interested in.
Google today announced that it is updating its Explusive Captions and Android real-time captioning feature.
Google says it recognizes that one way people express themselves is to elicit the sounds of their words. So we have developed a new period feature with expressive captions. Now, you can see if the sports announcer is calling for “Amaaashing Shot” or if someone is saying “nooooo” instead of simply saying “no.” Also, new labels for sounds begin to appear when people whist their throats or clean their throats.
Image credit: Google
This update is being deployed in English in the US, UK, Canada and Australia for devices running Android 15 or later.
Google also makes PDFs easy to access in Chrome. Previously, I was unable to interact with scanned PDFs in my desktop Chrome browser using a screen reader. Here, Chrome automatically recognizes these types of PDFs, and can highlight, copy, search text, and read it using a screen reader, just like any other page. This is thanks to the introduction of optical character recognition (OCR), Google says.
Additionally, you can use Page Zoom in Chrome on Android to increase the size of text that is displayed without affecting the layout of your web page. You can customize the amount you want to zoom in and then choose to apply preferences to all pages you visit or to specific pages. You can access this feature by tapping the 3-dot menu in the top right corner of Chrome.