Google Releases New Accessibility and Artificial Intelligence Features For Chrome and Android

Google revealed on Thursday that Chrome and Android will soon get new AI and accessibility features. Most significantly, you can now ask Gemini what’s on your screen and what’s in pictures using TalkBack, Android’s screen reader.

In order to provide AI-generated descriptions for photos to those who are blind or have low eyesight, even in situations where Alt text is unavailable, Google integrated Gemini’s capabilities into TalkBack last year. People can now post questions regarding their photos and receive answers.

For instance, you can ask questions about the brand and color of a friend’s new guitar and receive a description if they text you a picture of it. Additionally, you may now ask questions and receive descriptions of the entire screen of your phone. You can therefore ask Gemini about the material of an item you’re interested in or whether there is a discount available when you’re shopping on an app.

Expressive Captions, Android’s real-time captioning feature that use AI to record what people say and how they say it, is getting an update, Google also revealed today.

Google says that it has created a new duration feature on Expressive Captions because it recognizes that dragging out the sound of one’s words is one way individuals express themselves. You will now be able to tell whether a sports commentator is announcing a “amaaazing shot” or when someone is shouting “nooooo” instead of just “no.” New labels for sounds, such whistling or clearing one’s throat, will also begin to appear.

For Android 15 and later smartphones, the update is being released in English in the United States, the United Kingdom, Canada, and Australia.

Additionally, Google is improving Chrome’s ability to access PDFs. Up until recently, you couldn’t interact with a scanned PDF in your desktop Chrome browser using your screen reader. These PDFs are now automatically recognized by Chrome, so you can use your screen reader to read them and highlight, copy, and search for text just like you would on any other page. According to Google, this is because to the advent of optical character recognition, or OCR.

Additionally, you may now enlarge the text you view in Chrome on Android without changing the layout of the webpage by using Page Zoom. The degree of zooming in can be adjusted, and you can decide whether to apply the preference to every page you visit or only specific ones. Tapping the three-dot menu in Chrome’s upper-right corner will bring up this option.