Google’s best new Android feature in years is Circle to Search, which is why it took center stage at today’s Google I/O opening keynote. Google announced new updates to Gemini, the AI that powers the Android operating system. Most of the new features sound helpful, though they might seem more subtle than you were expecting.
Google says it will reveal more about Android on Wednesday, May 15, on the second day of its developer conference. Stay tuned for those announcements.
Circle to Search your homework
The first big Gemini update is now available through the Circle to Search gesture. Starting today, you can use the long-press shortcut to look up answers to various physics and math word questions. Google Search will return step-by-step instructions upon query. Note that you’ll have to opt-in for help with math and word problems from the Search Labs menu for the feature to work.
Google lauds in its blog that this is all possible with its LearnLM, its “new family of models fine-tuned for learning.” Honestly, my elder millennial self is jealous you kids have help like this these days. Google plans to release more STEM homework help later in the year, including the ability to identify graphs and diagrams.
Gemini gains more context
If it’s more context that you wanted for Google Gemini, it’s getting even more of it this year. Google announced at I/O that in an upcoming update, you will call Gemini to interact with the app you’re using. For instance, you can drag and drop a Gemini-generated image into another app like Messages or go deeper on a YouTube video without navigating away. Gemini Advanced subscribers will also have the option to query a PDF for a quick summary. This particular update will roll out “over the next few months.”
Gemini Nano goes multimodal on device
Gemini Nano—the smallest model in Google’s Gemini family and the one that runs on-device the Pixel hardware—will also see an update on compatible Pixel devices later this year. First, Gemini Nano will become multimodal. The AI will finally be able to process text input and audio or video simultaneously. It will benefit Android features like TalkBack, where Nano can generate descriptions for images and interface elements that don’t offer details.
Gemini Nano is also powering the scam detection feature in the Phone app that Google announced during the keynote. This feature scans the conversation on-device to determine if there’s a scam happening in real-time. Google says the conversation is processed on the hardware side and doesn’t go to a server. Instead, Nano listens for trigger words and phrases and then provides feedback that you may be involved in a scam.
Gemini for developers
Steve Ballmer of Microsoft fame said it first: the developer conference is all about the developers, developers, developers! Google reminded its developers that Gemini is available for use in Android Studio. Developers get early access to Gemini 1.5 Pro, which can help with coding and compiling.