Google promised a sweeping overhaul to its ubiquitous search at I/O last week — but users are already pushing its limits.
In some cases, the new AI-powered tool returns responses that aren’t accurate. Some are slightly nonsensical. Others are downright wrong.
It’s called AI Overview. Rather than giving you a list of third-party web pages, the new Google search function creates a new box with conversational answers culled from across the web and fueled by generative AI.
“Google will do the googling for you” was how head of search Liz Reid put it onstage last week.
But in searches shared on X, users have gotten contradictory instructions on boiling taro and even been encouraged to run with scissors after the AI appeared to take a joke search seriously.
When we asked whether a dog had ever played in the NHL, Google answered it had, apparently confused by a charity event for rescue pups.
Another time, Google’s AI assistant said a series of presidents graduated from UW-Madison, citing a webpage referencing when students at the school had the same names as US presidents.
Paul Squire
Inverse editor Jake Kleinman noted a query for the biggest movie flops of 2024 returned a list from a website speculating on future box office figures. AI Overview confidently said movies that hadn’t come out yet were already failures, based on that article.
A Google spokesperson told Business Insider that the examples being shared online were “extremely rare queries and aren’t representative of most people’s experiences.” The spokesperson said the “vast majority of AI Overviews provide high-quality information.”
The spokesperson said the examples only started gaining search interest because of the posts on social media. Google says it’s seen more search usage and user satisfaction since AI Overview launched.
“We conducted extensive testing before launching this new experience and will use these isolated examples as we continue to refine our systems overall,” the spokesperson said.
AI Overviews are available across the US, Google said at I/O, and will roll out to 1 billion users by year’s end — though some are already trying to figure out how to disable it.
The Google spokesperson said the company will introduce a “web” filter that exclusively turns up links.
In addition to questions of reliability, AI is facing concerns from business owners who have long relied on SEO to drive hits and ad dollars.
They’re worried that AI Overviews could decimate search traffic to their websites.
In a blog post, Reid said the “links included in AI Overviews get more clicks than if the page had appeared as a traditional web listing for that query,” adding that Google will “continue to focus on sending valuable traffic to publishers and creators.”