Google makes adjustments to AI Overviews after a rocky rollout

Google is making some changes to its AI Overviews, after the artificial intelligence-driven search feature gave what the company calls “odd and erroneous” responses to people’s online searches.

AI Overviews were introduced last month at Google’s annual I/O developer conference. Now, when people use Google Search to find information on certain topics, a box of AI-generated text appears at the top of the search results, annotated with links to external websites. Traditional search results appear below the AI Overviews, marking a major shift in how Google presents information.

According to a blog post from Google VP Liz Reid, AI Overviews results are generated using the company’s large language model (LLM), Gemini, and are designed for instances when someone wants to “get both a quick overview of a topic and links to learn more.”

Google technology expert Alex Joseph told ABC Audio that AI Overviews is able to field more complex questions than a traditional Google Search.

“With an AI Overview what [Google] can really do is synthesize a lot of information and get you the answer that you’re looking for very quickly,” said Joseph.

Instead of presenting users with pages of links to comb though, Joseph said, AI Overviews streamlines the process by summarizing information and providing users with a concise answer.

“They’ll have less friction, they won’t have to click through to a number of different websites, which can often be quite a bit of a bugbear if you just want some information very quickly,” notes Chris Stokel-Walker, technology journalist and author of the book “How AI Ate the World: A Brief History of Artificial Intelligence – and Its Long Future.”

However, Stokel-Walker said the new feature makes it harder for people using Google Search to verify the accuracy of the information they’re reading.

“We’ve kind of got used to over the last two decades of Google Search dominance to the results that we get to a search term being largely right,” he told ABC Audio. “Suddenly, if you get rid of that, as Google is proposing, and actually just shove an answer straight into the search results page that its created via generative AI, you have no real way of identifying and kind of analyzing that information to see whether it’s true or not.”

There are other concerns about the new feature as well. For one, generative AI technology, both from Google and elsewhere, has faced criticism for “hallucinating” – that is, generating information that’s unreliable and inaccurate.

For example, in the few weeks since AI Overviews have been available to the public, people using Google Search have been advised to eat at least one small rock per day, and it told one user that a good way to get cheese to stick to pizza is to mix glue into the tomato sauce – both of which are, of course, very bad ideas. It also said that Andrew Jackson, the seventh U.S. president who died in 1845, graduated from college in 2005.

Stokel-Walker said any benefit from AI Overviews ultimately comes down to a tradeoff between convenience and cost. “You no longer have kind of click through five or six different pages and maybe several pages of search results to find the right answer, but also it does mean that either the answer might be wrong or it might not be the one that you actually want to get,” he said.

“We’ve always been very clear about the limitations of LLMs, that there will be occasionally hallucinations,” Google’s Alex Joseph said, adding that’s why AI Overviews also cites the websites it uses to generate its answers.

“It’s part of the reason that we present all of the information for you holistically,” said Joseph. “These are quick shortcuts to help you get some information to you quickly, but they’re followed along with areas where you can go, double check, verify.”

Joseph also said not all queries are best served by an AI Overviews: “We only show them on queries where we have a high confidence that it’s going to be helpful and actually enhance the experience.”

In the wake of the unusual responses some social media users have reported, Google announced it had made “more than a dozen technical improvements” to AI Overviews. According to Liz Reid’s blog post, they include limiting the inclusion of user-generated content, as well as satirical or humorous webpages, in the data used to craft AI Overviews. Reid said Google also “launched additional triggering refinements to enhance our quality protections” regarding health content, and that it “aim[s] to not show AI Overviews for hard news topics, where freshness and factuality are important.”

The blog post also notes that “AI Overviews generally don’t ‘hallucinate’ or make things up in the ways that other LLM products might,” and that the incorrect answers are the result of “misinterpreting queries, misinterpreting a nuance of language on the web, or not having a lot of great information available.”

Aside from the accuracy concerns, Stokel-Walker said Google prioritizing AI Overviews over traditional search results could affect revenue and reshape how business is done on the web.

“Websites produce content; they try and make it attractive to Google. Google will show them in its search results. And as a result, people click through to their website, they then see ads off the back of that, and the publisher makes the money that allows them to put new content onto websites,” Stokel-Walker explained.

By replacing the top of the Google Search results page with AI-generated content, however, Stokel-Walker said websites could see fewer visitors – and therefore less ad revenue.

It’s an ironic situation, according to Stokel-Walker. That’s because Gemini, the LLM Google uses to create its AI Overviews, relies on the websites it’s now appearing ahead of.

“These websites do still need to exist, and they do need to have a way of making income, because otherwise, there’s nothing for those AI-generated search results to be based on,” Stokel-Walker pointed out.

In a statement to ABC News, Google said its testing has shown that the opposite is actually happening: that the links included in AI Overviews get more clicks than if the page had appeared as it typically does in search results. Google also said it will “continue to focus” on sending valuable traffic to publishers and creators.

Regardless of how the concerns about AI Overviews ultimately shake out, it’s just one of a slew of features the company has planned for its line of technology products.

“I think that doing something like this as quickly as Google is doing it is concerning,” said technology journalist C. Scott Brown of the website Android Authority.

Google has announced plans for additional features similar to AI Overviews, which will aim to answer questions about specific webpages or YouTube videos. Brown says those features will hit the market against the backdrop of increasing competition.

“And the reason it’s doing that is because it feels like it has to. It has to keep up with companies – especially like OpenAI, for example – that are creating generative AI technologies that are threatening Google’s core business, which is delivering information to people through Google search, and thus delivering them advertisements that allow Google to make billions and billions of dollars,” said Brown.

“With Google seeing these things as a threat, it can’t just rest. It can’t figure out how to do this cleanly, and do it right,” Brown added. “It just has to do it.”