After the viral launch of ChatGPT in late November 2022 saw users posing questions of every kind to the seemingly ‘all-knowing’ chatbot, Google famously declared a ‘code red’.
Ever since then, there’s been increasing talk of whether Google might be under threat from large language model (LLM)-powered chatbots like ChatGPT or search engines that use them – like Microsoft’s Bing, which integrated OpenAI into its search engine in March 2023, and rising player Perplexity.ai.
The perceived threat led Google to implement AI Overviews, a generative AI-powered feature that can summarise search results to provide a single answer, which was rolled out to all US users in May. However, the launch exposed flaws in the way that Google’s LLM produced its results and led to silly, problematic or even dangerous recommendations, such as that searchers eat one small rock per day or put glue on their pizza. Google was quick to rein in the feature, preventing it from being triggered by certain types of search and drawing on satirical or user-generated content.
All of this paints a picture of an ageing search giant that could soon be outflanked by nimbler competitors – but how true is it? Rand Fishkin, Co-Founder of SparkToro and former CEO of Moz, carried out a recent study that looked at Google searches and where they go – and concluded that there is very little threat to Google’s dominance.
Searches on Google are at “historic highs”
Fishkin’s study uses clickstream data from Datos, owned by SaaS platform Semrush, which offers anonymised and scaled datasets based on the browsing behaviour of tens of millions of panellists.
While Fishkin offers some caveats as to how representative the clickstream data is likely to be of the overall population – particularly on mobile, where there is minimal coverage of iOS mobile devices, something that may impact insights in the US in particular – he attests that historically, the insights are “as close as we can get” to matching actual behaviour.
Looking at the trend in Google desktop searches over the course of 21 months, from September 2022 to May 2024, Fishkin concluded that “searches per searcher are rising and, in the Spring of 2024, were at historic highs”. Datos did not have mobile search data available prior to January 2024, so the study uses desktop data only to provide a longer trend line.
“That data doesn’t fit well with the narrative that Google’s cost themselves credibility or that Internet users are giving up on Google and seeking out alternatives,” Fishkin observes.
“According to Datos’ panel, Google’s in no risk of losing market share, total searches, or searches per searcher. On all of these metrics they are, in fact, stronger than ever.”
Image source: Datos & SparkToro
In the study, Fishkin compared search volumes from the US and the EU in order to highlight any regional differences, for example thanks to the EU’s Digital Markets Act, which came into force in November 2022 and began to apply in May 2023. In September 2022, Google searches per searcher were markedly lower in the EU than in the US, around 120 versus the US 130 – but have climbed up to near the US level, which is now hovering at close to 140 searches per searcher.
Following a search, 37.1% of searchers in the US and 37.4% in the EU end their journey on the search results page (SERP) – so a little under two-fifths of searches are being satisfied directly on the SERP. In the US, 21.4% of searchers go on to conduct another search (22.3% in the EU) and 41.5% click on a search result or paid ad (40.3% in the EU).
Have Google’s AI overviews changed anything?
Of particular interest is whether the presence of Google’s AI overviews has done anything to affect the percentage of searches that are satisfied by the SERP. Many publishers and businesses fear the impact that the addition of AI Overviews could potentially have on traffic, as searchers are given a direct answer to their query without needing to peruse various results.
Fishkin carried out a comparison of search behaviour over time from January to May 2024 (when AI Overviews were rolled out in the US) – and found little indication of any impact. Clicks per search remained essentially flat between January and May on desktop in both the EU and the US (if anything, they rose fractionally between April and May in the US, but only to the level already seen in January and March).
On mobile, clicks per search in May rose slightly in both the EU and the US – which can be attributed to the fact that May saw a decline in mobile search volume, again in both regions.
Mobile searches per searcher fell significantly between April and May in both the US and EU – something that Fishkin says could have “spooked Google into rolling back” AI Overviews; but given that the trend was the same in the EU, it’s difficult to see any link.
Given that AI Overviews were only rolled out in mid-May and reined in just two weeks later (studies indicate that they are currently only displaying for 12.5% of searches, with some data showing they display on just 7%), they will have had little time to affect the data.
If Google feels able to up the percentage of queries that trigger an AI Overview, we may see this change – but right now, there’s no real evidence that they’ve nudged search behaviour in any particular direction.
“All sound and fury, signifying nothing”
Looking at the data on search behaviour and volume, Fishkin feels confident enough to declare that, “Both the fear of AI Overviews and the “death of Google search quality” are all sound and fury, signifying nothing. Or, at least, not very much.”
And truthfully, we didn’t need a study to tell us that – although it’s always worth having the hard data. Although many have complained about a perceived decline in Google search quality in recent years, they’re most likely a dedicated minority of searchers – search engine market share data doesn’t show any significant gains for rivals like Bing, Yahoo!, or DuckDuckGo, which it should if Google were losing out to alternatives in a notable way.
Google’s dominance in web search at this stage is difficult to challenge – web search technology is expensive, meaning that most smaller rivals will be licensing their technology from a larger player like Google or Bing anyway – and for the average internet user, Google works well enough, meaning that there’s no significant incentive to switch.
Generative AI brought with it the promise of an all-knowing ‘answer engine’ style search experience where users could simply receive a single answer to their query instead of having to sift through results – but a lot of search queries don’t have a definitive answer, and Google’s AI Overview struggles show how challenging it can be to get this right, even with Google’s scale and resources.
However, the gaffes from AI Overviews haven’t significantly dented Google’s standing so far, which may be thanks to how quickly Google clamped down when things started going awry. The net result of this is that – so far – AI Overviews haven’t much changed the experience of using Google. But right now, Google doesn’t appear to be suffering from that lack of change, either.