Analyzing the Influence of AI Overviews on Web Traffic

Subscribe for free to Growth Memo's weekly expert insights and boost your skills!

Just two weeks after the launch of AI Overviews (AIOs) by Google, we are already able to examine early data to assess the impact on organic traffic.

I was curious about the effects of AIOs, especially with the rise of misinformation and potentially harmful suggestions from Google's AI responses.

I discovered that AIOs could potentially decrease organic traffic, especially for queries seeking immediate answers. However, there is also a possibility that some AIOs could drive more traffic to the websites they reference.

When AIOs Show Up

I used ZipTie to crawl the search results for 1,675 queries in the health vertical on May 22 to understand how, when, and maybe why Google shows AIOs.

I mixed ZipTie’s data with Search Console and Ahrefs to understand the implications for organic traffic, the influence of backlinks, and estimated domain traffic.

42% Of Queries Show AIOs

AIOs appeared in 42% of the searches (704 out of 1,675 queries), a significantly higher percentage compared to the 16% reported by ZipTie1 for ecommerce and the 15% reported by SEO Clarity2.

This higher rate can be attributed to Google's decision to display AIOs for more complex queries, which are more common in the health sector than in ecommerce.

I noticed a correlation between the length of search queries and the likelihood of triggering AIOs. It seems that longer queries are more likely to result in AIOs. However, it is concerning to see a high number of AIOs in the health sector, where misinformation can be particularly harmful.

Bar graph titled "Correlation between AIOs and query attributes."

It appears that there is no correlation between keyword difficulty and search volume. Even though the domain I have access to in Google Search Console is more frequently referenced for low-difficulty queries.

Image Credit: Kevin Indig

AIOs are often found together with People Also Ask (PAA), Featured Snippets (FS), and Discussions & Forums modules (D&F) on search engine results pages (SERPs). This is because these features are typically associated with searches for information.

Knowledge Panels and Top Ads did not show any connection, and they also appeared less frequently compared to other SERP features.

Bar graph displaying the correlation between AI-driven AIOs and SERP features.

Image Credit: Kevin Indig

Though correlations are weak, the data indicates that AIOs are more likely to appear for queries related to questions and comparisons.

Bar chart showing correlation between AIOs and query syntax for terms.

Image Credit: Kevin Indig

Who Shows Up In AIOs

In the dataset, the 704 AIOs mentioned a total of 4,493 sites, averaging 6.3 links per AIO.

I discovered a significant correlation (0.917) between the sites appearing in the top 10 organic search results and the sites referenced in AIOs for the 1,675 keywords.

Surprisingly, Reddit (No. 92 on the list) and Quora (No. 17) barely contributed to citations.

A table showing various domains, the number of times each domain is cited, and their rankings in the top 10 organic search results.

Image Credit: Kevin Indig

I found no strong relationships between domains that get a lot of AIO citations and their organic traffic, backlinks (referring domains), or ranking keywords.

Sites that rank well for keywords are more likely to be cited in AIOs, but it’s still unclear when Google decides to cite a site that doesn’t rank well for a keyword.

Traffic Impact Of AIOs

The main question we need to answer is how AIOs affect the organic traffic of URLs that are mentioned and not mentioned.

To dig deeper into this, I analyzed the organic clicks from Search Console for a specific domain using 1,675 non-branded keywords (US) during the week of May 7 and compared it to the week of May 14.

After filtering out low-traffic keywords, a total of 560 All-In-One printers appeared for 1,344 keywords. Among these, the target domain was mentioned 171 times across 461 different URLs.

To ensure that changes in rankings do not affect the outcome, I removed keywords with a rank change greater than 0.9 or less than -0.9. This left us with 52 out of the initial 521 URLs.

I discovered that there is a significant negative relationship of -0.61 between cited URLs and changes in website traffic. This means that when URLs are mentioned in AIOs, they receive fewer clicks. For example, one domain experienced a decrease of -8.9% in clicks when mentioned in AIOs.

Scatter plot showing the correlation between AI citations and traffic impact.

Image Credit: Kevin Indig

However, results can vary by user intent. Most URLs that lost clicks, most likely due to AIOs, targeted questions like “how to get viagra without prescription.”

AIOs are similar to Featured Snippets in terms of their impact on the number of clicks received, depending on the complexity of the keyword.

It is important to consider that a significant portion of traffic losses can be attributed to a small number of URLs. When these outlier URLs are removed from the analysis, there is a slight positive correlation of 0.1. This suggests that certain AIOs may direct more traffic to specific URLs based on the user's intent for more information.

When AIOs appear on a site without a citation, there is typically a decrease in organic traffic by an average of -2.8%. This suggests that users may still be clicking on organic search results. However, it is important to accurately measure the impact on organic clicks for the same keywords to confirm this. It is advisable to consider this result cautiously.

Caveats

It is uncertain how often AIOs can change. There doesn't appear to be a consistent rate of change over time, but Google has recently scaled back many AIOs. After the initial crawl, most AIOs couldn't be replicated within five days. This could be a response to the numerous complaints about inaccurate and potentially dangerous answers.

Starting with 1,675 queries is a positive step, but we would need at least 100 times more to draw stronger conclusions.

We also need a lot more traffic data than n=1.

Canary In The Coal Mine

Strong SEO performance is crucial for appearing in AI Overviews, although Google may still show URLs that are not performing well. The specific content features that increase the likelihood of being cited in AIOs are still not fully understood.

AI overviews have a significant impact beyond just Google Search. They represent a key AI feature that is reshaping the profitability of one of the world's most valuable companies.

AI has generated a lot of excitement since the release of ChatGPT3 in November 2022. Recently, we have been contemplating the true impact and significance of AI.

AIOs serve as a warning signal. They have the potential to demonstrate the benefits of AI in products, or they could potentially burst the current AI hype bubble.

1 X (Twitter) of Bartosz Góralewicz

2 AI Overviews: Measuring the Impact on SEO

Featured Image: Paulo Bobita/Search Engine Journal

Editor's P/S:

The article examines the early impact of Google's AI Overviews (AIOs) on organic traffic, particularly in the health sector. AIOs, which provide AI-generated answers to search queries, have been shown to appear in 42% of health-related searches, potentially reducing organic traffic for URLs that are mentioned in these overviews. However, the article also suggests that certain AIOs may drive more traffic to specific URLs based on user intent.

The article highlights the need for further research to fully understand the impact of AIOs on organic traffic and the specific content features that increase the likelihood of being cited in these overviews. It also raises concerns about the potential for misinformation and harmful suggestions in AI-generated responses, especially in sensitive areas like health. Overall, the article provides valuable insights into the evolving landscape of search engine optimization and the potential implications of AI for organic traffic.