Google Refines AI Overviews After Bizarre Responses, Limits Content

Google refines its AI-powered search feature after users started getting weird, oftentimes troubling responses

Ben Wodecki, Jr. Editor

June 3, 2024

2 Min Read
A screenshot of a Google AI Overview response

Google has enhanced its AI Overviews feature with "technical improvements" following a series of viral bizarre responses.

AI Overviews (formerly Search Generative Experience) uses Google’s Gemini model to generate natural-sounding answers to search queries.

Following its recent expansion following Google’s I/O event, users began sharing screenshots of outlandish Overviews responses.

The weird responses included suggesting non-toxic glue be added to pizza, that jumping off the Golden Gate Bridge was a cure for feeling depressed and eating rocks was a good source of minerals.

View post on X

To address the issue, Google said it has made more than a dozen “technical improvements” including an improved detection mechanism for “nonsensical queries.”

Google has also limited the use of user-generated content in responses.

Several of the problematic responses came from user posts on Reddit. Google has access to Reddit user posts through a licensing deal, even though Reddit content is notoriously candid and often offensive.

View post on X

After several responses went viral, Google claims it found an issue in less than one in every 7 million Overview queries and that some of the widely shared generations were fake.

The company says Overviews went through extensive testing before it launched but that issues arose once access was widened. The feature was previously only available in the U.K. as part of a pilot but recently expanded to all Google search users in the U.S.

Related:Generative AI Helps Google Redefine Search, Google I/O 2024

Google offered several explanations for the issues, including that the AI-powered feature misinterpreted language on web pages when searching for relevant information to include in responses.

The feature also struggled with data voids, when there’s a lack of web content for a query. When encountering such a question, Google said Overviews would then look to related satirical content, which would lead to the feature generating inaccurate responses.

To address this, AI Overviews is now limited in its ability to include satirical and humorous content Google says further refinements are being made to searches relating to news and health.

“At the scale of the web, with billions of queries coming in every day, there are bound to be some oddities and errors,” said Liz Reid, Google’s head of search. “We’ll keep improving when and how we show AI Overviews and strengthening our protections, including for edge cases, and we’re very grateful for the ongoing feedback.”

Google’s Overviews issue is the second AI-related problem the company has faced this year. In March it was discovered that Google’s Gemini model was generating images with historical inaccuracies.

Related:Google to Train AI Models on Reddit Posts. What Could Go Wrong?

Read more about:

ChatGPT / Generative AI

About the Author(s)

Ben Wodecki

Jr. Editor

Ben Wodecki is the Jr. Editor of AI Business, covering a wide range of AI content. Ben joined the team in March 2021 as assistant editor and was promoted to Jr. Editor. He has written for The New Statesman, Intellectual Property Magazine, and The Telegraph India, among others. He holds an MSc in Digital Journalism from Middlesex University.

Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!

You May Also Like