Generative AI Helps Google Redefine Search, Google I/O 2024

Google is using generative AI to enhance its search experience. The company revealed an updated search overview and teased an upcoming AI agent project.

Announced at its annual I/O event, Google has expanded access to AI Overviews, a feature where the company’s flagship Gemini model generates natural-sounding answers to user search queries.

Google recently piloted AI Overviews in the U.K. but will now expand it to all Google search users in the U.S. before a later wider rollout.

The AI-powered search feature gives users a general response to their queries and includes relevant links to explore further.

Previously, Google search users had to ask individual questions and piece together an overall answer.

AI Overview can handle more complex, multi-layered queries to make searching easier. Users can now ask questions like “Local leisure center that offers pilates classes for women after 6 p.m.”

“There's so much innovation happening in search thanks to Gemini,” said Google CEO Sundar Pichai. “We can create much more powerful search experiences, including within our products.”

During the I/O keynote, Pichai said AI Overviews have led to increased use of search, with the generative AI feature having already generated billions of responses.

Google plans to make the feature available to more than a billion users by the end of the year.

Related:Google Expands Gemini Lineup With Large, Small Model Updates at Google I/O 2024

Use AI to Search Photos

Google’s improved search capabilities have also been expanded to its other services.

Its flagship Gemini model is powering new search abilities in Google Photos, allowing users to search through their library of images using natural language.

The Ask Photos feature lets users sift through images by asking queries like “What themes have we had for Lena’s birthday parties?”

Ask Photos returns photos from the user’s library and provides a detailed text response based on the query.

“Gemini goes beyond a simple search, recognizing different contexts,” Pichai said. “You can really take it all in and relive amazing memories all over again.”

Ask Photos will be made available to Google Photos users later this summer.

Google Building AI Agents

While AI Overviews and Ask Photos are coming to Google users soon, I/O gave a glimpse at what the company is working on in the background: AI agents.

Under the working name Project Astra, Google is attempting to build a universal AI agent, an assistant tool that can perform multiple tasks.

Google DeepMind is working on the project, its CEO, the newly knighted Sir Demis Hassabis, unveiled a demo of the project’s work so far.

Related:Google Unveils Next-Gen Video, Image, Music Models at Google I/O 2024

Using a smartphone, the user points Astra at objects in an office environment, asking it various questions like: “What machines make noise in the room?”

The AI was able to analyze the camera feed and pick out objects, generating a response to queries through natural-sounding audio.

Astra was even able to recall where the user left their glasses.

“For a long time, we’ve wanted to build a universal AI agent that can be truly helpful in everyday life,” said Hassabis. “Our work making this vision a reality goes back many years. It's why we made Gemini multimodal from the very beginning.

The Google DeepMind CEO said the engineering challenge developing Astra has been trying to cut down response times so the bot can answer queries conversationally.

“Building on our Gemini model, we developed agents that can process information faster by continuously encoding video frames, combining the video and speech input into a timeline of events and caching this for efficient recall,” Hassabis said. “We've also enhanced how they sound with a wider range of intonations. These agents better understand the context you're in and can respond quickly in conversation, making the pace and quality of interaction feel much more natural.”

While it’s a project that remains in development, Hassabis said Astra will find its way into Google products including the Gemini app for Android and iOS “later this year.”

Related:Google Custom TPUs Power Faster AI Workloads, Google I/O 2024

The Project Astra showcase was similar to demos of new ChatGPT capabilities at OpenAI’s Spring Update event just a day earlier.

“Considering yesterday's announcements from OpenAI and today's updates from Google, it's increasingly evident that traditional search will likely be replaced long-term by more natural language interactions and chatbot experiences through text, voice, or imagery,” said Damian Rollison, director of market insights at SOCi. “Search is entering a new era.”