‘Implicit code execution’ helps improve accuracy by 30%

Ben Wodecki, Jr. Editor

June 8, 2023

2 Min Read

At a Glance

  • Google unveils an update to its Bard chatbot showing it can more accurately generate code ad responses to math questions.
  • A new method allows Bard to identify prompts that might benefit from code and write it in the background.

Bard, Google’s answer to OpenAI’s ChatGPT, is getting a boost to its ability to handle math queries and coding questions.

Google engineers have utilized a new technique dubbed implicit code execution to help the chatbot detect computational prompts and run code in the background.

The new method allows Bard to respond more accurately to mathematical tasks, coding questions and string manipulation prompts, the company said.

In a blog post, Bard Product Lead Jack Krawczyk and Bard Engineering Vice President Amarnag Subramanya said that the method takes inspiration from a dichotomy in human intelligence—"Thinking, Fast and Slow.”

The approach involves two modes of thinking: System 1, which is “fast, intuitive and effortless” and System 2, which is “slow, deliberate and effortful.”

The pair explained: “In this analogy, large language models can be thought of as operating purely under System 1 — producing text quickly but without deep thought. This leads to some incredible capabilities but can fall short in some surprising ways."

“Traditional computation closely aligns with System 2 thinking: It’s formulaic and inflexible, but the right sequence of steps can produce impressive results, such as solutions to long division.”

Google’s Bard team combined both types of systems to improve the accuracy of Bard's responses. The chatbot now identifies prompts that might benefit from logical code, writes it “under the hood,” executes it and uses the result to generate a more accurate response.

The method has resulted in Bard’s response accuracy to computation-based word and math problems using Google’s internal challenge datasets by around 30%.

Stay updated. Subscribe to the AI Business newsletter

Despite improvements however, the Bard engineers note that the application “won’t always get it right — for example, Bard might not generate code to help the prompt response, the code it generates might be wrong or Bard may not include the executed code in its response.”

Among other Bard improvements announced today include giving users the ability to export tables generated by Bard to Sheets.

Google has been working to improve Bard since its launch in March. In April, it replaced the chatbot’s underlying LaMDA model with PaLM and then later PaLM 2.

Read more about:

ChatGPT / Generative AI

About the Author(s)

Ben Wodecki

Jr. Editor

Ben Wodecki is the Jr. Editor of AI Business, covering a wide range of AI content. Ben joined the team in March 2021 as assistant editor and was promoted to Jr. Editor. He has written for The New Statesman, Intellectual Property Magazine, and The Telegraph India, among others. He holds an MSc in Digital Journalism from Middlesex University.

Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!

You May Also Like