Case Study: Candy Giant Mars Uses AI to Test Which Ads Lead to Sales

Part of a special issue on AI and customer engagement in Management and Business Review, a publication backed by top business schools worldwide. Original title is Mars’ ACE: Using AI and Behavioral Data in Ad Testing with High Correlating Sales

Advertising testing is a time-honored tradition used by marketers to gain insights into a consumer’s response to ads. Typically, research is done using standard survey methods where respondents are asked about ad-recall, ad-preference, or future purchase desire before the ad is launched.

But traditional ad testing is limited by its declarative, survey-style approach; there is a need for an alternate solution that would drive sales and create a sustainable competitive advantage. In this endeavor, Mars developed its own proprietary method of assessing effectiveness by using consumer behavioral methods and applying artificial intelligence (AI).

We utilized behavioral data − such as eye-tracking, facial reactions, skin conductivity, EEG and others – and applied AI algorithms to analyze the data and ultimately link benchmarks to sales success. The result is an 85% accuracy in predicting whether ads would likely lead to a sale or not.

The Mars approach

Mars has been proudly family-owned for over a century. It is this independence as a privately held company that enables us to think along a timeline spanning generations, not fiscal quarters, so we can invest in the long-term future of our business, our people and the planet — all guided by our enduring principles. And our longevity gives us the freedom to try new things, such as developing our own tools to forge new paths.

Related:Case Study: AI for Customer Engagement at Google

In the consumer packaged goods (CPG) industry, Mars is widely recognized for its strong insights culture, an evidence-based approach to marketing decisions and a reluctance to use biased or unscientific approaches to understanding consumers. For more than 15 years, Mars focused its media research on best-in-class behavioral tools. We measured the direct sales impact of media exposures at the household level, using industry-leading single source panels and natural experiments.

In a data-driven agile marketing world, measuring sales impact is important but not a fast enough process to drive agile business decisions. To answer the internal demand for agility, we developed a behavioral proxy measure linked to business performance and which performed with extreme agility. We call it Agile Creative Expertise, or ACE.

ACE and sales predictability

ACE is an ad-testing tool powered by AI that places emphasis on the emotional response of a test audience as they watch a video ad while also considering traditional metrics such as skip and click-through rates. The viewers, which were recruited by Mars partners and are remunerated, consented to be tracked. Their data is anonymized and aggregated when the results are shared with Mars. We tested a few hundred pieces of content for which we already had a clean and reliable in-market sales performance measure (single source). For each of these pieces of content, we collected a second-by-second attention signal, a second-by-second emotion signal, a second-by-second view/skip signal, and a sales measurement.

Related:Mastercard: Using AI to Spot Micro Trends for Effective Customer Engagement

Then we added the data we collected when we completed the largest ever CPG behavioral science study with Nielsen (300 creative executions, tested on participants in seven countries who consented to have their data collected). We used AI on this huge dataset to understand if and how the emotion, attention and view-through signals were predictive of the sales impact. The result was an algorithm that is close to 85% predictive.

In 2019, AI models were continuing to evolve, enabling emotional recognition directly from a mobile device. This facilitated the potential for technology to jump from the lab into the living rooms, kitchens and cars of consumers. After more than six years of testing and re-testing numerous AI-fueled advertising research approaches, Mars Horizon − our Insights team at Mars − launched ACE in 2020.

Validation, considering other solutions

To ensure ACE’s credibility, we had to link it to sales predictability. Without this correlation, the company would have rejected any new approach to advertising research.

To this end, we rated our content from 1-Star (no impact on sales) to 4-Star (more than 15% impact on sales and in the top quintile of all content in the category). Pre-ACE, about 40% of all content we developed was 3- or 4-Star rated. With ACE, this percentage increased to above 70%.
We were able to validate that ACE correctly predicted the sales response in more than 4,000 ads. We are confident that facial response data can provide a proxy metric for sales and, ultimately, the effectiveness of the creative. We are continuing to monitor the correlation with sales on an ongoing basis, and course-correct benchmarks as we get more data.

We found that this testing approach is proving more predictive of in-market success, as well as faster and more affordable than the more traditional methodologies we had been using until now.

In parallel, we had also investigated a number of more technology-based solutions (such as skin conductance, EKG). Our conclusion was that these more-invasive approaches add complexity but do not provide any additional information regarding consumers’ attention and engagement.

Deployment and adoption

ACE was widely embraced throughout the company. In 20 months, we completed more than 1,000 projects in over 25 countries. This helped Mars improve the effectiveness of its digital adverts on TikTok, Facebook, Instagram, YouTube, Amazon and other digital platforms, generating an additional $30 million in sales.

Today, we use ACE as an ad-testing tool to inform our brand’s creative choices in all digital channels. We proved that an execution with a higher ‘Attention’ score will travel across media channels better and will be the best choice to drive sales impact. At the same time, 'Attention' is fully utilized to understand the moments in your creative that are winning consumers’ minds, and those that are confusing them.

Future of AI in ad testing

Through this AI-based approach, we ensure that our ads score high on attention and emotion − this is a ‘guarantee’ that we will increase consumer engagement.

Ultimately, advertising is a mix of science and art. And art is difficult to evaluate properly without the human input. Consumers will always play an important role in future advertising research, and at Mars, we are excited to continue producing adverts and digital content that resonates.

We believe the role AI plays today in ad-testing can only become more valuable. The ability of marketers to optimize an ad will improve as tools proliferate that copy real consumer behaviors, using AI for such things as eye-tracking and to determine attention. And one day, maybe the AI itself will make those optimizations on the fly.