AI model drift is the no. 1 hurdle to adopting responsible AI

Deborah Yao, Editor

January 26, 2023

2 Min Read

The adoption of AI in financial services has expanded beyond the ‘early adopters’ to encompass ‘fast followers,’ with initiatives now scaling across business units, according to a recent survey from credit-scoring giant FICO.

AI deployment has broadened beyond internal Centers of Excellence to individual business units, with 52% saying their companies are putting a higher priority on AI today compared to a year ago.

But as more firms rush toward AI adoption, 71% said AI ethics and responsible AI are not yet a core part of their organizational strategy.

“Demand for AI products and tools in financial services is booming. However, we have also found that many financial firms are playing catch-up on responsible AI initiatives,” according to the report, which was conducted in the fourth quarter of 2022.

Responsible AI is seen as bringing in several benefits, with the top one being improved customer experiences (74%) followed by new revenue opportunities (69%) and brand protection (63%).

AI model drift and Zillow

The top hurdle to implementing responsible AI is model drift, according to FICO and partner Corinium in their survey of 100 C-level AI and data executives at U.S. and Canadian financial firms with at least $500 million in annual revenue.

AI drift means the model’s accuracy may be compromised because the data that feeds into it is not monitored regularly and appropriately for changes. Machine learning models learn lessons from the past and apply it to the future. But real-world data does change, and in real time.

AI drift, which was cited by just slightly more than 20% of respondents, still outpaced other culprits: disagreement over how to measure and set metrics around model bias and securing the budget to implement responsible AI.

Zillow offers a cautionary tale about AI model drift, according to the report. In late 2021, the real estate company’s property valuation algorithm wrongly overvalued houses it was buying by more than $500 million. Zillow had failed to adequately adjust its model, trained on past data, to changing real estate conditions. It lost $420 million in a quarter and cut 25% of its staff.

AI drift is relative common in financial services, according to the survey. More than a third of respondents said their governance processes in place to prevent drift were “very ineffective” or “somewhat ineffective.”

However, financial firms do check for bias, with 82% saying they evaluate the fairness of the outcomes, while 40% check for segment bias in the model output and 39% have a codified mathematical definition for data bias.

About the Author(s)

Deborah Yao


Deborah Yao runs the day-to-day operations of AI Business. She is a Stanford grad who has worked at Amazon, Wharton School and Associated Press.

Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!

You May Also Like