Too few randomized controlled trials are carried out on AI tech

Ben Wodecki, Jr. Editor

March 14, 2023

3 Min Read

At a Glance

  • Wellcome Trust’s head of data for science and health says AI requires diverse data and people
  • Tariq Khokar warns not to be 'lured in' by the promise of an innovative medical technology if it has not been properly and th

Diversity in AI should encompass both people and the data being used to train models, according to the Wellcome Trust’s head of data for science and health Tariq Khokar.

Khokar warned that data quantity is not as important as quality to make sure AI works well in health care settings.

The data science lead said that a lack of diverse data could mean researchers miss out on potential discoveries. He cited the example of PCSK9, a protein found in the liver that is related to lower cholesterol levels. In white people, this protein only occurs in 0.06% whereas, in black people, it's 2.3%, showing that having a lack of ethnically diverse data could be detrimental to research.

“If you're not training these models on appropriately diverse data, this is the kind of stuff we’re going to expect,” said Khokar, who was speaking at Tech Show London last week.

“We have to think about diversity for the future of AI, both diversity of data and diversity in the people who are working with it and making decisions about it as well.”

Does AI provide any real benefits?

Khokar spoke about what he considers to be the impressive capabilities of AlphaFold, the protein folding model from Google-owned DeepMind.

DeepMind’s model has predicted structures for 200 million proteins, potentially giving drug developers a powerful tool to understand how the body’s proteins relate to diseases.

Khokar said AlphaFold “absolutely crushed” earlier tools used to determine protein sequences, likening it to a far more emphatic result than Liverpool’s recent 7-0 drubbing of Manchester United in Premier League soccer.

But with improved performances made possible by AI tools, Khokar showed that the cost of sequencing a single human genome is coming down. It previously cost $100 million to sequence a single genome but this is expected to drop to around $100, he said, adding that this reduction makes human genomic sequencing a commercially viable prospect.

While Alphafold and AI-powered human genomic sequencing offer viable uses, the lack of testing of other AI health care-focused applications is an issue, according to the speaker.

He pointed to AI-powered apps for mindfulness and wellbeing, like Headspace, and said there’s “almost no regulation or evaluation” of whether these platforms work or not.

According to Khokar, few to no randomized controlled trials (RCT), which measure the effectiveness of a new medical treatment or tool, are carried out on medical AI technology.

He cited figures from renowned American cardiologist Eric Topol that there were reportedly only five RCTs for AI uses in medicine back in 2020. Three years later, Khokar said the number stood nearer to 40, but was far from enough.

“An RCT isn't the only way you can evaluate AI tools, there are other ways of doing it. But in general, it's very easy to be lured in by the story of better performance of cool, new technology, and forget that unless it actually makes an improvement in someone's life, it’s not worth it.”

Khokar’s team at the Wellcome Trust is working to evaluate mindfulness apps, working alongside the U.K. Medicines and Healthcare products Regulatory Agency to develop guidelines and standards for evaluating digital mental health tools. The Wellcome Trust is also working on a project in Vietnam to use digital tools to forecast potential outbreaks of the Dengue virus.

AI Business asked Khokar whether AI technologies in the medical field should be subject to industry-specific legislation and rules. He said that there are going to be industry-agnostic regulations for data governance and data portability, but there are enough specific rules about health and clinical settings that require narrow guidance.

"A lot of the best guidance is built on the existing ways that health technologies are evaluated,” he said. “There are standards like SPIRIT-AI and CONSORT-AI that are built on a set of clinical guidelines for evaluating existing medical technologies.”

Read more about:

Health care

About the Author(s)

Ben Wodecki

Jr. Editor

Ben Wodecki is the Jr. Editor of AI Business, covering a wide range of AI content. Ben joined the team in March 2021 as assistant editor and was promoted to Jr. Editor. He has written for The New Statesman, Intellectual Property Magazine, and The Telegraph India, among others. He holds an MSc in Digital Journalism from Middlesex University.

Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!

You May Also Like