Penn Engineering and CHOP get $6m to bolster AI models against adversarial attacks

Hoping to make AI less susceptible to confusion caused by anomalous data

Chuck Martin, Editorial Director AI & IoT

November 6, 2020

2 Min Read

Hoping to make AI less susceptible to confusion caused by anomalous data

The University of Pennsylvania’s School of Engineering and Applied Science and the Children’s Hospital of Pennsylvania (CHOP) have been awarded a $6 million grant to develop techniques that can defend AI systems against emerging types of cyber attacks.

The Multidisciplinary University Research Initiative grant, a Department of Defense (DoD) program, was based on the research team’s proposal to leverage “insights from human cognitive development to make artificial intelligence systems better at protecting themselves from malicious disruptions.”

The intent is to build the same robust, adaptive learning capabilities exhibited by biological intelligence into AI-based systems, so that they can work with broader categories of information and are not easy to confuse.

Operating in the real world

For example, one potential danger is that instead of directly accessing the software controlling a self-driving car’s accelerator, someone could alter a speed-limit sign, so the car’s AI does not recognize it.

The study aims to benefit researchers and designers of autonomous systems by increasing awareness of the dangers such systems could present when operating in the real world. The research team also plans to create new tools and technologies to reduce risks emerging from the adoption of AI.

While deep learning approaches require a significant amount of labeled data to be effective, their knowledge is often limited to a narrowly focused domain on which they were trained. Their ability to make correct decisions falls apart when applied in novel settings, while humans naturally apply lessons learned in one context to others with no explicit training.

“Robust, concept-learning techniques will assure that trained models operate effectively in the presence of malicious attacks, offering a substantial improvement over the vulnerability of today’s systems that can be easily compromised by even small anomalies,” explained Insup Lee, professor in Penn Engineering’s departments of Computer and Information Science and Electrical and Systems Engineering.

Lee is also the director of the Precise Center, which is researching the technical strategies for advanced safety and security of cyber-physical systems.

About the Author(s)

Chuck Martin

Editorial Director AI & IoT

Chuck Martin, a New York Times Business Bestselling author, futurist and columnist, is Editorial Director at Informa Tech, home of AI Business, IoT World Today and Enter Quantum. Martin has been a leader in emerging digital technologies for more than two decades. He is considered one of the foremost Internet of Things (IoT) experts in the world and his latest book is titled "Digital Transformation 3.0" (The New Business-to-Consumer Connections of The Internet of Things).  He hosts a worldwide podcast titled “The Voices of the Internet of Things with Chuck Martin,” where he converses with top executives from the companies driving the Internet of Things.

Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!

You May Also Like