AI powers touchless computers to enable access for all

Big tech companies, students, NHS partner in accessibility initiative

Imagine being able to use a computer without using your hands. Students at University College London have come up with an AI-powered ‘motionless’ computer to do just that – and help people with disabilities.

IBM, Intel, Microsoft and the U.K. National Health Service (NHS) collaborated to develop AI-powered software that runs natively on a computer and can be used on most platforms.

The system, which was demonstrated at IBM’s U.K. headquarters, uses several AI techniques, including speech recognition and hand-gesture recognition to track the user’s movements.

The AI software doesn't require a specific webcam or peripheral to run. The simplicity behind its design is due to the students wanting to develop it for people with limited mobility including those in nursing homes and care facilities.

“It's not just to enable touchless interactions with a computer for during the pandemic, but also (to allow) anyone, whatever your abilities are ... to control a computer,” a student said.

Notably, the software is smart enough to recognize interruptions. Users can place their hand down after using it, raise it again, and the software will continue from where it left off.

UCL students demonstrated an application designed for use in care homes where users can do things ranging from using streaming services to web browsing, all in one place. The students also showed the system navigating map tools, music platforms like Spotify and simply browsing the web.

Another use case is for nurses to annotate diagrams without touching a screen, thereby eliminating hygiene issues. For annotations, users can move their hand closer to make the pen size thicker and vice versa. That usage was due to the incorporation of Windows Inking, the software suite in Windows 10 for sketching.