Improving Accessibility of Digital Devices for Visually Impaired People Using Tactile-Kinesthetic Microgestures

Illustration by ChatGPT of tactile micro-gestures (but it is actually missing a microgesture :))

I had the chance to participate to the PhD Defense Jury of Dr. Faisandaz in November on the digital accessibility of tactile interfaces thanks to the use of Thumb-to-Finger (TTF) microgestures—precise thumb movements on the same hand’s fingers—to enhance tactile interactions. It was very interesting, here is a little summary of his PhD.

What is a microgesture?

Thumb-to-Finger (TTF) microgestures are movements performed by the thumb on different parts of the same hand’s fingers. These include gestures like a "tap" (a quick contact, similar to a mouse click) or a "swipe" (a sliding motion along the finger). These gestures leverage the hand's natural dexterity and can be performed without visual feedback, making them particularly suited to the needs of visually impaired users.

Context and Objectives

Currently, visually impaired and blind people rely on applications such as VoiceOver (iOS) or TalkBack (Android) on their smartphones, which, while useful, have limitations: complex interactions, a steep learning curve for shortcuts, and inconsistent gesture recognition. This thesis aimed to improve accessibility and usability of digital devices by introducing TTF microgestures as a complementary interaction method.

Key Contributions

1. Feasibility Assessment: A thorough analysis of 33 common microgestures tested without visual feedback demonstrated their expressiveness and precision.

2. Development of a Gesture Vocabulary: A set of 8 optimized microgestures was created specifically for touchscreen devices.

3. Practical Demonstrations: Applications like audio-tactile map exploration, text selection, and copy-paste tasks highlighted the utility of these gestures.

4. Innovative Prototypes: Three glove prototypes were designed to capture and interpret TTF microgestures.

   
   

Results

Experiments showed that TTF microgestures enable smoother and faster interactions with digital devices, while being intuitive and easy for visually impaired users to learn. This approach significantly reduces accessibility barriers and fully utilizes the portability of smartphones.

For more information, do not hesitate to read the PhD thesis of Dr. Faisandaz titled: "Increasing input interaction expressiveness using eyes-free multifinger interaction" andAnd big congratulations to him on all his hard work!