Eye Strokes: An Eye-gaze Drawing System for Mandarin Characters

Avatar for Alex Qiang CHEN
Alex Qiang CHEN    
Assistant Professor

Read More 

Avatar for Xuet Ying Tan (tan Tock Seng HOSPITAL)
Xuet Ying TAN (Tan Tock Seng Hospital)    
Researcher

This project uses a technique that uses eye gaze as an input modality to identify Mandarin characters.

Problem Statement: 

Elderly patients with motor neuron disease (MND) and who have limited language literacy, who are unfamiliar with Mandarin Hanyu Pinyin, and have low technological proficiency, struggle to express their thoughts. Their inability to move or communicate makes it difficult to provide appropriate care.

The system captures eye movements as strokes within a character to predict and identify the intended Mandarin word, enabling MND patients to communicate effectively.

https://doi.org/10.1145/3654702 

The interface for a Chinese Character Eye-Gaze Drawing System. It displays a central canvas with a partially drawn Chinese character, surrounded by interactive circular buttons for "Start," "Undo," and "Clear." Along the bottom, there are selectable Chinese character candidates.

 

A diagram illustrating the clinical setup for the eye-gaze system. A patient with Motor Neurone Disease (MND) is lying in bed, using an eye-tracking device mounted on a floor stand to interact with a screen positioned above them.

 

The system uses eye-gaze information to distinguish between ambient gaze, intentional selection, and drawing strokes, enabling users to draw Mandarin characters to communicate their thoughts.