SymbolPath: A Continuous Motion Overlay Module for Icon-Based Assistive Communication

Abstract

Individuals with severe speech im_pairments often rely on augmentative and alternative communication (AAC) sys_tems to interact with others. Individuals with poor literacy skills or compromised motor functions typically use icon-based AAC systems, which enable faster and less effortful message formulation compared to letter-based systems. Yet navigating to and selecting from a series of icons to formulate a message can be cumbersome and prohibitively slow. Moreover, current icon-based AAC devices require a series of discrete movements — users must lift their hands or arms repeatedly to select individual icons and avoid selecting multiple icons at once. Concomitant upper limb motor impairments common to this user group make such repetitive and precise movements slow and fatiguing. We have developed an icon-based AAC system called SymbolPath that enables continuous motion icon selection through the use of path analysis and a free-order language model. Message formulation using SymbolPath consists of drawing a continuous line through a set of desired icons instead of selecting each icon individually. The system determines the most likely subset of desired icons on the drawn path and rearranges them to form a meaning_ful and grammatically correct sentence, which is then spoken aloud using text-to-speech (TTS) synthesis. In addition to demonstrating the SymbolPath module, we will present initial usability test results and plans for iterative modifications to the system.