Using Speech Recognition to Enhance Communication Capabilities for Individuals with Physical Disabilities

Project: Research project

Project Details


This project builds on the PI's previous efforts, placing a special emphasis on communications-oriented activities and individuals with physical disabilities. The research will focus on improving user interaction with speech recognition systems. Previously, the PI developed a novel error specification technique that uses confidence scores to facilitate the recognition error correction process. While earlier efforts by others to use confidence scores for this purpose had largely been unsuccessful, the PI's initial evaluation of his technique has yielded promising results; whereas earlier efforts relied on identifying specific words that were incorrect, the PI's technique can prove beneficial even if it is only possible to identify the regions where errors may exist. In the new project, the PI will integrate confidence scores with techniques from natural language processing, to facilitate the process of correcting recognition errors. He will develop novel interaction techniques and models, that support new interaction scenarios and allow for better understanding of the relationship between users and speech recognition systems. Questions to be addressed include the following: Can we expand existing models to effectively describe interactions between users and speech recognition systems when error correction activities are explicitly separated from dictation? How can we support the process of finding and correcting recognition errors when these activities are temporally separated from dictation? How do the abilities and limitations of individuals (e.g., traditional users and individuals with various disabilities) affect their strategies when completing tasks that involve deferred error correction?

Broader Impacts: This research extends an existing collaboration between UMBC's Information Systems Department and the Workforce and Technology Center (WTC). Speech recognition can be a critical technology for individuals with physical disabilities. While these individuals have been shown to employ different strategies as compared to traditional computer users, limited research has focused on the use of speech recognition technologies by this important group of potential users. Therefore, by explicitly investigating the challenges these individuals encounter when using speech recognition technologies, this project will directly impact this underrepresented group of users, and will furthermore facilitate greater independence and new employment opportunities for them. Because a broad range of users will be included in this project, the results will also benefit traditional computer users with no physical disabilities. Through the collaboration with the WTC, the PI will distribute his results via appropriate publications within the rehabilitation community, in addition to the HCI and ASR communities.

Effective start/end date9/15/038/31/08


  • National Science Foundation: $415,302.00


Explore the research topics touched on by this project. These labels are generated based on the underlying awards/grants. Together they form a unique fingerprint.