Towards more robust speech interactions for deaf and hardof hearing users

Mobile, wearable, and other ubiquitous computing devices are increasingly creatinga contextin which conventionalkey-board and screen-based inputs are being replacedinfavorof more natural speech-based interactions. Digital personal as-sistants use speech to control a wide range of functionality, from environmental controls to information access. However, manydeaf and hard-of-hearing users have speech patterns that vary from those of hearing users due to incomplete acoustic feedback from their own voices. Because automatic speech recognition (ASR) systems are largely trained using speech from hearing individuals, speech-controlled technologies are typically inaccessible to deaf users. Prior work has focused on providing deaf users access to aural output via real-time captioning or signing, but little has been done to improve users' ability to provide input to these systems' speech-based interfaces. Further, the vocalization patterns of deaf speech often makeaccurate recognition intractable for both automated systems and human listeners, making traditional approaches to mitigate ASR limitations, such as human captionists, less effective.To bridge this accessibilitygap, weinvestigate the limitations of common speech recognition approaches and techniques-both automatic and human-powered-when ap-plied to deaf speech. We then explore the effectiveness of an iterative crowdsourcing workfow, and characterize the po-tential for groups to collectively exceed the performance of individuals. This paper contributes a better understanding of the challenges of deaf speech recognition and provides insights for future system development in this space.