Useful blunders: Can automated speech recognition errors improve downstream dementia classification?

Changye Li, Weizhe Xu, Trevor Cohen, Serguei Pakhomov

Research output: Contribution to journalArticlepeer-review


Objectives: We aimed to investigate how errors from automatic speech recognition (ASR) systems affect dementia classification accuracy, specifically in the “Cookie Theft” picture description task. We aimed to assess whether imperfect ASR-generated transcripts could provide valuable information for distinguishing between language samples from cognitively healthy individuals and those with Alzheimer's disease (AD). Methods: We conducted experiments using various ASR models, refining their transcripts with post-editing techniques. Both these imperfect ASR transcripts and manually transcribed ones were used as inputs for the downstream dementia classification. We conducted comprehensive error analysis to compare model performance and assess ASR-generated transcript effectiveness in dementia classification. Results: Imperfect ASR-generated transcripts surprisingly outperformed manual transcription for distinguishing between individuals with AD and those without in the “Cookie Theft” task. These ASR-based models surpassed the previous state-of-the-art approach, indicating that ASR errors may contain valuable cues related to dementia. The synergy between ASR and classification models improved overall accuracy in dementia classification. Conclusion: Imperfect ASR transcripts effectively capture linguistic anomalies linked to dementia, improving accuracy in classification tasks. This synergy between ASR and classification models underscores ASR's potential as a valuable tool in assessing cognitive impairment and related clinical applications.

Original languageEnglish (US)
Article number104598
JournalJournal of Biomedical Informatics
StatePublished - Feb 2024

Bibliographical note

Publisher Copyright:
© 2024 Elsevier Inc.


  • Automatic speech recognition
  • Dementia
  • Explainable artificial intelligence
  • Natural language processing

PubMed: MeSH publication types

  • Journal Article
  • Research Support, N.I.H., Extramural


Dive into the research topics of 'Useful blunders: Can automated speech recognition errors improve downstream dementia classification?'. Together they form a unique fingerprint.

Cite this