Developing auditory-perceptual judgment reliability in otolaryngology residents

S. Misono, A. L. Merati, T. L. Eadie

Research output: Contribution to journalArticlepeer-review

8 Scopus citations


Objectives: To determine how standard residency training and educational background affect otolaryngology resident auditory-perceptual judgments compared with inexperienced listeners. A secondary aim was to assess the impact of a brief training session on the reliability of judgments among otolaryngology residents. Study Design: Mixed cross-sectional and cohort study. Methods: Twenty adult dysphonic and four normal speaker samples were selected from a database. Fifteen listeners at various stages of residency in otolaryngology judged all samples for breathiness and roughness using visual analog scales. Fifteen inexperienced listeners judged the same samples. Subsequently, 12 otolaryngology residents underwent a brief training session. During training, listeners were provided definitions of rating dimensions and accuracy feedback while rating 20 novel (training) stimuli. The feedback included averaged responses from experienced clinicians who had previously evaluated the same speech samples. The residents then completed posttraining evaluations. Results: As a group, otolaryngology resident listeners had significantly better interrater agreement for judgments of roughness compared with inexperienced listeners (P < 0.05) and trended toward better interrater agreement for judgments of breathiness (P = 0.058). Posttraining, no significant improvement in interrater agreement for judgments of roughness was observed; however, a significant improvement was found in interrater agreement of judgments of breathiness (P < 0.05). Conclusions: Compared with inexperienced (untrained) listeners, otolaryngology resident listeners had better interrater agreement overall for judgments of dysphonia. A short training module was associated with improved reliability in resident evaluation of breathiness. Results have implications for improving and standardizing resident evaluation of dysphonia.

Original languageEnglish (US)
Pages (from-to)358-364
Number of pages7
JournalJournal of Voice
Issue number3
StatePublished - May 2012

Bibliographical note

Funding Information:
We appreciate the participation of the listeners in our study and the assistance from student researchers in the vocal function laboratory at the University of Washington. We are also grateful for funding support provided by American Speech-Language-Hearing Foundation (New Investigator Grant, T.E.) supplemented by the Special Interest Division 3: Voice and Voice Disorders of the American Speech-Language-Hearing Association .


  • Auditory-perceptual measures
  • Dysphonia
  • Otolaryngology residency
  • Training and education


Dive into the research topics of 'Developing auditory-perceptual judgment reliability in otolaryngology residents'. Together they form a unique fingerprint.

Cite this