Multimodal training using pitch gestures improves Mandarin tone recognition in noise for children with cochlear implants

The present study investigates whether incorporating manual pitch gestures into audiovisual training can enhance the recognition of Mandarin lexical tones in prelingually deaf children with cochlear implants (CIs). Building on prior research that demonstrated the benefits of hand gestures in second-language tone learning, the study hypothesized that multimodal training integrating pitch gestures would be more effective than auditory-visual training alone. Eighteen preschool-aged CI users (15 with bimodal hearing) were assigned in equal numbers to either: (1) an experimental group receiving combined auditory-visual training with pitch gestures, or (2) a control group receiving auditory-visual training alone. Tone-identification tests were conducted before, during, and after eight training sessions. Both groups demonstrated similar performance before training. However, after training, the experimental group showed significantly greater improvement in tone recognition in challenging noise conditions, compared to the control group. These results underscore the advantages of integrating manual pitch gestures into auditory-visual training, suggesting that integrating pitch gesture production into rehabilitation could enhance outcomes for CI users, especially in tonal languages, such as Mandarin.