Multi-microphone adaptive array augmented with visual cueing.

Paul L. Gibson, Dan S. Hedin, Evelyn E. Davies-Venn, Peggy Nelson, Kevin Kramer

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

We present the development of an audiovisual array that enables hearing aid users to converse with multiple speakers in reverberant environments with significant speech babble noise where their hearing aids do not function well. The system concept consists of a smartphone, a smartphone accessory, and a smartphone software application. The smartphone accessory concept is a multi-microphone audiovisual array in a form factor that allows attachment to the back of the smartphone. The accessory will also contain a lower power radio by which it can transmit audio signals to compatible hearing aids. The smartphone software application concept will use the smartphone's built in camera to acquire images and perform real-time face detection using the built-in face detection support of the smartphone. The audiovisual beamforming algorithm uses the location of talking targets to improve the signal to noise ratio and consequently improve the user's speech intelligibility. Since the proposed array system leverages a handheld consumer electronic device, it will be portable and low cost. A PC based experimental system was developed to demonstrate the feasibility of an audiovisual multi-microphone array and these results are presented.

Original languageEnglish (US)
Title of host publicationProceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
Pages1000-1003
Number of pages4
Volume2012
DOIs
StatePublished - 2012
Event34th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2012 - San Diego, CA, United States
Duration: Aug 28 2012Sep 1 2012

Publication series

NameProceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS
ISSN (Print)1557-170X

Other

Other34th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS 2012
CountryUnited States
CitySan Diego, CA
Period8/28/129/1/12

Fingerprint Dive into the research topics of 'Multi-microphone adaptive array augmented with visual cueing.'. Together they form a unique fingerprint.

Cite this