Auditory Prosthetic Group

The Auditory Prosthetic Group of the German Hearing Center, part of the Hannover Medical School, conducts research in the field of signal processing and auditory modeling of implantable hearing prostheses, including cochlear implants and hybrid electroacoustic stimulation devices. The group is highly interdisciplinary, covering acoustics, signal processing, auditory modeling, and psychophysics.

The group started in September 2013 as part of the Hearing4all cluster of excellence.

Research

Signal processing for cochlear implants

Binaural signal processing strategies to improve speech intelligibility and sound localization
Coding of electrical stimulation patterns for bilateral wireless communication
Investigation of the effect of bilaterally synchronized electrical stimulation
Enhancement of interaural cues to improve spatial hearing
Binaural speech processing based on deep learning models

Deep Learning

Automatic enhancement of the singing voice based on deep neural networks
Real-time music processing to improve music perception
End-to-end speech coding and noise reduction strategies for cochlear implants.
Classical Music Remixing with Multi-Head Neural Networks
Separation of music sources informed by the brain

Electric-acoustic stimulation

EAS masking

Every day, more people with residual hearing receive cochlear implants (CIs) to receive electroacoustic stimulation (EAS), which is a combined auditory stimulation delivered electrically by CI electrodes and acoustically via a hearing aid (HA). For EAS users, low-frequency acoustic hearing produces almost no detection for speech in noise, but performance is greatly enhanced when combined with the electrical hearing delivered by the CI. However, it is still unknown how the acoustic and electrical stimulation modalities interact. Clinical experience has shown that some EAS users do not use the HA at all and rely solely on the electrical stimulation. In addition, it has been observed that EAS users require more time to reach maximum performance with the CI alone than do normal CI users. These limitations may have an impact on the definition of EAS implantation criteria. Possible explanations for these limitations are interactions between auditory and electrical stimulation. These interactions become increasingly important for people with large residual hearing because of the greater overlap between electrical and auditory stimulation. For example, the range of hearing losses for which EAS devices are prescribed has been shown to approach the range of age-related hearing loss. Age-related hearing loss, such as EAS, is characterized by loss of audibility in the high frequencies.
With the population in Europe aging with increasing hearing loss problems, new treatments are needed to rehabilitate people with age-related hearing loss. Cognitive decline and hearing loss seem to be related, and new treatments such as EAS are very promising to improve the quality of life of the elderly population. In this work, we aim to characterize the interaction between auditory and electrical stimulation. The characterization will consist of psychophysical, electrophysiological, imaging and backward telemetric measurements in EAS users. Psychophysically, acoustic and electrical interaction will be measured using simultaneous or forward masking experiments. Cone beam computed tomography (DVT) for each study participant will be used to assess insertion depth and electrode positions.
Electrical field imaging (EFI) from backward telemetry of the implant is used to characterize the electrode-neuron interface, and finally electrochochleographic (eChochG) measurements are used to assess the overlap between acoustic and electrical stimulation at the neural level. The outputs of this project are datasets, fitting procedures, and a model that characterizes the interaction between acoustic and electrical stimulation of each EAS user. These results will be used to optimize the configuration of EAS devices and to assess whether the indication for EAS implantation can potentially be extended to people with age-related hearing loss.

EAS Modeling

Thanks to recent improvements in cochlear implant (CI) surgical technology and electrode design, it is now often possible to preserve residual acoustic hearing in the low frequencies during CI insertion in the same ear. CI users with preserved low-frequency hearing receive combined electric-acoustic stimulation (EAS) and show improved speech performance scores, especially in noisy situations, compared with CI users without preserved residual hearing. However, simultaneous acoustic and electrical stimulation has been shown to cause interactions between the two modalities. One form of such interaction is masking. Masking between electrical and acoustic stimulation has been observed in auditory nerve fiber (ANF) spike trains in animals and in electrocochleographic (ECochG) responses and psychophysical experiments in humans (see EAS masking project). To date, it is unclear at which stage of the auditory pathway these masking effects occur (i.e., at the hair cell level, auditory nerve level, or more centrally).

In this project, we are developing a computer model of the acoustically and electrically stimulated auditory nerve to better understand the underlying interaction mechanisms in EAS users. Existing models simulate ANF spiking to acoustic or electrical stimulation alone.

Our idea is to couple a model of acoustically evoked spiking activity with a model of electrically induced spikes to simulate the electric-acoustically stimulated auditory nerve. Our current implementation shows appropriate spiking responses to both acoustic stimulation of the remaining hair cells and electrical stimulation of the ANF, as well as an inhibitory interaction between the two modalities.

Binaural fusion between electrical and auditory stimulation in bimodal CI subjects

Bimodal stimulation, consisting of electrical stimulation with a cochlear implant (CI) and acoustic stimulation with a contralateral hearing aid (HA), can improve speech perception in noise. Currently, at Hannover Medical School, more than 70% of newly implanted CI subjects have some type of residual hearing in the contralateral ear of the CI.

However, a wide variability in the benefits of bimodal hearing was observed, with some individuals even experiencing interference with this type of stimulation. This variability has been associated with binaural fusion between the CI and the HA. This work focuses on characterizing the basic mechanisms that enable binaural fusion between electrical and auditory stimulation in bimodal CI users through basic psychoacoustic and electrophysiological measurements.

Based on the fundamental understanding of bimodal fusion, this project investigates novel CI armatures to improve fusion between electrical and auditory stimulation. The evaluation of the CI armatures will be assessed using a novel cortical electrophysiological measure and behavioral measures to assess the benefit of the bimodal input.

Closed cochlear implants

Selective attention decoding

The cochlear implant (CI) acts as a type of artificial cochlea that converts acoustic signals detected by a microphone into electrical impulses, bypassing the damaged structures of the ear and directly stimulating the auditory nerve. Most CI users get good speech understanding without background noise.

However, CI users still have difficulty understanding speech in more challenging listening environments with multiple speakers, background noise, and reverberation (i.e., The Cocktail Party Problem). In such situations, normal-hearing listeners (NHs) may focus on a target speaker and effectively suppress other speakers present. This effect is referred to as selective attention. The ability to participate selectively is likely to be impaired in CI users and may be one of the reasons for speech comprehension limitations in difficult listening environments.

Selective attention can be further decoded by designing spatiotemporal filters, also called decoders, which are used to reconstruct speech from EEG data. The goal of this project is to investigate different types of decoders, including linear forward-backward models or nonlinear models such as deep neural networks. In addition, various methods for recording and further decoding selective attention will be investigated. Promising recording systems include the cEEGrid system, which is a wearable EEG recording device or an invasive electrode on the temporal lobe.

Objective measurements of speech perception

We are developing methods to objectively study speech perception of CI users by EEG. The ultimate goal of this project is to improve the fitting process of CIs, especially in children or disabled adults. We are using the following measures:
- Cortical auditory evoked potential (CAEP)
- Acoustic Change Complex (ACC)
- Auditory Steady State Response (ASSR).

Music and cochlear implants

Separation of music sources with deep neural networks

Music samples of the Real-time Music Source Separation Experiment. In this experiment, a multilayer perceptron was used to separate the vocal voice from the instrumental accompaniment to remix the music track. It has been shown that CI users enjoy music more when the singing voice is enhanced with respect to the background instruments. Our results indicate that CI users enjoy music more when the singing voice is enhanced by 8 dB with respect to the background accompaniment.

MusIC: Making music more accessible for cochlear implant users

Enhancement of singing voice of music to improve music perception for CI users
Real-time separation of music sources to enhance vocals
Parameterization of CI sound coding strategies for vocal music
Sound coding strategy for music

Music4u

Difficult to enjoy complex music such as classical pieces for CI users
Investigating which musical component (i.e., melody) in instrumental classical music CI users want to improve
Testing whether selective attention decoding is possible for CI users
Feeding EEG data into neural networks to improve source separation of the attended sound source

GROUP LEADER

Prof. Dr. -Ing.

Waldo Nogueira

Group management

Ear, Nose and Throat Clinic

Nina Aldag

PhD student

Jonas Althoff

PhD student

Franklin Alvarez

PhD student

Hanna Dolhopiatenko

PhD student

Tom Gajecki

PhD student

Daniel Kipping

PhD student

Dr.-Ing.

Benjamin Kruger

Research Engineer

2021

A sound coding strategy based on a temporal masking model for cochlear implants Kludt E, Nogueira W, Lenarz T, Büchner A PlosOne 8;16(1):e0244433. doi: 10.1371/journal.pone.0244433.

A subjective and objective evaluation of a codec for the electrical stimulation patterns of cochlear implants Hinrichs R, Gajecki T, Ostermann J, Nogueira W The Journal of the Acoustical Society of America. 149, 1324.

Amplitude growth functions of auditory nerve responses to electric pulse stimulation with varied inter-phase gaps in cochlear implant users with ipsilateral residual hearing Imsiecke, A. Büchner, T. Lenarz, W. Nogueira (2021). Trends in Hearing (Accepted).

Assessing the relationship between neural health measures and speech performance with parallel electrical stimulation in cochlear implant listeners Langner F, Arenberg JG, Büchner A, Nogueira W (Unde Review).

Effect of Channel Interaction on Vocal Cue Perception in cochlear implant users. Nogueira W, El Boghdady N, Gaudrain E, Baskent D Ear and Hearing (Under Review).

2021

Localization of Cochlear Implant Electrode Contacts using Particle Belief Propagation on Cone Beam Computer Tomography Data-Sets Hachmann H, Krüger B, Rosenhahn B, Nogueira W IEE ISBI

Phantom Stimulation for Cochlear Implant Users with Residual Low-Frequency Hearing, Ear and Hearing, Krüger B, Büchner A, Nogueira W Ear and Hearing, (Under Review).

The Effect of Synchronized Linked Band Selection on Speech Intelligibility of Bilateral Cochlear Implant Users Gajecki T, Nogueira W Hearing Research (Under Review).

Amount of frequency compression in bimodal cochlear implant users is a poor predictor for audibility and spatial hearing, Journal of Speech, Language, and Hearing Research. Sharma S, van Wanrooij MM, van Opstal AJ, Chalupper J, Mens LHM, Nogueira W Journal of Speech, Language, and Hearing Research. (Under Review).

2020

The Effect of Synchronized Linked Band Selection on Speech Intelligibility of Bilateral Cochlear Implant Users Gajecki T, Nogueira W. Hearing Research, 2020.

The role of electroneural versus electrophonic stimulation on psychoacoustic electric-acoustic masking in cochlear implant users with residual hearing. Kipping D, Krüger B, Nogueira W. Hearing Research 395, 108036. epub 2020 Sep 15.

Design and Evaluation of a Real-Time Audio Source Separation Algorithm to Remix Music for Cochlear Implant Users. Tahmasebi S, Gajȩcki T, Nogueira W. Frontiers in Neuroscience. 14, 434. doi: 110.3389/fnins.2020.00434. epub 2020 May 14.

Psychoacoustic and electrophysiological electric-acoustic interaction effects in cochlear implant users with ipsilateral residual hearing. Imsiecke M, Büchner A, Lenarz T, Nogueira W. Hear Res. 2020 Feb; 386:107873. doi: 10.1016/j.heares.2019.107873. epub 2019 Dec 18.

Psychoacoustic and electrophysiological electric-acoustic interaction effects in cochlear implant users with ipsilateral residual hearing. Imsiecke M, Büchner A, Lenarz T, Nogueira W. Hear Res. 2020 Feb;386:107873. doi: 10.1016/j.heares.2019.107873. epub 2019 Dec 18.

Toward Decoding Selective Attention From Single-Trial EEG Data in Cochlear Implant Users. Nogueira W, Cosatti G, Schierholz I, Egger M, Mirkovic B, Büchner A. IEEE Trans Biomed Eng. 2020 Jan;67(1):38-49. doi: 10.1109/TBME.2019.2907638. epub 2019 Mar 26.

Electric-acoustic interaction measurements in cochlear-implant users with ipsilateral residual hearing using electrocochleography. Krüger B, Büchner A, Lenarz T, Nogueira W. J Acoust Soc Am. 2020 Jan;147(1):350. doi: 10.1121/10.0000577.

Perception and prediction of loudness in sound coding strategies using simultaneous electrical stimulation F. Langner, C. M. McKay, A. Büchner, W. Nogueira (2020)., Hearing Research, 398, https://doi.org/10.1016/j.heares.2020.108091.Code.

Evaluation of an adaptive dynamic compensation system in cochlear implant listeners F. Langner, A. Büchner, W. Nogueira (2020). Trends in Hearing, doi:10.1177/2331216520970349.

Effect of spectral contrast enhancement on speech-on-speech intelligibility and voice cue sensitivity in cochlear implant users El Boghdady; W. Nogueira; E. Gaudrain; D. Baskent (2020). Ear and Hearing, doi: 10.1097/AUD.0000000000000936.

Amplitude growth of intracochlear electrocochleography in cochlear implant users with residual hearing B. Krüger, A. Büchner, T. Lenarz, W. Nogueira The Journal of the Acoustical Society of America, 147, 1147, https://doi.org/10.1121/10.0000744.

Implantable Neuroamplifer for ECoG using Flexible and Biocompatible Technology Marcoleta JP, Nogueira W, Becerra Yoma N, Wuth J, Jakimovski F., Fuenzalida VM, Doll T. Physica Status Solidi A: Applications and Materials Science, https://doi.org/10.1002/pssa.201900830.

Distributed mixed signal demultiplexer for electrocorticography electrodes Marcoleta JP, Nogueira W, Doll T Biomedical Physics & Engineering Express, 6, 055006.

2019
Interaction Between Electric and Acoustic Stimulation Influences Speech Perception in Ipsilateral EAS Users. Imsiecke M, Krüger B, Büchner A, Lenarz T,Nogueira W. Ear Hear. 2019 Oct 7. doi: 10.1097/AUD.00000000000807. [Epub ahead of print].

Towards Decoding Selective Attention in Normal Hearing Listeners and Bilateral Cochlear Implant Users With Concealed Ear EEG. Nogueira W, Dolhopiatenko H, Schierholz I, Büchner A, Mirkovic B, Bleichner MG, Debener S. Front Neurosci. 2019 Jul 18;13:720. doi: 10.3389/fnins.2019.00720. eCollection 2019.

Making Music More Accessible for Cochlear Implant Listeners: Recent Developments Nogueira W, A. Nagathil and R. Martin IEEE Signal Processing Magazine, 36, 1, 115-127.

Decoding Selective Attention in Normal Hearing Listeners and Bilateral Cochlear Implant Users with Concealed Ear EEG Nogueira W, Dolhopiatenko H, Schierholz I, Buechner A, Mirkovic B, Bleichner MG , Debener S Frontiers in Neuroscience, 13:720.

Artificial Speech Bandwidth Extension Improves Telephone Speech Intelligibility Nogueira W, Abel J, Fingscheidt T Journal of the Acoustical Society of America, 145.3, 1640-1649.

Psychoacoustic and Electrophysiological Electric-Acoustic Interaction Effects in Cochlear Implant Users with Ipsilateral Residual Hearing Imsiecke M, Büchner A, Lenarz T, Nogueira W Hearing Research, 386, https://doi.org/10.1016/j.heares.2019.107873.

2018
Electric-acoustic forward masking in cochlear implant users with ipsilateral residual hearing. Imsiecke M, Krüger B, Büchner A, Lenarz T, Nogueira W. Hear Res. 2018 Jul;364:25-37. doi: 10.1016/j.heares.2018.04.003. Epub 2018 Apr 9.

The effects of electrical field spatial spread and some cognitive factors on speech-in-noise performance of individual cochlear implant users-A computer model study. Jürgens T, Hohmann V, Büchner A, Nogueira W. Hear Res. 2018 Mar;359:64-75. doi: 10.1016/j.heares.2017.12.019. Epub 2017 Dec 28.

Perceptual changes with monopolar and phantom electrode stimulation. Klawitter S, Landsberger DM, Büchner A, Nogueira W. PLoS One. 2018 Apr 13;13(4):e0193842. doi: 10.1371/journal.pone.0193842. eCollection 2018.

A Synchronized Binaural N-of-M Sound Coding Strategy to Improve Speech Intelligibility in Noise of Bilateral Cochlear Implant Users Gajecki T, Nogueira W**. ITG Speech Communication, 2018.

Flexible High Density Active Neural Implants combining a Distributed Multiplexing Transceiver Architecture with Biocompatible Technology J. P. Marcoleta, W. Nogueira, U. Floriep, T. Doll Physica Status Solidi A: Applications and Materials Science, 215, 15, https://doi.org/10.1002/pssa.201700134.

Deep Learning Models to Remix Music for Cochlear Implant Users T. Gajecki and W. Nogueira Journal of the Acoustical Society of America, 143, 3602.

Contralateral Suppression of Human Hearing Sensitivity in Single-sided Deaf Cochlear Implant Users W. Nogueira, B. Krüger, A. Büchner, E. Lopez-Poveda Hearing Research,373, 121-129. https://doi.org/10.1016/j.heares.2018.06.001

Spike-conducting integrate-and-fire model G. Ashida, W. Nogueira eNeuro 27 August 2018, ENEURO.0112-18.2018; DOI: https://doi.org/10.1523/ENEURO.0112-18.2018.

2017
**Environmental sound recognition using short-time feature aggregation G. Roma, P. Herrera, W. Nogueira Journal of Intelligent Information Systems, 1-19, doi:10.1007/s10844-017-0481-4.

Simultaneous masking between electric and acoustic stimulation in cochlear implant users with residual low-frequency hearing. Krüger B, Büchner A, Nogueira W. Hear Res. 2017 Jun 30. pii: S0378-5955(17)30042-4. doi: 10.1016/j.heares.2017.06.014. [Epub ahead of print].

Adding simultaneous stimulating channels to reduce power consumption in cochlear implants. Langner F, Saoji AA, Büchner A, Nogueira W. Hear Res. 2017 Mar;345:96-107. doi: 10.1016/j.heares.2017.01.010. Epub 2017 Jan 16.

Loudness and pitch perception using Dynamically Compensated Virtual Channels. Nogueira W, Litvak LM, Landsberger DM, Büchner A. Hear Res. 2017 Feb;344:223-234. doi: 10.1016/j.heares.2016.11.017. Epub 2016 Dec 7.

Development of a parametric model of the electrically stimulated auditory nerve **Nogueira W, Ashida G. Lecture Notes in Applied and Computational Mechanics, Springer Verlag, 84: 349-362.

2016
Validation of a Cochlear Implant Patient-Specific Model of the Voltage Distribution in a Clinical Setting. Nogueira W, Schurzig D, Büchner A, Penninger RT, Würfel W. Front Bioeng Biotechnol. 2016 Nov 23;4:84.eCollection 2016.

Spectral contrast enhancement improves speech intelligibility in noise for cochlear implants. Nogueira W, Rode T, Büchner A. J Acoust Soc Am. 2016 Feb;139(2):728-39. doi: 10.1121/1.4939896.

Optimization of a Spectral Contrast Enhancement Algorithm for Cochlear Implants Based on a Vowel Identification Model. Nogueira W, Rode T, Büchner A. Adv Exp Med Biol. 2016;894:93-104. doi: 10.1007/978-3-319-25474-6_11.

Remixing music using source separation algorithms to improve the musical experience of cochlear implant users Pons J, Rode T, Janer J, Nogueira W. Journal of the Acoustical Society of America, 140, 6:4338.

Effects of age on long term memory for degraded speech C. Thiel, J. Özyurt, W. Nogueira**, S. Puschmann Front. Hum. Neuroscience, 10.

2015
Evaluation of a Transient Noise Reduction Algorithm in Cochlear Implant Users. Dyballa KH, Hehrmann P, Hamacher V, Nogueira W, Lenarz T, Büchner A. Audiol Res. 2015 Jun 11;5(2):116. doi: 10.4081/audiores.2015.116. eCollection 2015 Jun 11.

Stimulating on multiple electrodes can improve temporal pitch perception. Penninger RT, Kludt E, Büchner A, Nogueira W. Int J Audiol. 2015 Jun;54(6):376-83. doi: 10.3109/14992027.2014.997313. epub 2015 Jan 29.

Design and evaluation of a cochlear implant strategy based on a "phantom" channel. Nogueira W, Litvak LM, Saoji AA, Büchner A. PLoS One. 2015 Mar 25;10(3):e0120148. doi: 10.1371/journal.pone.0120148. eCollection 2015.

2011
Use of hearing aid preprocessing techniques with cochlear implant users W. Nogueira, L. Hoepner, V. Hamacher, Th. Lenarz, A. Buechner International Journal of Pediatric Otorhinolaryngology, 75, issue (May, 2011): 94. ISSN: 0165-5876 DOI: 10.1016/S0165-5876(11)70482-7, Elsevier Science.

2010
Speech Recognition Technology in CI Rehabilitation Nogueira W, Vanpoucke F, Dykmans P, De Raeve L, Van Hamme H, Roelens J (2010). Cochlear Implant International, 11, 1: 449-453.

2009
**Signal Processing Strategies based on Current Steering for Cochlear Implants Nogueira W, Litvak L, Edler B, Ostermann J, Buechner A (2009). EURASIP Journal on Applied Signal Processing, Special Issue on Hearing Instrumentation, 531213.

2007
Results from a Psychoacoustic Model-based Strategy for the N-24 and Freedom Cochlear Implants Buechner A, Nogueira W, Edler B, Battmer R. D. , Lenarz Th. Otology and Neurotology, 29, 2:189-92.

Current Steering and Results From Novel Speech Coding Strategies Buechner A, Brendel M, Krüeger B, Frohne-Büchner C, Nogueira W, Edler B, Lenarz T Otol Neurotol., 29, 2:203-207.

2005
A Psychoacoustic "NofM"-type Speech Coding Strategy for Cochlear Implants. Nogueira W, Buechner A, Lenarz T, Edler B. Journal on Applied Signal Processing, Special Issue on DSP in Hearing Aids and Cochlear Implants, 127, 18, 3044-3059.

Keywords

Electric Acoustic Interaction EAS-Masking & Modeling, EEG Neural Tracking, Music Signal Processing, Binaural Sound Coding Strategies,

Contact

Prof. Dr.-Ing. Waldo Nogueira

+49 (0)511 532 8025

Nogueiravazquez.waldo(at)mh-hannover.de

German Hearing Center
Karl-Wiechert-Allee 3
30625 Hanover

Specials

2022

Prof. Waldo Nogueira Vazques, expert in hearing prosthetics receives prestigious ERC grant from the European Union

For his project "READIHEAR ", the scientist receives the "ERC Consolidator Grant" of the European Research Council (ERC), one of the highest scientific grants of the European Union earmarked for excellence. He will be supported with a total of around two million euros over five years.

More additional info

Ongoing Projects
2021
music4u: Funding source: DFG Individual Grant
music4all Funding source: DFG Cluster of Excellence Hering4all
MTALK Funding source: Oticon Medical
BiMoFuse Funding source: William Demant Foundation
Finalized Projects

2018
Coding of Excitation Patterns Funding source: DFG Individual Grant

2017
Electrode Nerve Interface Funding source: DFG Individual Grant
Electric Acoustic Stimulation Funding source: MED-EL

Back to Research Groups