 |
Jan W H Schnupp,
|
A. Personal Statement
I have been pursuing research into the question of how the brain makes sense of sound for well over 30 years.
Hearing is the telepathic sense we all take for granted, allowing us to sense the presence of others in our environment with our eyes closed, or, via the medium of spoken language, to teleport our thoughts into another person’s mind via invisible vibrations. We learn at school that the key to these near miraculous abilities are “sound waves”, minuscule ripples of air pressure that radiate out from physically excited surfaces. But there is nothing “wave like” about our subjective perception of sound. Loudness maps in a very non-linear way onto wave amplitude, and while many people are taught that wave frequency maps onto the percept of pitch (that is whether a note is "high" or "low"), experts know that this is such a gross oversimplification as to be almost completely wrong. The timbre of a sound, perhaps its most distinguishing or characteristic feature, has so many perceptual “dimensions” to it that experts still do not agree on how many dimensional timbre is and what the most important dimensions are. Most of us are also able to perceive sound as highly spatial, being able not only to tell the direction where a sound comes from, but also, to an extent, the distance, and whether the sound travels through a small or large room to our ears. We are also pretty good at guessing the physical dimensions of sound sources, as well as what they are made of. And we are so adept at discovering regular rhythms in sounds that we effortlessly extrapolate them into the future or fill in gaps when part of a sound sequence is obscured by masking noises. All of this illustrates the astonishing sophistication of the central auditory nervous system, which uses still only partly understood neural computations to transform neural impulses triggered by physical sound waves into psychoacoustic, perceived qualities of sound such as loudness, pitch, timbre, rhythm, size or spatial direction, and uses those to create key aspects of the sensory world we inhabit. Improving our still rudimentary understanding of the neural mechanisms underpinning these remarkable transformations is not just a fascinating intellectual challenge, it is a necessity if we want to improve our ability to respond to the needs of hearing impaired cochlear implant users, whose ability to perceive the pitch or the spatial direction of sounds is dramatically diminished, or of those of unfortunate tinnitus sufferers who are disturbed by irritating phantom sounds that have no physical origin and appear to be created entirely in their heads.
My own journey into this area began in 1991 with doctoral and then postdoctoral research under the guidance of Prof Andrew King at the University of Oxford, during which I made pioneering contributions to the use of virtual acoustic space techniques and of linear filter models to better understand the processing of spatial cues in the auditory midbrain and cortex (e.g. Schnupp et al. (2001)) After a brief postdoctoral fellowship under the mentorship of Prof John Brugge at the University of Madison, Wisconsin I returned to Oxford, where I was able to start my own group and turn my attention to the study of neural representations of the pitch and timbre of complex sounds (e.g.Bizley et al. (2013)), and of how these interact with the representations of acoustic space, and how these representations are shaped by experience (e.g. (Schnupp et al. 2006)), or are modulated by context (e.g. Rabinowitz et al. (2013)).
Moving to Hong Kong in 2016 to set up a new laboratory at City University gave me an opportunity to start investigating how we might improve binaural hearing and pitch perception through prosthetic cochlear implants, using studies in animal models (rats) as well as volunteer patients. Human CI patients are known to have only poor sensitivity to interaural time differences (ITDs) as well a reduced sensitivity to termporal pattern cues for pitch, factors which contribute to their reduced auditory abilities compared to normal listeners. Remarkably, we were able to set up and train rats to discriminate ITDs delivered over CIs with a performance that is almost ten times better than that observed in a typical human patient (Rosskothen-Kuhl et al 2021), which gives a clear indication that the devices and methods used in current clinical practice are far from optimal in the manner in whcih they stimulate the auditory pathway. To be better able to turn these insights from the lab into technical advances that may benefit patients I moved to the Chinese University of Hong Kong in 2024, where I can more easily collaborate with clinical colleagues in otolaryngology.
The research in my laboratory combines a variety of techniques including psychoacoustic, behavior testing in humans and animals, in-vivo electrophysiology in animals under acoustic and electric stimulation, EEG studies on human volunteers, signal processing, and the modelling and statistical analysis of behavioral and neural responses. By integrating this wide range of approaches we are able to tackle overarching research questions from how the auditory brain creates and uses predictive coding to process speech and music to how one might improve neuroprosthetic devices to restore a good sense of spatial hearing and musical and lexical pitch to severely deaf patients.
References
Bizley, J. K., Walker, K. M. M., Nodal, F. R., King, A. J. and Schnupp, J. W. H. (2013) Auditory cortex represents both pitch judgments and the corresponding acoustic cues. Curr Biol 23:620-625.
Rabinowitz, N., Willmore, B., King, A. and Schnupp, J. (2013) Constructing Noise-Invariant Representations of Sound in the Auditory Pathway. PLoS Biol. 11:e1001710.
Rosskothen-Kuhl, N., Buck, A. N., Li, K., & Schnupp, J. W. (2021) Microsecond Interaural Time Difference Discrimination Restored by Cochlear Implants After Neonatal Deafness, ELife, 498105; doi: DOI: 10.7554/eLife.59300
Schnupp, J. W., Hall, T. M., Kokelaar, R. F. and Ahmed, B. (2006) Plasticity of temporal pattern codes for vocalization stimuli in primary auditory cortex. J Neurosci 26:4785-95.
Schnupp, J. W., Mrsic-Flogel, T. D. and King, A. J. (2001) Linear processing of spatial cues in primary auditory cortex. Nature 414:200-4.
B. Positions and Honors
Positions and Employment
2024-: Professor in Neuroscience, at the Gerald Choa Neuroscience Institute, Chinese University of Hong Kong.
2016-24: Professor in Neuroscience, at the Department of Biomedical Science, City University of Hong Kong.
2010-16: Professor in Neuroscience at the University of Oxford.
2008-09:Visiting researcher at the Italian Institute of Technology, Genova, Italy, in the laboratory of Dr Stefano Panzeri.
2005-15: Instructor at the Otto Loewi Minerva International Neuroscience Course in intracellular recording techniques, Eilat, Israel.
2002-10: University Lecturer (associate professor) at the Department of Physiology Anatomy and Genetics, University of Oxford, Fellow and Tutor in Medicine at St Peter's College and Stipendiary Lecturer at Oriel College, Oxford
2001- 02: Retained Lecturer in Neuroscience for Trinity College, Oxford
2001- 02: Postdoctoral Research Scientist (RS2) in the laboratory of Dr. A.J. King, Physiology, Oxford
1998:Author of “BrainWare”, a Software application for sensory electrophysiology, distributed by Tucker Davis Technology, Alachue, Florida.
1998-99:Visiting Honorary Research Fellow at the University of Wisconsin at Madison, collaborating with Prof. J. Brugge, studying the representation of moving and of natural sounds in the auditory midbrain and cortex.
1997-00: Defeating Deafness - Dunhill Medical Trust Research Fellow
1996-01: Research Fellow at Christ-Church, Oxford University. Studying neural mechanisms of sound localisation in mammals.
Honors
2010: Teaching excellence award, Medical Sciences Division, University of Oxford
2007: Runner up in the UK national final of the FameLab Science Communication competition
2002: Servier Young Scientist Award, 22nd European Winter Conference for Brain Research
1993: Wellcome Prize Studentship
1987: Sessional prize in Genetics, Department of Genetics & Biometry, University College London
C. Contributions to Science
My career in auditory neuroscience research spans almost 30 years and over 80 peer reviewed publications. For a full list see
https://scholar.google.com/citations?user=WCTI8v4AAAAJ&hl=en or
https://pubmed.ncbi.nlm.nih.gov/?term=schnupp_j
Below I shall give only a very brief and selective overview of some of the main research themes that I have made contributions to in the course of this career.
Studies of spatial hearing
My early work demonstrated the role of NMDA receptors and auditory-visual interactions in shaping the sensitivity of auditory midbrain neurons to sound source direction (Schnupp et al. 1995; King et al. 1998; Schnupp et al. 1998) , and used receptive field modelling, information theoretic analysis and virtual acoustic space techniques to dissect the contributions of different types of acoustic cues to the spatial tuning of midbrain and cortical neurons (Jenison et al. 2001; Schnupp et al. 2001; Mrsic-Flogel et al. 2003; Nelken et al. 2005)
More recently I returned to the subject of spatial hearing in the context of cochlear implantation. After demonstrating that laboratory rats are, contrary to popular belief, highly sensitive to interaural time differences (Li et al. 2019), we have gone on to find strong indications that the inability of human CI patients to use interaural time differences may be due to maladaptive plasticity rather than lacking input during a critical period. If confirmed, these findings may lead to important improvements in neuroprosthetic practices. Several manuscripts detailing this work are currently under review.
Adaptation and gain control
The receptive field modeling approaches we first used to probe tuning to spatial cues in cortical neurons we were later able to develop further to study cortical gain control mechanisms (Cooke et al. 2018) (Rabinowitz et al. 2012) (Rabinowitz et al. 2011) , and to show the importance of these mechanisms in allowing the central auditory cortical neurons to maintain representations of complex sounds such as speech in environments of increasing background noise (Rabinowitz et al. 2013). Arguing by analogy from the visual system we originally suspected that cortical parvalbumin positive interneurons might be responsible for this form of contrast adaptation, but our most recent work has not borne this out (Cooke et al. 2020). My colleagues in Oxford will continue this line of investigation independently.
Studies of periodicity pitch and timbre processing
Another major area of research which I have pursued, and which is most relevant to the current proposal, investigated how the auditory cortex represents perceptual features such as pitch and timbre, and how these representations interact with spatial tuning. Our work was able to show that the activity of primary auditory cortex neurons encodes perceived pitch and not just pitch cues such as periodicity (Bizley et al. 2013) We were also able to show that, contrary to previous reports, periodotopic maps in the cortex (Nelken et al. 2004) and the inferior colliculus (Schnupp et al. 2015) are not a consistent feature of the mammalian brain, and therefore not a possible substrate for the encoding of periodicity pitch. Rather, pitch representations are patchy and multiplexed, and sensitivities to pitch, timbre and space are interwoven until at least second order cortical fields (Bizley et al. 2009; Walker et al. 2011). However some of our recent human EEG experiments suggest specializations of function for pitch, timbre and spatial processing in high-level multisensory cortex that may even be lateralized, but revealing these specializations requires attentional control (Retsa et al. 2018).
Studies of rhythm processing and predictive coding
More recently, my lab and I have also made contributions to our understanding of the role of rhythmic structure of sounds in several aspects of hearing, from generating expectations and mismatch responses (Szymanski et al. 2011) , to the enhanced detection of targets (Lawrance et al. 2014) or the binding of sounds into auditory streams (Rajendran et al. 2013), to retuning of neurons in human cortex through temporal expectations (Auksztulewicz et al. 2019) , and even to pre-processing of sounds by midbrain and cortex neurons for the emergence of musical beat perception (Rajendran et al. 2017; Rajendran and Schnupp 2019; Rajendran et al. 2020).
Science education and outreach
My long and broad experience in researching the central auditory pathway has put me in an ideal position to write, with two esteemed colleagues, the leading textbook on auditory neuroscience (Schnupp et al. 2011) and I have set up an interactive web site, https://auditoryneuroscience.com, which features many interactive examples that make auditory phenomena more accessible to students of this important field.
References
Auksztulewicz, R., Myers, N. E., Schnupp, J. W. and Nobre, A. C. (2019) Rhythmic Temporal Expectation Boosts Neural Activity by Increasing Neural Gain. J Neurosci 39:9806-9817.
Bizley, J. K., Walker, K. M., Silverman, B. W., King, A. J. and Schnupp, J. W. (2009) Interdependent encoding of pitch, timbre, and spatial location in auditory cortex. J. Neurosci. 29:2064-75.
Bizley, J. K., Walker, K. M. M., Nodal, F. R., King, A. J. and Schnupp, J. W. H. (2013) Auditory cortex represents both pitch judgments and the corresponding acoustic cues. Curr. Biol. 23:620-625.
Cooke, J. E., Kahn, M. C., Mann, E. O., King, A. J., Schnupp, J. W. H. and Willmore, B. D. B. (2020) Contrast gain control occurs independently of both parvalbumin-positive interneuron activity and shunting inhibition in auditory cortex. J. Neurophysiol. 123:1536-1551.
Cooke, J. E., King, A. J., Willmore, B. D. B. and Schnupp, J. W. H. (2018) Contrast gain control in mouse auditory cortex. J. Neurophysiol. 120:1872-1884.
Jenison, R. L., Schnupp, J. W., Reale, R. A. and Brugge, J. F. (2001) Auditory space-time receptive field dynamics revealed by spherical white-noise analysis. J. Neurosci. 21:4408-15.
King, A. J., Schnupp, J. W. H. and Thompson, I. D. (1998) Signals from the superficial layers of the superior colliculus enable the development of the auditory space map in the deeper layers. J. Neurosci. 18:9394-408.
Lawrance, E. L. A., Harper, N. S., Cooke, J. E. and Schnupp, J. W. H. (2014) Temporal predictability enhances auditory detection. J. Acoust. Soc. Am. 135:EL357-EL363.
Li, K., Chan, C. H. K., Rajendran, V. G., Meng, Q., Rosskothen-Kuhl, N. and Schnupp, J. W. H. (2019) Microsecond sensitivity to envelope interaural time differences in rats. The Journal of the Acoustical Society of America 145:EL341.
Mrsic-Flogel, T., Schnupp, J. and King, A. (2003) Acoustic factors govern developmental sharpening of spatial tuning in the auditory cortex. Nat. Neurosci. 6:981-988.
Nelken, I., Bizley, J. K., Nodal, F. R., Ahmed, B., Schnupp, J. W. and King, A. J. (2004) Large-scale organization of ferret auditory cortex revealed using continuous acquisition of intrinsic optical signals. J. Neurophysiol. 92:2574-88.
Nelken, I., Chechik, G., Mrsic-Flogel, T. D., King, A. J. and Schnupp, J. W. (2005) Encoding stimulus information by spike numbers and mean response time in primary auditory cortex. J. Comput. Neurosci. 19:199-221.
Rabinowitz, N., Willmore, B., King, A. and Schnupp, J. (2013) Constructing Noise-Invariant Representations of Sound in the Auditory Pathway. PLoS Biol. 11:e1001710.
Rabinowitz, N. C., Willmore, B. D., Schnupp, J. W. and King, A. J. (2011) Contrast gain control in auditory cortex. Neuron 70:1178-91.
Rabinowitz, N. C., Willmore, B. D., Schnupp, J. W. and King, A. J. (2012) Spectrotemporal contrast kernels for neurons in primary auditory cortex. J. Neurosci. 32:11271-84.
Rajendran, V. G., Harper, N. S., Garcia-Lazaro, J. A., Lesica, N. A. and Schnupp, J. W. H. (2017) Midbrain adaptation may set the stage for the perception of musical beat. Proc Royal Soc B. Biological sciences 284.
Rajendran, V. G., Harper, N. S. and Schnupp, J. W. H. (2020) Auditory cortical representation of music favours the perceived beat. R. Soc. Open Sci. 7:191194.
Rajendran, V. G., Harper, N. S., Willmore, B. D., Hartmann, W. M. and Schnupp, J. W. H. (2013) Temporal predictability as a grouping cue in the perception of auditory streams. J. Acoust. Soc. Am. 134:EL98-E104.
Rajendran, V. G. and Schnupp, J. W. H. (2019) Frequency tagging cannot measure neural tracking of beat or meter. Proc. Natl. Acad. Sci. U.S.A. 116:2779-2780.
Retsa, C., Matusz, P. J., Schnupp, J. W. H. and Murray, M. M. (2018) What's what in auditory cortices?. Neuroimage 176:29-40.
Schnupp, J. W., King, A. J. and Carlile, S. (1998) Altered spectral localization cues disrupt the development of the auditory space map in the superior colliculus of the ferret. J. Neurophysiol. 79:1053-69.
Schnupp, J. W., King, A. J., Smith, A. L. and Thompson, I. D. (1995) NMDA-receptor antagonists disrupt the formation of the auditory space map in the mammalian superior colliculus. J. Neurosci. 15:1516-31.
Schnupp, J. W., Mrsic-Flogel, T. D. and King, A. J. (2001) Linear processing of spatial cues in primary auditory cortex. Nature 414:200-4.
Schnupp, J. W. H., Garcia-Lazaro, J. A. and Lesica, N. A. (2015) Periodotopy in the gerbil inferior colliculus: local clustering rather than a gradient map. Front. Neural Circuits 9:37.
Szymanski, F. D., Rabinowitz, N. C., Magri, C., Panzeri, S. and Schnupp, J. W. (2011) The laminar and temporal structure of stimulus information in the phase of field potentials of auditory cortex. J. Neurosci. 31:15787-801.
Walker, K. M., Bizley, J. K., King, A. J. and Schnupp, J. W. (2011) Multiplexed and robust representations of sound features in auditory cortex. J. Neurosci. 31:14565-76.
D. Research Support (current)
Please see the Projects page for a list of recent enxternally funded projects pursued by our group.