Kathleen Pichora-Fuller – Major Thematic Grant – 2000
In this study, perception researchers tackled how physical stimuli arising from sources in the environment are processed (physiologically by organisms or computationally by machines) such that particular experiences (states) or behaviours (actions) result.
Psychoacousticians study hearing by determining how listeners respond to artificially simple stimuli in which specific physical dimensions are manipulated independently; indeed many stimuli used in these experiments do not occur naturally in the world. In contrast, Gestalt psychologists study how listeners responded to intact examples of natural sounds. The latter approach has greater ecological validity, but to date it had not yielded quantitative models. It was thought by the Acoustic Ecology researchers that a more productive intermediate approach in the study of speech perception was the analysis-by-synthesis approach in which complex natural sounds have been modified or synthesized to determine which aspects of the sound pattern cue particular responses. Just as post-war electronics enabled analysis-by-synthesis research, in the late 1990s, computer speed and memory were by then sufficient to enable us to adopt an analysis-by-synthesis type approach to study how listeners respond to the complex array of cues that are present in real acoustical environments. The same computational tools that enable us to record and systematically manipulate dimensions of complex stimuli (virtual reality) also enable us to create computational models that are closer approximations of biological systems (neural networks).
The new approach developed in this study re-focussed research from "hearing" to "listening." This re-focusing reflected the more general shift in cognitive science from modular to integrated views of the brain and behavior. Whereas ears were once viewed as passive biological microphones that picked up sound and sent messages to the brain, the ears were now viewed as active sound grabbers. Over the last several decades auditory physiologists had learned how top-down control from the brain 'tunes' the auditory system even down to the level of the most peripheral sensory cells. 'Listening' captures the interplay of hearing and thinking that must be featured in future models.
Foundational research conducted under the Acoustic Ecology grant was critical to the awarding of two Canadian Foundation for Innovation project grants for which Acoustic Ecology members were project leaders and core investigators: Hearing, Accessibility, Assistive Technology, and Acoustic Design ($2.4 million), and the Institute of Computing Information and Cognitive Systems ($22.1 million).
From 2002, William McKellin, Department of Anthropology (2002-2003) continued the work started by Kathleen Pichora-Fuller.
Perception researchers study how physical stimuli arising from sources in the environment are processed (physiologically by organisms or computationally by machines) such that particular experiences (states) or behaviours (actions) result. Two research approaches are the psychophysical and the gestalt. Psychoacousticians study hearing by determining how listeners respond to artificially simple stimuli in which specific physical dimensions are manipulated independently; indeed, many stimuli used in these experiments do not occur naturally in the world. In contrast, gestalt psychologists study how listeners respond to intact examples of natural sounds. The latter approach has greater ecological validity, but to date it has not yielded quantitative models.
A more productive intermediate approach in the study of speech perception has been the analysis-by-synthesis approach in which complex natural sounds have been modified or synthesized to determine which aspects of the sound pattern cue particular responses. Just as post-war electronics enabled analysis-by-synthesis research, at the present time, computer speed and memory are now sufficient to enable us to adopt an analysis-by-synthesis type approach to study how listeners respond to the complex array of cues that are present in real acoustical environments. The same computational tools that enable us to record and systematically manipulate dimensions of complex stimuli (virtual reality) also enable us to create computational models that are closer approximations of biological systems (neural networks). A new approach will also re-focus research from 'hearing' to 'listening'. This re-focusing reflects the more general shift in cognitive science from modular to integrated views of the brain and behaviour. Whereas ears were once viewed as passive biological microphones that picked up sound and sent messages to the brain, the ears are now viewed as active sound grabbers.
Over the last two decades, auditory physiologists have learned how top-down control from the brain 'tunes' the auditory system even down to the level of the most peripheral sensory cells. 'Listening' captures the interplay of hearing and thinking that must be featured in future models. Cognitive science concerns how information is processed by humans and machines (computers). It includes efforts to model human information processing through the use of machine simulations of human performance, and the design of machines that accomplish the same functional outcome as humans whether or not they do so in the same way. An important ultimate application for cognitive science is the design of machines that interface with humans to enhance performance or compensate for lost function. Virtual reality (VR: e.g. auralization or visualization) concerns the use of computation to produce stimuli which mimic stimuli arising in the real world such that when these artificially produced stimuli are perceived by a human the result is an experience comparable to the one that would have resulted in the real world conditions being simulated. VR can be used to assist in the design of actual environments or to emulate real environments. Such artificial alternatives to actual environments may be manipulated intentionally to create novel experiences for artistic purposes. They may also deviate from actual environments unintentionally simply because the computed stimuli fail to reproduce an adequate set of cues to give good fidelity.
An important ultimate goal is to use VR to design enhanced environments or to provide substitutes for or interfaces with actual environments. The dual use of these kinds of computation could result in fine-tuning of models of humans and environments by having machine replicas of humans tested in machine replicas of environments. In the meantime, we can blend and use these kinds of computation to learn how to balance environmental and psychological requirements to optimize designs for human listeners.
