an ARTICLE BY emma mancer FROM A NEUROPSYCHOLOGY PERSPECTIVE
Over the past 20 years or so the study of multisensory integration has gained increasing popularity amongst educational, developmental, cognitive and& biological psychologists across the world. Multisensory integration refers to the ability to integrate and process more than one sensory modality at the same time, the response to which is larger than the sum of the individual inputs (Stein, Stanford, Ramachandran, Perrault, & Rowland, 2009). That is, once the sensory modalities are combined and processed by the brain, they can no longer be reduced back to their individual components.
The importance of multisensory experience for development has been shown through research indicating that both adult and juvenile brains are significantly shaped through environmental input (Bavelier & Neville, 2002). Shams and Seitz (2008) conclude that it is more than likely that the human brain has evolved to learn and develop optimally in a multisensory environment and it is therefore reasonable to assume that the key to enhanced learning in children may involve multisensory experiences. Furthermore, Stein, Stanford, and Rowland (2014) showed through studies of the midbrain that an infant’s ability to process and integrate multisensory stimuli is not predetermined and therefore must rely on environmental experiences to develop this ability. They further concluded that it is these multisensory experiences that alter the neural circuit in order to allow for more effective multisensory processing into adulthood.
Finding the exact neural mechanisms behind the brain’s ability to combine and process multisensory stimuli is becoming an increasingly studied yet still largely uncharted area. It is clear, however, that multisensory processing plays an important role in influencing how we react to stimuli in our environment as well as helping to modify our perceptions of the world (Wallace, 2004). This paper examines the hypothesis that multisensory experiences in childhood benefit learning by enhancing multisensory neuronal activity.
Enhanced multisensory neuronal activity (EMNA) refers to an enhancement of a multisensory neuron’s performance in the brain which occurs during, or as a result of, multisensory experience. The mechanisms identified in this paper as EMNA have been limited to multisensory neural convergence and multisensory neural receptive fields. There are many more mechanisms resulting from multisensory experience which have not been explored here such as increased neuronal activation, increased neural firing rates, reduced neural latency, increased neural networks, plasticity of brain areas and neural proliferation (Mesulam, 1998; Moucha, Pandya, Engineer, Rathbun, & Kilgard, 2005; Raffin, Richard, Giraux, & Reilly, 2016; Stein et al., 2009; Wallace & Stein, 1997).
Multisensory neurons are defined by their inputs involving stimulation from two or more unisensory neurons which converge into the one multisensory neuron. Multisensory neurons which a are found in particular abundance in the deep layers of the superior colliculus of the midbrain (Meredith, 2004). While the prevalence of multisensory neurons has been identified through many experiments, some of which are discussed below, Meredith (2004) explains the difficulty in observing and testing multisensory neurons. As the common method of stimulating and testing neurons is through using electrical impulses this method fails to identify those neurons that are being excited by one unisensory neuron and inhibited by another. It is for this reason that the true number of multisensory neurons in the brain may be grossly underestimated. Of those that have been identified it has been shown that all multisensory neurons have multisensory receptive fields, receiving input from several unimodal neurons at one time resulting in either an overall excitatory or inhibitory response (Chalupa & Rhoades, 1977). This convergence of the stimuli from two or more neurons onto one single neuron allows for more specific processing based on the unique combinations and strength of each input and is known as neural convergence. Neural convergence is an important mechanism for learning as multisensory neurons need to ‘learn’ how to process and integrate the multimodal inputs in order to enhance their responses (Wallace & Stein, 2001). This in turn ensures that the resulting behaviour from the individual is both appropriate and effective for the current environment. It is noted, however, that convergence doesn’t mean that integration is occurring as true multisensory integration at the neuronal level is thought to be achieved when the magnitude of the multisensory neural firing is greater than the sum of the individual inputs (Stein et al., 2014). This means that not only has the multisensory neuron received input from unisensory neurons but it has combined the stimuli and exhibited a completely new response based on the results of those combined input. The exact mechanism for this integration has been described as the multisensory neuron computing the statistical probability that a target is present in its receptive field based on the different locations and strength combinations of cross modal inputs that it receives (Anastasio & Patton, 2004). If the multisensory neuron deems it probable that the multiple inputs are derived from the same stimuli (for example they occurred close in time and/or space) then it will elicit an excitatory response causing a behavioural change such as movement towards a stimuli or attention being diverted. It has been found that unimodal stimuli which converge onto a multisensory neuron but are not close in space and/or time are interpreted by the multisensory neuron as not related and therefore the multisensory neuron produces an inhibited response (Stein et al., 2014).
The development of these statistical calculations are not innate for a multisensory neuron and are instead developed through ‘learning’ association through ongoing multisensory experiences (Raij & Jousmaki, 2004). It could therefore be suggested that ongoing multisensory experiences in one’s early years may enhance this process and in turn improve an individual’s capacity to interact appropriately with and interpret their environment. Wallace and Stein (2001) found that while newborn monkeys showed evidence of having multisensory neurons in the super colliculus, they did not yet have the ability to integrate this multisensory information as adult monkeys did. This is an important finding as it suggests that multisensory integration is a process that is dependent on multisensory experiences throughout development. This idea is further supported by research that shows that learning information through audio visual pairings results in significantly higher recall rates than when the same information is presented in a single modality (Guo & Guo, 2005; Schroger & Widmann, 1998; Seitz, Kim, & Shams, 2006). These studies show that multisensory integration can be learned as well as developed naturally over the course of a maturation period. Without multisensory experiences these converging connections are developed, however lack the ability to adequately integrate and process the multisensory information (Yu, Rowland, Xu, & Stein, 2013). It is therefore suggested that convergence from unimodal neurons onto a single multisensory neuron results in a more specific neural interpretation of events in the environment however require repeated multisensory experiences to develop complex processing abilities.
Multisensory experiences can also enhance multisensory neuronal activity through the refinement of multisensory neural receptive fields. It has been shown that the receptive field of a multisensory neuron in the super colliculus can be either increased or decreased depending on the spatial variability and modulation rates of a sensory stimulus (Moucha et al., 2005). It is therefore appropriate to examine how sensory experiences in childhood can be manipulated to optimise this neural specificity to benefit learning.
It has been shown that the earliest audio responsive neurons in newborn cats respond to stimuli in only an excitatory manner indicating that there is limited or no receptive field specificity of these neurons in the early stages of development (Wallace & Stein, 2001). In reference to primates, the multisensory receptive field of newborn monkeys is more refined than that of cats, however the receptive fields of these neurons are still substantially larger than that of adult monkeys (Wallace & Stein, 2001). Meredith and Stein (1985) also found that the most profound change in multisensory neurons was their receptive fields which, as juveniles, often responded to an entire hemi field however were limited to specific regions of sensory space in adults. These findings illustrate the importance of both maturational development and sensory experience to specify these neurons. In addition to this Moucha et al. (2005) found that changes in the receptive fields of multisensory neurons can be induced by altering sensory experience in not only juveniles but also adults. It has also been found that background sounds during a learning task can induce the same positive effect on recall as congruent audio pairings while further research shows that the same effect is seen from incongruent multisensory input such tactile stimulation during a visual task (Macaluso, Frith, & Driver, 2000; Moucha et al., 2005). It can therefore be argued that, if multisensory experiences have such a profound, measurable effect on the receptive fields of multisensory neurons, that multisensory learning experiences throughout a child’s schooling could be used to further enhance these effects.
Decreased receptive fields in multisensory neurons are important for learning as response to multisensory stimuli is enhanced when stimuli fall within receptive fields and depressed when one unisensory stimuli falls outside this specific receptive field (Wallace & Stein, 1997). This is important as more specific receptive fields may mean an individual is able to more effectively integrate multisensory experience but also to respond effectively to multisensory environmental cues (Stein et al., 2014). In terms of learning, this could be the difference between a child being distracted by background noise or colourful posters in the classroom or their learning being enhanced by them. As we live in a highly multisensory world the earlier in development a child is exposed to multisensory experiences the more developed their ability to integrate multisensory stimuli should be. Stein et al. (2014) illustrated this importance by explaining that the ability to effectively integrate multisensory stimuli in the environment maximises the brain’s ability to use the available information and enhances neurological salience of events leading to greater recall. The above findings may have significant implications on learning environments for childcare centres, pre-schools, schools and universities with more refined multisensory receptive fields leading to a greater ability to interact with their environment, including in the classroom. The use of multisensory stimulation in the classroom could have a very easy and cost effective practical implementation in the education system through the use of background music or tactile fiddle toys in learning environments.
It is clear from the research presented that multisensory experiences play a vital role in developing one’s ability to integrate, process and respond to their environment in the most effective way. There are many implications of such findings, in particular those relating to education. It could be argued that multisensory experiences are the best way for young children to learn as it enhances recall ability, helps develop neural specificity and helps them develop process to detect targets and determine which stimuli in the environment should be attended to or ignored (Anastasio & Patton, 2004; Bahrick & Lickliter, 2000; Eimer, 2004; Kim, Seitz, & Shams, 2008; Macaluso et al., 2000; Moucha et al., 2005; Wallace & Stein, 2007). It also appropriate to conclude that multisensory learning experiences in childhood may be the perfect way to capitalise on these neuronal processes as much research has shown that the impact of these experiences on multisensory neuronal enhancement are more effective in childhood when compared with adulthood (Bauer, Dávila-Chacón, & Wermter, 2014; Bavelier & Neville, 2002; Stein et al., 2014). While these experiences are already being used within many classrooms around Australia as a way to engage young children it would be beneficial to conduct further research in this area to enable better targeting of these specific neuronal mechanisms.
In conclusion it has been shown that neuronal convergence and receptive field specificity in multisensory neurons are two primary mechanisms for multisensory integration and processing for both adults and children. While children are born with multisensory neurons they are unable to integrate this information and therefore multisensory experiences are required over a lifetime to enhance this process (Bauer et al., 2014; Stein et al., 2014; Wallace, 2004) . It is therefore hypothesised that multisensory learning experiences in childhood will lead to smaller receptive fields in multisensory neurons in adolescents and therefore lead to a greater cognitive ability and improved learning outcomes. Further research is required to test this hypothesis and to lead to implementation in a school environment, however it is believed that educators, children, families and society in general would greatly benefit from such research.
Anastasio, T., & Patton, P. (2004). Analysis and modeling of multisensory enhancement in the deep superior colliculus. In C. S. Gemma Calvert, Barry Stein (Ed.), The handbook of multisensory processes (pp. 265-284). Baskerville: Massachusetts Institute of Technology.
Bahrick, L. E., & Lickliter, R. (2000). Intersensory redundancy guides attentional selectivity and perceptual learning in infancy. Developmental psychology, 36(2), 190-201. doi:10.1037//0012-1618.104.22.168
Bauer, J., Dávila-Chacón, J., & Wermter, S. (2014). Modeling development of natural multi-sensory integration using neural self-organisation and probabilistic population codes. Connection Science, 1-19. doi:10.1080/09540091.2014.971224
Bavelier, D., & Neville, H. J. (2002). Cross-modal plasticity: where and how? Nature Review Neuroscience, 3(6), 443-452. doi:0.1038/nrn848
Chalupa, L. M., & Rhoades, R. W. (1977). Responses of visual, somatosensory, and auditory neurones in the golden hamster's superior colliculus. The Journal of physiology, 270(3), 595. Retrieved from https://www-ncbi-nlm-nih-gov.ezproxy.une.edu.au/pmc/articles/PMC1353534/pdf/jphysiol00802-0051.pdf Eimer, M. (2004). Electrophysiological studies of multisensory attention. In C. S. Gemma Calvert, Barry Stein (Ed.), The handbook of multisensory processes (pp. 549-562). Baskerville: Massachusetts Institute of Technology.
Guo, J., & Guo, A. (2005). Crossmodal interactions between olfactory and visual learning in Drosophila. Science, 309(5732), 307-310. doi:10.1126/science.1111280
Kim, R. S., Seitz, A. R., & Shams, L. (2008). Benefits of stimulus congruency for multisensory facilitation of visual learning. PLOS ONE, 3(1), e1532. doi:10.1371/journal.pone.0001532
Meredith, M. (2004). Corticocortical connectivity of cross-modal circuits. In C. S. Gemma Calvert, Barry Stein (Ed.), The handbook of multisensory processes (pp. 343-356). Baskerville: Massachusetts Institute of Technology. Meredith, M., & Stein, B. (1985). Descending efferents from the superior colliculus relay integrated multisensory information. Science, 227(4687), 657.
Moucha, R., Pandya, P., Engineer, N., Rathbun, D., & Kilgard, M. (2005). Background sounds contribute to spectrotemporal plasticity in primary auditory cortex. Experimental Brain Research, 162(4), 417-427. doi:10.1007/s00221-004-2098-4
Raffin, E., Richard, N., Giraux, P., & Reilly, K. T. (2016). Primary motor cortex changes after amputation correlate with phantom limb pain and the ability to move the phantom limb. Neuroimage, 130, 134-144. doi:10.1016/j.neuroimage.2016.01.063
Raij, T., & Jousmaki, V. (2004). MEG studies of cross-modal integration and plasticity. In G. Calvert, C. Spence, & B. Stein (Eds.), The handbook of multisensory processes (pp. 515-528). Baskerville: Massachusetts Institute of Technology.
Shams, L., & Seitz, A. R. (2008). Benefits of multisensory learning (report). Trends in Cognitive Sciences, 12(11), 411. doi:10.1016/j.tics.2008.07.006 A
Stein, B. E., Stanford, T. R., Ramachandran, R., Perrault, T. J., & Rowland, B. A. (2009). Challenges in quantifying multisensory integration: Alternative criteria, models, and inverse effectiveness. Experimental Brain Research, 198(2-3), 113-126. doi:10.1007/s00221-009-1880-8
Wallace, M. (2004). The development of multisensory integration. In G. Calvert, C. Spence, & B. Stein (Eds.), The handbook of multisensory processes (pp. 625-642). Baskerville: Massachusetts Institute of Technology.
Wallace, M., & Stein, B. (2007). Early experience determines how the senses will interact. Journal of neurophysiology, 97(1), 921-926. doi:10.1152/jn.00497.2006
Yu, L., Rowland, B. A., Xu, J., & Stein, B. E. (2013). Multisensory plasticity in adulthood: cross-modal experience enhances neuronal excitability and exposes silent inputs. Journal of neurophysiology, 109(2), 464. doi:10.1152/jn.00739.2012