Authors: Anne-Laure Vittek, Cécile Juan, Lionel G Nowak, Pascal Girard, Céline Cappe
Categories: Original Article, audiovisual, medial pulvinar, multisensory, single-units, AcademicSubjects/MED00310, AcademicSubjects/MED00385, AcademicSubjects/SCI01870
Source: Cerebral Cortex (New York, NY)
Authors: Anne-Laure Vittek, Cécile Juan, Lionel G Nowak, Pascal Girard, Céline Cappe
The pulvinar is a heterogeneous thalamic nucleus, which is well developed in primates. One of its subdivisions, the medial pulvinar, is connected to many cortical areas, including the visual, auditory, and somatosensory cortices, as well as with multisensory areas and premotor areas. However, except for the visual modality, little is known about its sensory functions. A hypothesis is that, as a region of convergence of information from different sensory modalities, the medial pulvinar plays a role in multisensory integration. To test this hypothesis, 2 macaque monkeys were trained to a fixation task and the responses of single-units to visual, auditory, and auditory–visual stimuli were examined. Analysis revealed auditory, visual, and multisensory neurons in the medial pulvinar. It also revealed multisensory integration in this structure, mainly suppressive (the audiovisual response is less than the strongest unisensory response) and subadditive (the audiovisual response is less than the sum of the auditory and the visual responses). These findings suggest that the medial pulvinar is involved in multisensory integration.
Our perception of the environment is multisensory. Horn and Hill (1966) were among the firsts to report on the existence of multisensory neurons while recording in the superior colliculus. Single-units recordings in this subcortical structure later enabled Stein and Meredith to define the rules of multisensory integration (Meredith and Stein 1983, 1986). Associative cortical areas of the temporal, frontal, and parietal lobes were later shown to have multisensory properties (Stein and Stanford 2008; Meredith et al. 2020). For example, the superior temporal sulcus, the orbitofrontal cortex, and area 7 of the primate receive inputs from many unisensory cortical areas and individual neurons in these regions can respond to stimuli from several sensory modalities (Benevento et al. 1977; Leinonen et al. 1979; Bruce et al. 1981; Barraclough et al. 2005; Schlack et al. 2005; Sugihara et al. 2006; Avillac et al. 2007; Falchier et al. 2011 for a review). More recently, evidence that even unisensory cortical areas have multisensory properties has been reported (Cappe, Rouiller, et al. 2009 for a review): anatomical connections between unisensory areas of different modalities have been observed (Falchier et al., 2002; Cappe and Barone 2005; Falchier et al. 2011). Moreover, studies showed that the responses of neurons in the auditory (Lakatos et al. 2007; Kayser et al. 2008), visual (Wang et al. 2008), and somatosensory (Zhou and Fuster 2004) cortices can be modulated by the addition of another modality. All these results led Ghazanfar and Schroeder to the provocative proposition that the whole cortex might be multisensory (Ghazanfar and Schroeder 2006). Besides cortex, subcortical structures may also be involved in multisensory integration. In addition to the superior colliculus, the inferior colliculus also shows evidence for multisensory interaction (Gruters and Groh 2012 for a review). Audiotactile interaction has been observed in the cochlear nucleus (Heeringa et al. 2018; Ansorge et al. 2021). Multisensory neurons have been found in several nuclei of the thalamus. For example, in the ventral posterior medial region, somato-visual neurons were recorded in the mouse (Allen et al. 2017) and visuo-tactile integration was observed in rats (Bieler et al. 2018). Unisensory auditory, visual, and tactile neurons, along with bimodal and trimodal neurons, were found in the centre median and parafascicular nucleus of the macaque (Matsumoto et al. 2001).
Among thalamic nuclei, the medial pulvinar appears as an ideal candidate for multimodal integration. Located posteriorly in the thalamus, the medial pulvinar presents reciprocal connections with unisensory cortical areas (auditory, visual, and somatosensory cortices) (Cappe et al. 2007; Cappe, Rouiller, et al. 2009; Homman-Ludiye et al. 2020), polysensory areas (superior temporal cortex, prefrontal cortex, posterior cingulate cortex, insular cortex) (Romanski et al. 1997; Gutierez et al. 2000; Homman-Ludiye et al. 2020), and motor areas (dorsal premotor cortex) (Morel et al. 2005; Homman-Ludiye et al. 2020). Furthermore, among the thalamic nuclei receiving inputs from several modalities, the medial pulvinar is the one with the densest connectivity and with the greatest overlap (Cappe, Morel, et al. 2009). It also receives afferents from subcortical structures such as the superior colliculus (Trojanowski and Jacobson 1975; Benevento and Standage 1983) and is interconnected with the amygdala (Jones and Burton 1976). These numerous connections suggest that it is a region of convergence of information from different sensory modalities.
Relatively little is known about the functions of the pulvinar (Froesel et al. 2021 for a review). First, lesion studies in humans (Ward and Arend 2007; Snow et al. 2009; Wilke et al. 2018) and inactivation studies in monkeys (Wilke et al. 2010) showed deficits in visual attention and distractor filtering as well as deficits in localizing visual targets and in the control of visually guided eye and limb movements. However, these behavioral studies did not examine sensory modalities other than visual. Second, the pulvinar is involved in the communication between cortical areas (Saalmann and Kastner 2011; Benarroch 2015). Third, electrophysiological studies showed that neurons in the pulvinar respond to auditory (Gattass et al. 1978; Yirmiya and Hocherman 1987), somatosensory (Mathers and Rapisardi 1973; Gattass et al. 1978; Avanzini et al. 1980), and visual stimuli (Mathers and Rapisardi 1973; Gattass et al. 1978; Benevento and Miller 1981; Wilke et al. 2009; Maior et al. 2010; Komura et al. 2013; Nguyen et al. 2013; Van Le et al. 2013, 2014, 2016). However, only three studies tested stimuli of different modalities on the same neurons. The first one (Mathers and Rapisardi 1973) reported responses to visual and somatosensory stimuli in the pulvinar of the squirrel monkey but did not report on multimodal interaction in this structure. The second one (Gattass et al. 1978) found convergence of auditory, visual, somatic, and olfactory inputs on neurons of various regions of the pulvinar but did not examine multisensory interaction. The third one (Avanzini et al. 1980) reported visual, acoustic, and somesthesic responses in neurons of the pulvinar-lateralis posterior complex of the cat’s thalamus. This study reported an inhibitory interaction between visual and somesthesic stimuli in the pulvinar nucleus, but interactions between auditory and visual stimuli in this structure were not explored. Thus, to the best of our knowledge, the features of audiovisual interactions in the primate pulvinar are currently unknown. In the present study, we recorded single-units in the medial pulvinar of 2 awake macaques performing a fixation task. Single-unit data confirmed the presence of auditory, visual, and audiovisual neurons in the medial pulvinar. Furthermore, it revealed the existence of a new type of multisensory neurons, which we named “complex multisensory”. These neurons responded to more than one modality, but the modality to which they responded was different depending on the stimulus category. We also provide evidence of multisensory integration in this structure, which was mostly suppressive and subadditive.
Two adult male rhesus monkeys (Macaca mulatta), weighting 7 and 5 kg and 5-years-old, participated in this experiment. They were naive to all procedures. They were housed together at night and separated during the day in cages for food and water delivery. Water was available ad libitum as soon as they were in cage, whereas access to food (fruits, vegetables, and cereals) was controlled and given after each training or recording session. A weight loss criterion of 10% was decided for stopping the experiment, but this never happened. All procedures were approved by the National Committee for Ethical Reflection on Animal Testing in compliance with the guidelines of the European Community on Animal Care (authorization 01000.02).
The monkeys were trained in a fixation task. They were sitting in a primate chair (Crist instruments), with the head fixed, in a darkened, sound-attenuated box, in front of a 31-cm distant screen (BenQ, 60-cm diagonal, 1920 × 1080 pixels, 120 Hz). Two loudspeakers (Creative Gigaworks t20 serie II) were positioned one on each side of the screen. The training took place 5 days a week, on the morning. As the monkeys were food deprived, the training was their first access to food. Each correct trial was rewarded with diluted compote (0.05 mL/trial).
The task was as follows. To initiate a trial, the monkey had to fixate a 0.5° point into a square fixation window of 2 × 2° for a random time between 500 and 1200 ms. Then, a stimulus was presented for 250 ms (auditory, visual, or audiovisual stimulus). The monkey had to maintain fixation during stimulus presentation, while the fixation point was extinguished. If fixation was not broken, the monkey was rewarded after a random delay (300–700 ms). Otherwise, the trial was aborted. The intertrial interval was 1000 ms. The task was controlled with EventIDE software (Okazolab Ltd), and eye tracking was performed with an eye-tracker (ISCAN ETL 200, Woburn, MA, USA).
A total of 9 stimuli was 3 auditory, 3 visual, and 3 audiovisual stimuli. The auditory stimuli consisted in white noise, in a macaque “coo” call, and in a rattlesnake rattle. The auditory stimuli were stereo 44–48 kHz waves normalized to 60 dB, with a 3-ms fading-in and fading-out. The visual stimuli were a picture of random dots, a rhesus macaque head photograph, and a rattlesnake picture. All images were normalized in RGB colors with a color depth of 24, sized 453 × 453 pixels with a 72 × 72 dpi resolution (final size 19 × 19°), with a mean luminance of 120 cd/m^2^, and presented in the center of the screen. There was only one exemplar of each stimulus (macaque, snake, noise). The audiovisual stimuli were the synchronous presentation of an auditory and a visual stimulus, always semantically congruent. As pulvinar neurons can respond to both low level features (Mathers and Rapisardi 1973; Gattass et al. 1978; Benevento and Miller 1981) and high level stimuli (Maior et al. 2010; Nguyen et al. 2013; Van Le et al. 2013), it cannot be determined whether individual neurons responded to the components of the stimuli or to the stimuli as a whole. The stimuli were randomly presented, about 20 times each. A 4 × 4° white square appeared in the upper right corner of the screen (hidden from the monkey’s view by black tape) at the same time as the stimulus presentation, to precisely indicate onset and offset of the stimulus.
A first surgery was performed to implant an MRI-compatible headpost (Crist Instrument) to the skull. After 1 month of recovery, the monkeys were trained in the fixation task. When they reached 90–95% correct trials, a second surgery was performed to stereotaxically implant a footed stainless-steel recording chamber (Crist Instrument) above the right somatosensory cortex (S2) (centered on average AP = 6.5 and ML = 0.75). The skull within the chamber was removed to allow direct access to the brain. The dura mater was left intact and was protected by a sterile silicone patch.
All surgeries were performed under anesthesia, induced with a mixture of tiletamine/zolazepam (Zoletil 50®, 5 mg/kg) and glycopyrrolate bromide (Robinul®, 0.01 mg/kg), and maintained with isoflurane (1.5%) after intubation. Analgesics were administered (Tolfedine 4 mg/kg and buprenorphine chlorhydrate [Vetergesic®] 0.01 mg/kg) during surgery and the following days. An antibiotic treatment (Amoxicillin [Clamoxyl® LA] 15 mg/kg) was administered during the first week.
The location of the recording chamber was determined before surgery by comparing stereotaxic 3 T anatomical MRI scans to sections of stereotaxic atlases of the brain of M. mulatta (Paxinos et al. 2000; Saleem and Logothetis 2007). After implantation, the locations of the recording sites were confirmed by comparing the atlas sections to stereotaxic anatomical MRI scans of the monkeys’ head with a tungsten microelectrode inserted at target coordinates. The locations of the recording sites are relative to the zero coordinates defined in the stereotaxic atlases.
Neuronal activity was recorded with tungsten microelectrodes (5–7 MΩ at 1 kHz, Frederick Haer Company, Bowdoinham, ME). The electrodes were inserted daily into the medial pulvinar (ML between 4.9 and 9.8 mm, AP between 5 and 9.8 mm, depth between 17.9 and 23.8 mm below the cortical surface). The electrode insertion was performed with an oil hydraulic micromanipulator (Narishige MO-972) attached to the recording chamber.
The recorded signal was sampled at a frequency of 40 kHz with a 1401 power acquisition interface (CED, Cambridge, UK), amplified with a gain of 1000 (NL104) and band-pass filtered between 200 Hz and 8 kHz (NL125) by a Neurolog system (Digitimer, Hertfordshire, UK). The 50-Hz noise was eliminated with a Humbug device (Digitimer). The signal was displayed and recorded with Spike2 software (CED). Data were analyzed with custom scripts in the Spike2 software environment.
The collected data usually contained spikes issued from several neurons. For single-unit isolation, spike sorting was performed with the PCA-based cluster analysis of Spike2 software. First, 1- to 1.5-ms-wide portion of the voltage trace was extracted online around putative spikes whenever the peak amplitude exceeded a threshold set at 3 times the noise level. The trace chunks were then analyzed offline by PCA. A 3D rendering of the PCA results allowed visualizing one or several clusters, where each cluster groups spike shapes that are similar. Spikes grouped in a given cluster were ascribed to single-unit activity if the corresponding interspike interval distribution showed a clear refractory period, i.e. no interval < 1.2 ms. In contrast, multiunits show no refractory period since they group several independent neurons. Based on our statistical analysis (see below), we found that in all but 3 cases, single-units simultaneously recorded at a given site exhibited distinct response patterns with respect to stimuli categories and modalities.
Only neurons with sufficiently high activity (more than 800 spikes on the whole recording period) were analyzed. To determine if there was a response to the stimuli, we first calculated, for each stimulus presentation, the mean firing rate per trial during stimulus presentation (number of spikes occurring during the 0.25-s-long stimulus presentation divided by 0.25 s). We similarly calculated the mean spontaneous activity per trial during the period preceding each stimulus presentation (−500–0 ms). The distributions of the ≈20 values of spontaneous and evoked firing rates per trial obtained for each type of stimulus were then compared with the Wilcoxon signed-rank test. A P-value of 0.05 was used as a threshold to indicate the presence of significant excitatory or inhibitory responses. Responses to different stimuli were then compared using the Kruskal–Wallis test and the Mann–Whitney test applied to the distributions of mean firing rates calculated for each stimulus presentation. Again, the threshold P-value was 0.05, and a Bonferroni correction was applied for multiple comparisons.
This allowed classifying neurons in 4
The first class corresponds to unisensory visual neurons. These neurons showed significant responses to both visual and audiovisual stimuli, but the response to visual and audiovisual stimuli did not differ significantly.
The second class corresponds to unisensory auditory neurons, characterized by a significant response to both auditory and audiovisual stimuli, but without significant differences between these 2 responses.
The third class corresponds to multisensory neurons that we named “classic” because they have already been encountered and described in previous multisensory studies (e.g. Horn and Hill 1966; Meredith and Stein 1983; Allman et al. 2009; Kayser et al. 2011; Olcese et al. 2013; Meredith et al. 2020). This class includes several subtypes of bimodal
The fourth class, which has not been described previously, corresponds to “complex” multisensory neurons. We named these cells “complex” because of the profile of their responses. Depending on the stimulus category, these neurons displayed either unisensory or multisensory responses. Complex multisensory neurons were further subdivided according to the modalities to which they responded.
A first group of complex neurons behaved as unimodal auditory neuron with 1 or 2 stimulus categories and as unimodal visual neurons with the remaining categories (A & V cells). For example, a cell that responded significantly to the sound of the rattlesnake and to the image of the monkey would belong to this subtype. The other 3 groups of complex neurons presented multimodal responses to 1 or 2 stimulus categories and unimodal responses to the remaining stimulus categories.
Thus, neurons could be unimodal visual for 1 or 2 stimulus categories and bimodal for the remaining ones (V & AV cells). For example, a cell could respond as a unimodal visual cell for the noise stimuli and as a bimodal cell for the snake and/or rhesus stimuli.
Likewise, neurons could be unimodal auditory for 1 or 2 stimulus categories and bimodal for the remaining ones (A & AV neurons).
The fourth type of complex audiovisual neurons corresponds to auditory–visual-audiovisual neurons (A & V & AV); these cells were auditory for one stimulus category, visual for another category, and audiovisual for the last category. This subclassification is independent of the number of stimulus categories used, as it is based only on the modalities that actually activated the cells.
The statistical tests to classify the neurons were performed on the individual stimuli (i.e. visual macaque) and with all the stimuli of a given modality pooled together (i.e. visual). Thus, a neuron could be classified as visual because it responded significantly to one visual stimulus or because it responded when all responses to visual stimuli of all stimuli were grouped together.
We also computed peri-stimulus time histograms (PSTHs) to illustrate the time course of the responses. These PSTHs were computed following conventional methods, by summing the number of spikes occurring within 10-ms-wide bin across stimulus repeats, then normalizing this sum by the bin width to obtain a firing rate in spikes/s. The PSTHs were then smoothed with a 45-ms moving average.
To characterize multisensory integration, we used the 2 indexes proposed by Avillac et al. (2007), which are derived from previously used indexes (Meredith and Stein 1986; Populin and Yin 2002): the first one is the amplification index (AMI), whose formula is\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$$ AMI=\frac{\left| AV\right|-\left|{U}{\mathrm{max}}\right|}{\left| AV\right|+\left|{U}{\mathrm{max}}\right|} \times100,
The second index is the additivity index (ADI), whose formula is\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$$ ADI=100\times \frac{\left| AV\right|-\left(\left|A\right|+\left|V\right|\right)}{\left| AV\right|+\left|A\right|+\left|V\right|}, $$\end{document}with *AV*, *V*, and *A* the median of the mean firing rate per trial in the audiovisual, visual, and auditory conditions, respectively. This index allows categorizing neurons in 3 groups on the basis of the linearity of the multimodal subadditive (index < 0), supraadditive (index > 0), or additive (index = 0). When neurons were multisensory for more than one stimulus category, the corresponding indexes were averaged. With the formula presented above, ADI and AMI are restricted to the range −100% to +100%. Two other indexes were computed to characterize the stimulus selectivity of the neurons’ responses. The first is the selectivity index (SEI), whose formula is\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$$ SEI=\frac{F{R}_{\mathrm{max}}-F{R}_{\mathrm{min}}}{F{R}_{\mathrm{max}}+F{R}_{\mathrm{min}}}, $$\end{document}with *FR*~min~ and *FR*~max~ the minimal and maximal responses, respectively, among the 9 responses. This index shows whether a neuron was selective for one stimulus (index close to 1) or whether it responded similarly to the best and worse stimuli (index close to 0). We also computed a sparseness index (SPI), whose formula is\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{upgreek} \usepackage{mathrsfs} \setlength{\oddsidemargin}{-69pt} \begin{document} }{}$$ SPI=\frac{{\left({\sum}_{FRI=1,n}F{R}_i/n\right)}^2}{\sum_{FRI=1,n}\left(F{R_i}^2/n\right)}, $$\end{document}with *FR~i~* the response to the stimulus *i*, and *n* = 9 (number of stimuli). This index indicates whether the neuron responded similarly to all stimuli (index close to 1) or whether it responded exclusively to one or few stimuli (index << 1). ### Latencies Latencies were calculated using the half-rise latency method (Levick 1973; Nowak et al. 2010). To improve the temporal resolution while preserving a good signal-to-noise ratio, 10 PSTHs with a bin width of 10 ms were computed with a 1-ms offset increment for each calculation. The 10 PSTHs were then averaged with a 1-ms resolution. The peak amplitude and the half-peak amplitude were determined. Only peaks with a half-peak amplitude higher than the mean + 3 SD in PSTH prestimulus baseline (−500–0 ms) were considered. The time after stimulus onset at which the half-peak amplitude is first reached corresponds to the half-rise latency. A further constraint was that the amplitude remained ≥ half-peak height for at least 7 consecutive ms. Latencies have been obtained for 69 neurons, yielding a total of 160 latencies as a given neuron could respond to several stimulus categories and modalities. Note that this method, which requires assessment of significance over the time course of the response, is more constrained than that used for determining presence of significant response over the whole stimulus presentation (“classification of neurons” above). This explains why the number of neurons with significant latencies is less than the number of neurons with significant responses. To examine the influence of stimulus modality, we reduced the dataset by taking, for each neuron, the shortest latency within each modality. ### Data availability The datasets supporting the current study have not been deposited in a public repository because they are still used for the analysis of local field potentials but are available from the corresponding author on request. ## Results ### Medial pulvinar contains auditory, visual, and audiovisual neurons We recorded spiking activity in response to visual, auditory, and audiovisual stimuli in the medial pulvinar of 2 macaques engaged in a fixation task (Fig. 1). 374 single-units were recorded in 164 sites and 328 analyzed for responsiveness to the stimuli (46 neurons were excluded because of too low activity or fluctuating baseline). Most (213/328, 65%) medial pulvinar neurons responded to auditory, visual, and/or audiovisual stimuli. These single-units were all spontaneously active, with a median spontaneous firing rate of 3.9 spikes/s. During stimulus presentation, the neurons either showed no change in firing rate (no response), an increase of firing rate (mean + 4.2 spikes/s relative to spontaneous activity level, global population response Fig. 2A), or a decrease in activity (mean −1.2 spikes/s, global population response Fig. 2B).   The 213 responsive neurons were individually found to be auditory, visual, audiovisual, or nonspecific to one modality (19 neurons for which the response to at least one stimulus category was significant only when all modalities were pooled together—these neurons were not included in the analysis of multimodal interactions). Unisensory cells responded to only one sensory modality and responded identically to audiovisual stimuli. The 2 neurons presented in Fig. 2C, D are unisensory cells (2.C auditory, 2.D visual). The neuron presented in Fig. 2C was classified as a *unisensory auditory* the auditory stimuli induced an increase in activity that was statistically significant when compared with baseline activity. Similarly, there was a statistically significant increase in activity in response to the bimodal audiovisual stimuli, but the activity level was not statistically different from the one obtained in response to the auditory stimuli alone. Presentation of the visual stimuli had no significant effect on activity. The neuron presented in Fig. 2D was a *unisensory visual* neuron. The activity increased by the same amount in response to visual and bimodal audiovisual stimuli, but the activity did not change in the presence of the auditory stimuli presented alone. At the population level, the medial pulvinar appears to have slightly more auditory than visual unisensory neurons (χ^2^ = 6,94, df = 1, *P* = 0.008): among the 213 responsive neurons, 66 (31%) were classified as auditory, and 39 (18%) as visual (Fig. 3A).  We found 53 audiovisual neurons (25% of the responsive neurons, Fig. 3A) whose response features corresponded to those described in previous studies (e.g. Stein and Meredith 1993; Allman et al. 2009; Kayser et al. 2011; Olcese et al. 2013; Meredith et al. 2020). In order to account for this anteriority, we named these neurons “classic” audiovisual neurons. These neurons could be further subclassified in 3 classes that correspond to the different types of bimodal neurons that have been previously described. The first class includes neurons that did not respond significantly to either auditory or visual stimuli when presented alone but that responded to the combined auditory–visual stimuli (Benevento et al. 1977; Meredith and Stein 1986; Olcese et al. 2013). It represents 81% of the classic audiovisual neurons in our sample. The second class, which represents 11% of the classic audiovisual neurons, groups cells which responded in all sensory conditions; an example is presented in Fig. 2E. For this neuron, the visual, auditory, and audiovisual stimuli induced an increase in activity that was statistically significant in all cases when compared with baseline activity. Such neurons have been described in many studies, for example in the superior colliculus (e.g. Horn and Hill 1966; Meredith and Stein 1983, 1986; Bell et al. 2001; Populin and Yin 2002). The third class corresponds to neurons with multisensory suppression (e.g. Meredith and Stein 1983, 1986; Meredith et al. 1987): they responded to both unimodal auditory and visual stimuli, but their response to the bimodal stimuli was not significant. They represent 8% of the classic audiovisual neurons. ### Medial pulvinar contains complex audiovisual neurons Thirty-six neurons were found to possess “complex” audiovisual responses (17% of the responsive neurons, Fig. 3A). These are cells for which the sensory modality that evoked a response depended on the stimulus category. For example, the neuron in Fig. 4 presented a statistically significant increase in activity in response to the noise stimulus in the visual, auditory, and audiovisual modalities. This neuron would therefore be classified as *bimodal* when the stimulus was the noise stimulus in either modality. Likewise, the neuron displayed a bimodal response when the snake stimulus was used, as the activity increased in both visual, auditory, and audiovisual modalities. However, when the macaque stimulus was used, a significant increase in activity was observed only in the visual and audiovisual modalities; this cell did not respond to the auditory macaque stimulus. Furthermore, it responded equally well to the visual and audiovisual macaque stimuli. Therefore, when macaque images or sounds were used as stimuli, this neuron showed features of a *unisensory visual* neuron. Given this dual response pattern, it was classified as visual-audiovisual complex multisensory neuron (V & AV), like 28% of the complex audiovisual neurons. We found 3 other types of complex audiovisual neurons (Fig. 3B): auditory-audiovisual (A & AV, 56% of the neurons), which responded as *unisensory auditory* neurons for 1 or 2 stimulus categories, and as *bimodal* neurons with the remaining categories (A & AV neurons). We also encountered auditory–visual neurons (A & V, 8%), which behaved as unisensory auditory neuron with 1 or 2 stimulus categories and as unisensory visual neurons with the remaining categories. The last type of complex audiovisual neurons corresponds to auditory–visual-audiovisual neurons (A & V & AV, 8%); these cells were auditory for one stimulus category, visual for another category, and audiovisual for the last category. This subclassification is independent of the number of stimulus categories used, as it is based only on the modalities that actually activated the cells.  ### Medial pulvinar neurons are poorly selective The sparseness and the SEIs indicate that medial pulvinar neurons responded to many stimuli and tended to be poorly selective. The SPI quantifies the variability of the responses to all 9 stimuli used. It is indicative of a sparse representation when the index approaches 0 and of a distributed representation when the index reaches values close to 1. In our sample, it ranges from 0.42 to 1, with a median at 0.93 (Fig. 5). The response features of unisensory neurons may bias the SPI toward high values. Indeed, unisensory neurons responded similarly to the unisensory and the corresponding audiovisual stimuli. However, the distribution of the SPI remained similar when computed only on multisensory neurons (minimum: 0.42, 0.93, and 0.99; not illustrated).  The SEI compares the response of the neuron to the best and the worst stimulus. An index close to 1 is indicative of a very selective neuron, which responded to only one stimulus, whereas an index close to 0 is indicative of a poorly selective neuron, which responded almost as well to the worst as to the best stimulus. The SEI ranges from 0.10 to 1, with a median at 0.43 (Fig. 5). Figure 6 illustrate averaged PSTHs for each stimulus and modality. Although some neurons could respond selectively to one stimulus category, there was no overall preference for one or another category at the population level. In addition, at the population level response, amplitudes were not different between snake, macaque, and noise stimuli, whatever the modality (Kruskal–Wallis test, all *P*-values > 0.05).  ### Multisensory integration in the medial pulvinar is subadditive and suppressive Additivity and AMIs, calculated for each multisensory cell, revealed subadditive and suppressive multisensory integration at the population level in the medial pulvinar. The AMI allows determining if the multisensory response is enhanced (index > 0) or suppressed (index < 0) compared with the best unisensory response. The ADI informs about the linearity of interaction and can reveal subadditive (index < 0), additive (index = 0), or supra-additive (index > 0) interactions between unisensory and multisensory responses. AMI ranges between −61% and 34%. At the population level, the AMI revealed a dominance of suppressive interaction (median = −3%; sign test, *P* = 0.004). ADI ranges between −74% and 13%. Unsurprisingly, at the population level, the ADI revealed an essentially subadditive audiovisual interaction (median = −34%; sign test, *P* = 10^−23^). There is a continuum from subadditive and depressed to supra-additive and enhanced responses (Fig. 7) and the majority of the cells are subadditive depressed cells.  ### Faster responses for auditory and audiovisual than for visual stimuli Stimulus modality strongly affected response latencies (Kruskal–Wallis test, *P* < 0.0001). The medians of the half-rise latencies for auditory and audiovisual responses were 89 ms (*n* = 35) and 75 ms (*n* = 46), respectively. The distributions for auditory and audiovisual latencies largely overlapped and did not differ significantly (Posthoc Dunn’s test, *P* > 0.99). Visual latencies, with a median equal to 141 ms (*n* = 30), were longer than both auditory (*P* = 0.003) and audiovisual latencies (P < 0.0001). The cumulative distribution shows a systematic lag for visual responses by about 50 ms (Fig. 8).  Latencies were also compared between stimuli in each modality. These comparisons did not reveal differences for the auditory and audiovisual modalities (mixed effect model, *P* = 0.13 and 0.09, respectively). However, as summarized in Fig. 8B, differences between stimuli were observed in the visual modality (*P* = 0.01) and revealed longer latencies for macaque stimulus (median 183 ms, *n* = 18) compared with noise (median 137 ms, *n* = 13) and snake (median 150 ms, *n* = 15) (Posthoc Tukey’s *P* = 0.002 for comparison between macaque and noise, *P* = 0.02 between snake and macaque, *P* = 0.06 between noise and snake). ### Sensory modalities are intermingled in the medial pulvinar A 3D map was constructed to report the location of the recording sites in the explored region (2D representation in Fig. 9). No topographic organization is visible for the sample of recorded neurons of all types were present in variable proportions at each recording site. This was confirmed by a PCA in 3D space by using the three dimensions of recording site coordinates (AP, ML, and depth).  ## Discussion ### Unisensory and multisensory neurons in the medial pulvinar In the present study, we investigated neuronal responses to auditory, visual, and audiovisual stimuli in the medial pulvinar of awake monkeys. Numerous studies have demonstrated neuronal responses to visual stimuli in the pulvinar (Mathers and Rapisardi 1973; Gattass et al. 1978; Avanzini et al. 1980; Benevento and Miller 1981; Wilke et al. 2009; Maior et al. 2010; Komura et al. 2013; Nguyen et al. 2013; Van Le et al. 2013, 2014, 2016), although few studies have specifically examined the medial part of the pulvinar (Maior et al. 2010; Van Le et al. 2013). A few studies have also shown the presence of responses to auditory (Gattass et al. 1978; Yirmiya and Hocherman 1987), somatosensory (Mathers and Rapisardi 1973; Gattass et al. 1978; Avanzini et al. 1980), and olfactory stimuli (Gattass et al. 1978) in the pulvinar. Only one study has documented multisensory interactions in the cat pulvinar (Avanzini et al. 1980); suppressive interactions were reported for somesthetic and visual stimuli. We confirmed that pulvinar neurons are responsive to visual stimulations and auditory stimulations. We also reported multisensory audiovisual neurons and we described and quantified multisensory integration in the medial pulvinar. Most of the multisensory neurons that we encountered correspond to previously described bimodal cells (e.g. Horn and Hill 1966; Benevento et al. 1977; Stein and Meredith 1993; Olcese et al. 2013; Meredith et al. 2020). There exist another type of multimodal cells, the modulatory (or subthreshold) neurons responding to a single modality when tested with unimodal stimuli (similarly to unisensory cells), but for which the unisensory response would have been different from that obtained with the audiovisual stimuli. Although often observed in cortex (Allman and Meredith 2007; Avillac et al. 2007; Meredith et al. 2020), we did not find such cell in our sample. However, some of our bimodal neurons were modulated and presented responses statistically different between the unisensory and the audiovisual stimuli. Another novelty in our results lies in the identification of complex audiovisual neurons. The modality to which these neurons responded depended on the stimulus category. This type of neurons has not been described previously. Indeed, identifying these neurons requires different types of multisensory stimuli. Instead, previous studies most often relied on the use of a single multisensory stimulus. ### Multisensory integration in the medial pulvinar The presence of numerous audiovisual neurons prompted us to examine their multisensory integration properties. Multisensory integration was globally suppressive (AMI < 0) and subadditive (ADI < 0). Studies on multisensory integration performed in the superior colliculus of anesthetized and awake cats and monkeys (Meredith and Stein 1983, 1986; Meredith et al. 1987; Wallace et al. 1996, 1998; Bell et al. 2001; Perrault Jr et al. 2003) reported either response suppression or response enhancement, depending on stimulus features. Enhanced responses were most often observed when stimuli were coincident in space and time and when stimulus intensities were in the low to medium range. Studies examining multisensory integration in cortex reported varying degrees of bimodal interactions. Some studies reported a dominance of suppressive effect (Benevento et al. 1977; Sugihara et al. 2006; Avillac et al. 2007; Kayser et al. 2008), other studies a dominance of enhanced responses (Wallace et al. 1992; Allman and Meredith 2007; Olcese et al. 2013) and still other studies found roughly equivalent numbers of suppressed and enhanced cells (Barraclough et al. 2005; Meredith et al. 2020). As in the superior colliculus, manipulation of stimulus congruency and intensities can alter and possibly reverse the sign of multimodal interactions (Avillac et al. 2007; Kayser et al. 2008; Olcese et al. 2013). In addition to the recording sites and stimulus features, the task can also influence multisensory Bell et al. (2003) showed that the mere presentation of the fixation point could decrease multisensory integration in the superior colliculus. However, in our study, the fixation point was extinguished during the presentation of the visual and auditory stimuli. Another explanation for the dominance of suppressive multisensory integration is the strength of the our stimuli were well above perception threshold. According to the law of inverse effectiveness (Meredith and Stein 1983, 1986), weak stimuli benefit more from multisensory integration and favor the occurrence of enhanced responses (Meredith and Stein 1986; Perrault Jr et al. 2003; Kayser et al. 2008). Finally, in our behavioral task, the audiovisual stimuli had no more value than the unisensory stimuli. Multisensory integration was not mandatory for the task. This could also have contributed to the occurrence of subadditive and suppressive audiovisual interactions. ### Latencies Our study disclosed median latencies of 89 ms for auditory stimuli, 75 ms for audiovisual stimuli, and 141 ms for visual stimuli. One study reported a range of visual latencies comparable to ours (Maior et al. 2010), while shorter latencies have been reported in the pulvinar in other studies (Petersen et al. 1985; Nguyen et al. 2013; Van Le et al. 2013). Differences between studies could explain this first, our method for latency measurement is different from that used in these studies; second, our recordings were restricted to the medial pulvinar. We found that visual latencies were much longer, by about 50 ms, than auditory latencies. Longer visual latencies have been observed in the superior colliculus in awake (Wallace et al. 1998; Populin and Yin 2002) and anesthetized (Meredith et al. 1987; Wallace et al. 1996) animals, with comparable or even larger differences (50–80 ms). Studies exploring multimodal cortical areas have also shown longer latencies for visual responses compared with auditory responses (Bruce et al. 1981; Schlack et al. 2005). It is known that the strength of cross-modal interactions depends on the relative timing of each stimulus modality (Meredith et al. 1987). However, we did not adjust the stimulus onsets to account for the difference in latency between modalities. One reason for not doing this is that we had no a priori knowledge of what the latency difference would be in the medial pulvinar. Another reason is that, in natural settings, objects may manifest as simultaneous auditory and visual cues, although our daily lives furnishes many instances of visual cues preceding auditory cues, and vice versa. It is to be noticed that interactions are quite close to optimal with a temporal disparity of ±100 ms (Allman and Meredith 2007). Moreover, the temporal overlap of the responses to auditory and visual stimuli (Fig. 6) suggests that multisensory interactions would have ample room to develop despite differences in onset latency. ### Functions of the medial pulvinar The functional role of the pulvinar remains largely unknown, yet some hypotheses have been put forward (Froesel et al. 2021). Anatomical studies have shown many sensory connections with the pulvinar (Cappe et al. 2007; Cappe, Rouiller, et al. 2009), overlapping territories of projections from different sensory cortices to the medial pulvinar (Cappe, Rouiller, et al. 2009) and projections from the medial pulvinar to the premotor cortex (Cappe, Morel, et al. 2009). This suggested that the medial pulvinar is a place of sensorimotor convergence. Our study indeed demonstrates that the medial pulvinar contains multisensory audiovisual neurons with various multisensory integration characteristics. It has been hypothesized that the pulvinar plays a major role as mediator or modulator of signal transfer between corticalareas (Saalmann and Kastner 2011; Benarroch 2015). Indeed, the replication principle postulates that all directly linked cortical areas would also be indirectly connected via the pulvinar (Sherman 2016, 2017). These cortico-pulvino-cortical pathways would improve the signal-to-noise ratio of information transmitted by modulating the synchrony between these areas (Fries 2015). These cortico-thalamo-cortical pathways would be faster than cortico-cortical projections. The multimodal convergence and interactions we report at the single-unit level suggest that the replication principle could also apply to multisensory–motor interactions between distant cortical regions (Sherman and Guillery 2002; Cappe, Morel, et al. 2009). Considering abovementioned 3 points, the pulvinar would be involved in sensory distractor filtering (Fischer and Whitney 2012) and in visual selection and attention (Saalmann and Kastner 2009), allowing the detection of important stimuli (behaviorally relevant or requiring fast motor responses for example). In addition, the pulvinar may be an important modulator of behavioral flexibility, contributing to the selection of sensory inputs (Froesel et al. 2021). Current hypotheses state that the pulvinar has evolved by selection pressure to allow fast and accurate detection of predators such as snakes or emotional faces (Isbell 2006; Maior et al. 2010; Van Le et al., 2013, 2014, 2016; Arend et al. 2015). Moreover, threat recognition is altered in patients with lesions restricted to the medial part of the pulvinar (Ward et al. 2007). Unlike in the studies by Van Le et al. (2013, 2014), we found that the strength of the response to snake and macaque stimuli were similar. This discrepancy may be explained by the fact that our monkeys were keeping fixation while passively viewing or listening to the stimuli. We also repeatedly used a single potentially threatening stimulus to which the monkey had probably quickly become accustomed. On the other hand, and in agreement with these previous studies (Van Le et al. 2013, 2014), we found that visual latencies were faster for snake compared with macaque faces. Our multisensory indexes revealed mostly suppressive and subadditive interactions. Although beyond the scope of our data, we hypothesize that relevant and unexpected multimodal stimuli would lead to response enhancement and possibly supra-additive integration in order to trigger faster adaptive responses. One perspective to explore this function would be to record neuronal activity in response to stimuli of different saliency and of varying relevance for the monkey while monitoring its behavioral response.