Authors: Yong Jiang, Sheng He, Jiedong Zhang
Categories: Biological Sciences, attention, covariability, early visual cortex, posterior parietal cortex
Source: Proceedings of the National Academy of Sciences of the United States of America
It is important to understand mechanisms of information encoding in the brain, for which the covariability of neural responses is highly relevant. Recent neurophysiology studies have revealed that cognitive processes such as attention could improve neural encoding through modulating the covariability in the neuron population. However, it is unclear whether covariability modulation could generally lead to improved information encoding throughout the information processing stream or only in certain stages of neural information processing. Our results show that attention significantly modulated covariability and benefited information encoding in the early visual cortex but not in the posterior parietal cortex, even though stimulus information was encoded in both regions. Covariability modulation is differentially adopted in different cortical regions, depending on the representation property.
Keywords: attention, covariability, early visual cortex, posterior parietal cortex
The variability of neural responses is broadly observed across the neural system, even with the same sensory input. These variabilities can be shared among individual neurons, showing response covariability in the neuron population, which is highly relevant to the quality of neural encoding of sensory information (1–6). Recent evidence shows that cognitive processes, like attention, could modulate the covariability in neuron populations to improve the representation in the sensory cortex (7–18). Specifically, to the same visual inputs, correlation of trial-to-trial variability among neurons (i.e., noise correlation, one of the ways to measure the covariability) is modulated by attention state, and such modulation is found to be highly relevant to the behavioral performance (9). While plenty of evidence, primarily from monkey neurophysiology, has demonstrated the covariability modulation within the sensory cortex, it is unclear whether the covariability modulation could generally enhance stimulus encoding that is attended throughout the information processing pathway or it is only adopted at certain stages during attentional modulation. Further, if the covariability modulation could only be observed in certain cortical regions, what properties in neural representation are potentially driving it across the visual pathway?
In the early visual cortex, attention on average decreases the covariability in the neuron population. Both theoretical analysis and experimental evidence suggest such modulation benefits stimulus encoding and information readout for behavioral output (1, 2, 6, 7, 9, 19), as covariability could not be averaged out and is detrimental to the estimation of neural representation during information readout. Beyond the early visual cortex, the posterior parietal cortex (PPC) has been shown to process and store abstract visual information that is highly task relevant (20–24). Given that the visual information is encoded in the PPC as well, it is unclear whether attention modulates the covariability in PPC similarly as in the early visual cortex (2).
Meanwhile, the PPC is also highly involved in guiding spatial attention in the visual system. It is considered an important region to combine goal-directed and stimulus-driven information to generate spatial priority maps for attentional allocation (25–28). Therefore, it is possible that neural response patterns in the PPC drive the covariability modulations in the early visual cortex. Specifically, would the neural response patterns in the PPC predict the degree of covariability modulation in the early visual cortex?
To investigate the attentional modulation of covariability within the early visual cortex and within the PPC, as well as the shared variability between them during visual processing, in the current study, ultrahigh-field (7T) MRI was used to examine neural responses in the human brain. Although the covariability in neuron populations of monkey’s early visual cortex has been extensively studied, corresponding neural mechanisms in the human visual cortex are not fully investigated, it is important to examine the existence of the corresponding mechanisms, especially the role of attention in the human brain (3, 29). Here, taking advantage of high-resolution of high-field fMRI and its relatively large coverage, we investigated the trial-to-trial fluctuations of fine-scale neural responses in both the early visual cortex and PPC and their layer profiles. Results show that in the early visual cortex, attention clearly modulated the covariability among voxels and benefited the visual stimulus encoding. In addition, such modulation was primarily seen in the superficial layer. In contrast, covariability modulation was not observed in the PPC, where the covariability was shown to have little impact on information encoding. Moreover, attention enhanced the shared variability between the early visual cortex and PPC, apparently emphasizing the transition between high-dimensional representation in the early visual cortex and low-dimensional representation in the PPC. Finally, the spatial patterns of neural response in the PPC could predict the degree of covariability modulation in the early visual cortex, supporting that the top–down mechanism drives the covariability in the sensory cortex.
During the 7T fMRI scan, participants performed an orientation discrimination task (Fig. 1A). A cross was continuously presented at the center of the screen, and participants were asked to fixate on it during the task. In each trial, after an auditory cue indicating the starting of the trial, the left or right arm of the cross turned thicker to indicate the location to be attended. Then, after 2.5 s, a grating was presented at the left side and another at the right side of the fixation with a duration of 0.2 s. The orientation of the left grating was tilted 2° from horizontal with either clockwise (CW) or counterclockwise (CCW), which was randomized across trials. The orientation of the right grating was near vertical with the same amplitude of tilt, and the tilt direction was independent from the left grating. Participants were asked to report the tilt direction of the grating on the attended side, by key pressing 6 s after the stimulus onset, prompted by another auditory cue. An auditory feedback on correctness was given at the end of the trial. Eleven participants completed the experiment, and each participant finished 208 trials in the scanner within 2 h. All participants could follow the instruction, and the mean accuracy of the performance was 84.26%.
Fig. 1. Experiment task and attentional modulation on neural responses in V1. (A) Behavioral task in the scanner. Participants were asked to identify the grating orientation (2° tilted either CW or CCW) at the attended location, which was indicated by the central cue. (B) ROI in the early visual cortex and PPC shown on the cortical surface. The color bar indicates the response significance to visual grating compared with baseline. The Inset shows the positioning of the slices of fMRI scan. (C) Averaged fMRI response time course of attended or unattended trials in V1. The black bar on the time axis indicates the presenting time of the grating stimulus. The gray shaded region indicates the peak duration of the response time courses, in which the fMRI signals were extracted as the neural responses for single trials to calculate the noise correlation in the voxel population. The colored shaded regions reflect ±1 SEM. (D) The averaged noise correlations of attended and unattended trials in the voxel population from V1. Error bars reflect ±1 SEM. ∗∗ indicates the paired t test with significance of P < 0.01.
Functional scan with a resolution of 1 mm isotropic was used to cover the early visual cortex and the posterior part of the parietal cortex. Two regions of interest (ROIs) were identified in each hemisphere for processing the visual gratings (grating presenting vs. baseline) in the current one in V1 and another in the posterior intraparietal sulcus (pIPS) in the PPC (Fig. 1B). The location of the pIPS was consistent with the retinotopic organized brain regions IPS1-3, which is functionally similar to the lateral intraparietal cortex in the monkey brain in the processing hierarchy of the dorsal pathway (30).
With an independent dataset from localizing the ROI, neural responses were estimated for both attended and unattended conditions in V1. Not surprisingly, stronger fMRI responses were observed for the attended condition (attending to the contralateral visual field of the ROI) than the unattended condition (attending to the ipsilateral visual field) [t(10) = 2.65, P = 0.02, using peak responses that averaged across 4 to 6 s] (Fig. 1C). The neural responses were also estimated for each hemisphere separately, and no hemispheric differences in the attentional modulation effect were observed (SI Appendix, Fig. S1).
Here, we focused on the attentional modulation on covariability in V1 by estimating the noise correlation of trial-to-trial variabilities in fMRI responses between each pair of voxels within V1. To estimate the noise correlation within each ROI, for each voxel and each trial, the neural responses were calculated as the differences between the peak responses and the responses at the beginning of the trial. Then, the correlation of neural responses between a pair of voxels across attended trials or across unattended trials was calculated, and the correlations of all possible voxel pairs within the ROI were averaged to estimate the noise correlation of this ROI. The noise correlations in the left and right V1 were estimated separately and then averaged in each individual. To balance the base correlation amplitudes between hemispheres, the attended and unattended noise correlations in each hemisphere were normalized to the mean of the attended and unattended value before averaging across hemispheres (Materials and Methods).
The results showed that within V1, the noise correlations in the attended condition were significantly lower than those in the unattended condition [t(10) = 4.02, P = 0.002] (Fig. 1D), indicating that attention decreased the noise correlation in the sensory cortex. To further investigate this observation, several additional analyses were performed. First, attention might enhance the neural representation of the slight variations of grating orientations (2° CW vs. 2° CCW) across trials, which may in turn potentially affect the estimations of attentional modulations on noise correlation. To address this concern, the noise correlations were calculated separately for CW and CCW trials and then averaged to estimate the noise correlations in attended and unattended conditions. The results showed essentially the same effect of attentional modulations on noise correlation in V1 [t(10) = 4.16, P = 0.002] (SI Appendix, Fig. S2A). Second, the SD of the neural responses in each voxel across trials could be different in attended and unattended conditions, which could potentially influence the noise correlation estimates. To address this concern, SDs were estimated separately in attended or unattended condition, and they were not significantly different [t(10) = 1.81, P = 0.10] (SI Appendix, Fig. S2B). In addition, significant decreases of noise correlation were observed in the attended than in unattended condition in similarly localized ROIs of V2/V3 [t(10) = 2.36, P = 0.04] (SI Appendix, Fig. S2C). Together, these results support that attention indeed modulated covariability of fMRI measured neural responses in the early visual cortex.
Next, the spatial profiles of attentional modulation of noise correlation were examined. Specifically, the relationship between changes in noise correlation due to attention and the spatial distance between a pair of voxels was estimated. Voxel pairs were grouped based on the between-voxel distances in the volume space, and the noise correlations in attended and in unattended trials were calculated for each group separately. With 2 × 5 (attended/unattended × different distance groups) ANOVA, the results showed a significant main effect of attention state [F(1, 10) = 7.81, P = 0.02] and a significant main effect of spatial distance [F(4, 40) = 3.24, P = 0.02], but the interaction was not significant [F(4, 40) = 0.13, P = 0.97] (Fig. 2A). These results revealed a general decrease of noise correlation with the increased distance between voxels, but the degree of attentional modulation was invariant to the distance between voxels.
Fig. 2. Spatial and temporal profiles of covariability modulation. (A) The noise correlation of attended and unattended trials with voxel pairs grouped based on spatial distance. Similar attentional modulations were observed across different distance groups. (B) Schematic illustration of temporal filters being applied to the trial-to-trial neural response sequence. The x axis represents the trial number in the whole experiment. (C) The noise correlations in attended and unattended conditions in the high-passed sequences and low-passed sequences. Attention modulation of noise correlation was only observed in the fast fluctuation of trial-to-trial responses. ** indicates the paired t test with significance of P < 0.01. ∗ indicates the paired t test with significance of P < 0.05.
The covariability was measured with noise correlation, which was calculated based on the response variation of each voxel across the whole scan. These variations were composite of relatively fast fluctuations (e.g., the trial-to-trial fluctuations) and relatively slow fluctuations (31–33) (e.g., the state changes like arousal level). The covariability modulation induced by attention could be contributed by either of these two components, or both. To examine the timescales of response fluctuations driving the covariability modulation, for each voxel, the neural responses for each trial (peak vs. baseline) were concatenated based on their temporal order to generate the neural response sequences. The sequences of attended trials and unattended trials were generated and analyzed separately. Then, a low-pass (more than 4 trials per cycle) or a high-pass (less than 4 trials per cycle) temporal filter was applied to the neural response sequences, and noise correlations were separately estimated for the filtered sequences (Fig. 2B). Results showed a significant decrease of attention-induced noise correlation only in the high-pass sequences [t(10) = 3.39, P = 0.007], but not in the low-pass sequences [t(10) = 1.21, P = 0.25], and the interaction was significant [F(1, 10) = 7.39, P = 0.02] (Fig. 2C). These results indicate that the relatively fast trial-to-trial fluctuations over seconds, rather than the slow drift of states over many minutes, mainly contributed to the covariability modulation induced by attention.
Presumably, the decreased covariability in the attended condition is beneficial, one idea is that decreased covariability may improve the stimulus representations within the sensory cortex. To test this idea, classifiers were trained to decode the stimulus information in V1, and the relationship between decoding performances and the covariability was tested. The results showed that the orientation information (CW vs. CCW) could be successfully decoded in V1 in the attended condition (Support Vector Machine, SVM decoding accuracy = 56.3%, P = 0.04), but not in the unattended condition (accuracy = 50.9%, P = 0.80) (Fig. 3A). As attention will improve the behavioral performance to the task-relevant information, it may primarily modulate the covariability among neural responses encoding task-specific information. To test this prediction, first, the voxels in V1 were sorted based on their ability to discriminate (absolute t-value) two visual stimuli (CW vs. CCW) in the attended condition; then, the 20 most and 20 least discriminative voxels were selected and the noise correlations were separately estimated within each group. Results showed significant attention-induced decreases of noise correlation in the most informative voxel group [t(10) = 2.85, P = 0.02] (Fig. 3B), but not in the least informative group [t(10) = 1.62, P = 0.14]. The interaction was not significant [F(1, 10) = 3.47, P = 0.10]. Additional analysis was also done in the way that noise correlation was first estimated within CW or CCW trials (similar to the analysis for SI Appendix, Fig. S2A), and similar results were observed (i.e., significant attentional effect in the most informative voxel group [t(10) = 3.17, P = 0.01], but not in the least informative voxel group [t(10) = 1.35, P = 0.21], with marginally significant interaction [F(1, 10) = 4.05, P = 0.07].
Fig. 3. The relationship between covariability and information encoding in the V1 neural population. (A) Performances of stimulus orientation decoding in V1. The orientation information could be successfully decoded in the attended condition. (B) The noise correlations in voxel populations that were highly discriminative or weakly discriminative to stimulus orientations (CW vs. CCW, estimated by absolute t value). Attention significantly decreased the noise correlation in the high discriminative voxel population. (C) The neural responses of the voxel population with covariability removed (shuffled) showed better orientation decoding performances than that based on original neural responses. (D) The attention-induced noise correlation decrease was more prominent in the superficial layer in V1. Error bars reflect ±1 SEM. * indicates the significance of P < 0.05.
In addition to voxel sorting, the trials were also sorted based on the quality of their information representation (i.e., distances to the hyperplane of the visual orientation decoder). In V1, there was a trend for more informative trials to have lower noise correlation (SI Appendix, Fig. S3).
To further test the relationship between the decrease of covariability and the enhancement of stimulus representation, the covariability was removed within the neural responses in V1 to see whether the stimulus representation, as reflected in the decoding performances, would improve after the manipulation. Specifically, the temporal sequence of each voxel’s trial-to-trial response was shuffled independently, thus removing the covariability among voxels but leaving the mean response and the variance of each voxel the same as before shuffling (see details in Materials and Methods). Such shuffling was repeated multiple times and the distribution of decoding accuracy was estimated. The result showed that the removal of covariability among voxels enhanced the stimulus encoding in V1, leading to the improved decoding performances (Cohen’s d = 1.55) (Fig. 3C).
In addition, the high-field fMRI allowed us to examine the neural responses in different cortical layers in V1 (deep, middle, and superficial). A two-way ANOVA showed that the interaction between layer and attention was not significant [F(2, 20) = 0.41, P = 0.67]. In each layer, a significant decrease of noise correlation in the attended condition was only found in the superficial layer [t(10) = 2.80, P = 0.02], not in the middle and deep layer (ts < 0.15, ps > 0.88) (Fig. 3D).
After examining the attentional modulation on covariability and its relationship with stimulus representation in V1, we then investigated the covariability and neural representation in different attention states in the pIPS. Here, both the covariability within the pIPS and between the pIPS and V1 were examined. Within the pIPS, attention significantly increased the neural responses to the visual gratings (averaged across 4 to 6 s as peak responses, t(10) = 5.70, P < 0.001) (Fig. 4A), and no significant difference was observed for the neural response variance across trials (SD) [t(10) = 1.81, P = 0.10]. However, no significant difference was observed for the noise correlation between attended and unattended conditions within the pIPS [t(10) = 0.18, P = 0.86] (Fig. 4B). As stronger covariability modulation was observed in more informative voxels in V1 (Fig. 3B), here in the pIPS, to examine the potential attention modulation of noise correlation related to informativeness of voxels, the voxels were sorted and grouped based on their differential responses to grating orientation (absolute t-value, CW vs. CCW, same as the voxel grouping analysis in V1). Meanwhile, as significant decoding performances were obtained for attended condition only in the left pIPS (accuracy = 58.5%, t(10) = 2.68, P = 0.02; for the right pIPS, accuracy = 44.9%, t(10) = 1.04, P = 0.32, the difference between two hemispheres was significant, t(10) = 2.50, P = 0.03), indicating that the valid discriminative information was more prominent in the left pIPS fMRI signal; thus, the voxel grouping analysis was only conducted in the left pIPS. Note that there was no hemispheric difference of attentional effect on BOLD signal changes [F(1, 10) = 0.02, P = 0.87]. For the noise correlation, the interaction between attention and hemisphere was marginally significant [F(1, 10) = 4.39, P = 0.06], but the attentional effect was not significant in either hemisphere (ts < 1.86, ps > 0.09) (SI Appendix, Figs. S1 and S4). The top and bottom 20 informative voxels were selected, and the noise correlation was estimated in each group separately. However, a significant decrease of noise correlation induced by attention was observed in neither voxel groups [for the most informative voxel group, t(10) = 0.47, P = 0.76; for the least informative group, t(10) = 1.78, P = 0.11] (Fig. 4C). Taking these observations together, in the pIPS, the attention increased the neural responses and promoted the stimulus encoding, but did not impact the covariability, even within the most informative voxel group. One potential explanation of these observations is that, unlike in the early visual cortex, the neural representations in PPC receive little benefit from the modulation of interneuron covariability; thus, attention does not change the noise correlation during the stimulus encoding. To examine this explanation, the noise correlation was removed within the left pIPS to see whether there was any improvement in information encoding, which is the same analysis that was conducted in V1 (Fig. 3C). The result showed that while in V1 the removal of noise correlation strongly boosted the decoding performances (Cohen’s d = 1.55), it only slightly benefited the decoding performances in the IPS (Cohen’s d = 0.33) (Fig. 4D).
Fig. 4. The neural modulation of attention in the pIPS. (A) Averaged fMRI response time courses of attended and unattended trials in the pIPS, with the shaded region indicts the peak period (same as that in V1). The colored shaded regions reflect ±1 SEM. (B) The averaged noise correlations of attended or unattended trials in the pIPS, with no significant difference being observed. (C) The noise correlation in different attention conditions and in voxel groups with different orientation discriminability. No significant difference was observed in either group. Error bars reflect ±1 SEM. (D) The removal of covariability had little benefit on orientation decoding performances.
In addition to revealing the different mechanisms of attentional modulation of within-region covariability between the early visual cortex and the PPC, it is important to examine the attentional modulation on the between-region covariability, related to the information flow between two regions during the task. Within V1, different neurons process similar visual features extracted from the previous stage in the hierarchy (e.g., Lateral Geniculate Nucleus, LGN), a decreased covariability leads to generally more independent processing between neurons, which benefits stimulus encoding. In contrast, given that the visual features encoded in the pIPS are extracted from the sensory cortex, an increased trial-to-trial correlation between V1 and pIPS would reflect more efficient information transfer and thus benefit the neural representation in the visual system. Consistent with this view, in previous studies, the trial-to-trial correlation between two regions was also used to estimate the functional connectivity between the regions. To estimate the covariability between two regions, the neural responses were averaged across voxels within each region, and the correlation of trial-to-trial neural response fluctuations between V1 and pIPS was estimated. The group-averaged correlation between V1 and pIPS was 0.50 in attended trials and 0.48 in unattended trials. Unlike the decreased noise correlation within V1, the trial-to-trial response correlation between V1 and pIPS increased in the attended condition [t(10) = 2.31, P = 0.04] (Fig. 5A), suggesting stronger information flow between the two regions during attended trials. Further, principal component analysis (PCA) and canonical correlation analysis (CCA) were used to reveal the nature of attentional enhancement of information flow. Recently, studies have demonstrated that the attention modulation in the sensory cortex is “low rank” (6, 9, 19, 34). Here, PCA was first used to extract top 40 components (dimensions) explaining most neural response variance in each brain region. Then with CCA, the correlations between two brain regions were estimated for attended and unattended conditions separately, and the correlation difference was calculated. Such procedure was repeated for different dimension combinations (e.g., top N1 dimensions in V1 vs. top N2 dimensions in the pIPS) to generate a 40-by-40 matrix with each point representing the correlation difference averaged across participants, and the cluster permutation test was used to identify the significant correlation changes between attended and unattended conditions. Results revealed that attention enhanced correlation between the two regions, with the enhancement reaching a significant level in two clusters (Fig. 5 B and C). One cluster was observed between high dimensions (30 to 40 dimensions) in V1 and low dimensions (1 to 2 dimensions) in the pIPS, and the other cluster was between high dimensions in both V1 (20 to 25 dimensions) and pIPS (37 to 40 dimensions). The observed attention-induced correlation boost between high-dimensional representation in V1 and low-dimensional representation in the pIPS suggests that attention may improve the transmission from high-dimensional to low-dimensional neural representations in the visual processing pathway.
Fig. 5. Attentional effects on the information flow between the pIPS and V1. (A) The noise correlation between the pIPS and V1. Stronger correlations were observed in the attended than in unattended condition. Error bars reflect ±1 SEM. ∗ indicates the significance of P < 0.05. (B) The attentional effect (attended vs. unattended) on correlation between the pIPS and V1 estimated by CCA with different number of principal components included in each region. The solid lines indicate the significant clusters. The significant cluster located on the upper left suggests that attention boosted the information flow between the high-dimensional representation in V1 and low-dimensional representation in the pIPS. The dashed lines indicate the range of data presented in the figure below. (C) Attention enhanced correlation between neural representations in the pIPS and high dimensional representations in V1 (30 to 40 components). The result showed that the attention enhanced the correlation between two regions, and the attentional enhancement peaked at the first one or two components in the pIPS. With more pIPS components included, the correlation enhancement became lower.
While no significant modulation of covariability driven by attention was found within the PPC, the PPC and other downstream regions in the dorsal pathway could still play a key role in modulating the covariability modulation in the early visual cortex. To examine this possibility, we tested whether the neural responses in the pIPS could predict the degree of attentional modulation of noise correlation in V1. The experimental trials were sorted into two groups based on the level of lateralization of the neural responses in the pIPS (i.e., high vs. low fMRI response difference from the contralateral–ipsilateral to the attended visual field). Then, the noise correlations in V1 were estimated separately in each trial group. In the trial group with high pIPS-response lateralization, attention significantly decreased the noise correlation in V1 (Fig. 6, Left) [t(10) = 2.37, P = 0.04], but not in the trial group with low pIPS-response lateralization [t(10) = 1.33, P = 0.21]. The interaction was not significant [F(1, 10) = 1.25, P = 0.29]. To examine the hemispheric difference, a modified analysis was performed in each hemisphere. Specifically, for left V1, attended trials were sorted based on the fMRI responses in left IPS, and the difference in noise correlation in left V1 was not significant between high-IPS-response and low-IPS-response trials [t(10) = 0.79, P = 0.45]. When the same analysis was performed in right V1 based on the right IPS responses, the noise correlation was lower in high-IPS-response trials than in low-IPS-response trials [t(10) = 2.64, P = 0.02]. The interaction was not significant [F(1, 10) = 2.44, P = 0.15] (SI Appendix, Fig. S5). To further test whether certain cortical layers in the pIPS play a more important role in driving the noise correlation in V1, pIPS neural responses were extracted from three layers and were analyzed separately. A three-way ANOVA showed that the interaction among layer, attention, and contralateral bias was not significant [F(2, 20) = 0.47, P = 0.63]. The top–down modulation was found in the superficial layer of the pIPS [stronger modulation in the high response lateralization trial group than in the low response lateralization trial group, F(1, 10) = 5.74, P = 0.04], but the effect was not significant in the other two layers (ps > 0.26) (Fig. 6, Right). Finally, consistent with a top–down modulation interpretation, V1 noise correlation could be predicted from response lateralization of the pIPS, but not from the response lateralization of V1 itself, as the interaction between attention and contralateral bias was not significant [F(1, 10) = 0.98, P = 0.35]. No significant noise correlation change between attended and unattended conditions in V1 was found in either high or low lateralized group (ts < 1.19, ps > 0.26) (SI Appendix, Fig. S6).
Fig. 6. Neural response bias in the pIPS predicting covariability changes in V1. The trials were grouped based on the contralateral bias of pIPS neural responses to the attended visual field (contralateral vs. ipsilateral). The attention-induced noise correlation decrease in V1 was observed only in the trial group with high contralateral bias of the pIPS (left column). Such effect was most prominent in the superficial layer of the pIPS (right column). Error bars reflect ±1 SEM. ** indicates the significance of P < 0.01. * indicates the significance of P < 0.05.
Our results show that attention generally decreased the covariability in the early visual cortex, especially the superficial layer, to improve the neural representation to discriminate subtle changes in visual orientation. In contrast, the covariability had little impact on neural representations in the PPC and was not modulated by attention. In other words, the PPC and early visual cortex played different roles in covariability modulation. However, attention did promote the information flow between the early visual cortex and PPC. Further, CCA analysis reveals that part of the enhanced information flow was contributed by stronger correlation between high-dimensional variability in the early visual cortex and low-dimensional variability in the PPC. Moreover, the PPC, especially in the superficial layer, could predict the changes of covariability in the early visual cortex, suggesting that the PPC could drive the top–down modulation of covariability in the early visual cortex.
Our fMRI results are highly consistent and extend previous findings from monkey neurophysiology studies (7–10, 12–14, 35, 36). The use of ultrahigh field 7T fMRI allowed us to attribute such attentional modulation in V1 primarily to the superficial layer, which is consistent with previous observation in the monkey visual cortex (14), suggesting that the modulation is driven by feedback signals from higher visual system (37–41). In addition, the broad field of view of fMRI and extended measures over time enabled the examination of spatial and temporal scales of neural response variations in which the covariability is mostly modulated. Our results suggest that within spatial scale from 1 mm to about 15 mm, the attentional modulation on covariability is invariant to the interneuronal distance. For temporal scales, two components of neural response fluctuations were examined in the current study. Our results show that the attention-induced noise correlation decrease was found only in the fast component of trial-to-trial response fluctuations within a minute or faster, rather than the slow state drift (31–33) over the timescales of minute or slower.
In the PPC, attention boosted neural responses and improved the orientation representations, similar to the observations in the early visual cortex. However, different from the early visual cortex, attention did not modulate covariability in the PPC, even in the voxel population with high sensitivity to the orientation, presumably due to the lack of benefit from reducing the covariability in the PPC where visual features are abstractly represented, in contrast to the demonstrated benefit of covariability reduction to the stimulus encoding in the sensory cortex. In support of this interpretation, little improvement to the orientation decoding performances was found after removing the covariability in the PPC voxel population.
Thus, the covariability contributes differently at different stages of visual processing hierarchy, but why? Evidence from both monkey neurophysiology and human fMRI has shown that the PPC extracts task-relevant visual information from the early visual cortex for further computation to generate behavioral output (20–23), and possibly, it may not inherit their covariability among neuronal responses. In the early visual cortex, one of the potential sources of covariability is the high-dimensional representation generated from low-dimensional input from the retina or LGN (i.e., more V1 neurons than retinal ganglion cells) (2). However, categorical representations are computed in the PPC, which are likely low dimensional, to further guide behavioral output. Consequently, the covariability that impacts neural representation in the early visual cortex is unlikely to be preserved during the transition from high-dimensional early visual cortical to low-dimensional PPC representations.
Supporting this interpretation, the correlation of trial-to-trial neural response variability between the early visual cortex and PPC, which represents the information flow between these two regions, was enhanced by attention (36, 42, 43). Moreover, CCA analysis showed that such attentional enhancement was significantly driven by two kinds of correlations, one of them was between high-dimensional representation (more than 30 dimensions) in early visual cortex and low-dimensional representation (1-2 dimensions) in the PPC, suggesting that the attention not just enhanced the transition of complex visual representations between two regions, but such enhancement was biased toward the transition of complex high-dimensional representation into a low-dimensional task-relevant neural coding (20–23). These observations are consistent with previous monkey neurophysiology studies that showed attention has low-rank effect and attentional enhancement of low-dimensional representations in interregion communication through a shared subspace (34, 42–44). In our study, the low-dimensional representations were found in the PPC of the human brain. Note that in the CCA analysis, the threshold for a significant attentional effect was calculated by a cluster-based permutation test, which may fail to detect attentional modulation with a small effect size.
These results provide clear evidence to clarify the generality of covariability modulation in the information processing hierarchy. They suggest that the attentional modulation of covariability does not generally occur across the neural system. Apparently, whether the covariability is modulated depends on whether the information encoded there benefits from covariability modulation, which is further related to the dimensionality of the neural representations in specific brain regions. Attention modulates the neural representations in different ways between different processing stages to maximize the benefit of the modulation to information encoding, demonstrating the strong flexibility and efficiency of the cognitive system. Moreover, the current study provides evidence for the covariability modulation within the PPC, as well as for the bottom–up transmission of information and the top–down control of covariability modulation in V1 from the PPC. These findings are important for understanding how covariability is transmitted and influences neural representation at the system level of the whole brain.
While the reduction of covariability in the early visual cortex improves the stimulus encoding, what are the neural sources driving such modulation in the brain? Our results showed that the lateralized neural response in the PPC could predict the degree of the covariability modulation, which suggests that the PPC serves as one of such neural sources. Top–down attention involves multiple neural networks over the whole brain, including the frontal eye field in the prefrontal cortex and subcortical region superior colliculus (45). Lesion of frontal eye field reduced but did not abolish attentional modulation of neural responses as well as their covariability in the sensory cortex, suggesting that the attentional feedback signal could originate from additional brain regions (13). Previous studies have demonstrated that the PPC may receive both top–down and bottom–up attention information and encode priority map to guide attention in the visual field (25–28). Therefore, it is reasonable that the PPC specifically modulates the subpopulation of neurons in the early visual cortex that corresponds to the location highlighted in the priority map and reduces the shared variability within the neuron population. While the PPC could potentially serve as another region sending modulatory signal to the sensory cortex, its causal role in covariability modulation needs to be further examined in future lesion studies.
While the early visual cortex generally showed similar results between two hemispheres, some hemispheric differences were observed in PPC. First, the orientation decoding performances were significant in the left PPC, but not in the right PPC, suggesting that stimulus feature information extracted from the early visual cortex was better represented in the left PPC. Meanwhile, neural responses in the right PPC prominently modulated the covariability in the early visual cortex, suggesting stronger top–down modulations from the right PPC. The right lateralization of top–down attentional modulation is consistent with extensive observations in lesion studies that neglect is often observed in individuals with right parieto-temporal lesions (46). However, there is no clear evidence found in the literature for the left lateralization of task-relevant information in the PPC, although there were reports that the left PPC plays a more dominant role in the mental imagery task (47). It is unclear whether the same mechanism in the left PPC was recruited in the current task as in the mental imagery task, although the sources of input may be different.
Covariability modulation has been demonstrated to have computational benefits in many neural processes and has been observed across many neural systems in multiple species. Our results reveal that the covariability modulation induced by attention is not uniformly adopted but is adaptively implemented in different processing stages, depending on its benefit to the stimulus encoding. The nature of covariability and the fidelity of information representation in the neuron population may drive the relationship between covariability and information encoding.
Eleven (five females; age 23 to 28 y) participants took part in the experiment. All participants had normal or corrected to normal visual acuity. They were recruited from the Chinese Academy of Sciences community, provided written informed consent, and received payment to compensate for the time they spent in the experiment. The experimental protocol was approved by the Committee on the Use of Human Subjects of the Institute of Biophysics, Chinese Academy of Sciences (#2017-IRB-004).
The grating stimulus was generated via the Psychtoolbox (http://psychtoolbox.org/) and MATLAB (MathWorks, Natick, MA, USA) using a Mac Pro (Late 2013). Stimuli were presented on a translucent screen behind the head coil of the MRI scanner using a projector (1,024 × 768 pixels, 60 Hz). Participants watched the stimuli through a mirror above their eyes tilted at 45° that was attached to the head coil.
During the whole experiment, a fixation cross was continuously presented at the center of the screen, and participants were asked to maintain fixation. They completed eight runs with each run including 26 trials and a 6-s blank period at the beginning of the run. In each trial, after an auditory cue indicating the starting of the trial, the thickness of the left or right arm of the cross increased to indicate the locations need to be attended. Then, after 2.5 s, a grating was presented at each side of the fixation in the lower visual field for 0.2 s. The gratings were 7° in diameter and presented at an eccentricity of 5°. The orientation of the left grating was near horizontal with a 2° tilt either CW or CCW, which was randomized across trials. The orientation of the right grating was near vertical also with a 2° tilt, and the tilt direction was independent from the left side. Participants were asked to identify the tilt direction of the grating on the attended side and respond by key pressing after another auditory cue presented 5.8 s later. An auditory feedback was given at the end of each trial. Eleven participants finished the experiment and each participant finished 208 trials in the scanner with duration around 2 h.
MRI data were collected on a Siemens Magnetom 7 Tesla MRI system (passively shielded, 45 mT/s slew rate) (Siemens, Erlangen, Germany), with a 32-channel receive 1-channel transmit head coil (NOVA Medical, Inc, Wilmington, MA, USA), at the Beijing MRI Center for Brain Research. High-resolution T1-weighted anatomical images (0.7 mm isotropic voxel size) were acquired with an MPRAGE sequence (256 sagittal slices, acquisition matrix = 320 × 320, field of view = 223 × 223 mm, GRAPPA factor = 3, TR = 4,000 ms, TE = 3.05 ms, TI = 0 ms, flip angle = 0°, pixel bandwidth = 240 Hz per pixel). Gradient Echo-Echo Planar Imaging (GE-EPI) sequences were used to collect functional data in the main experiment (TR = 2,000 ms, TE = 25.2 ms, 1.0 mm isotropic voxels, FOV = 128 × 128 mm, image matrix = 128× 128, GRAPPA factor = 3, Flip angel = 80, partial Fourier 6/8, 32 slices of 1.0 mm thickness, flip angle is about 80, pixel bandwidth = 1,184 Hz per pixel). Because of the limitation of FOV, for 4 participants with larger brain, GE-EPI parameters were slightly modified to cover both occipital and parietal cortex (TR = 2,400 ms, TE = 25.2 ms, 1.0 mm isotropic voxels, FOV = 128 × 128 mm, image matrix = 128 × 128, GRAPPA factor = 3, Flip angel = 80, partial Fourier 6/8, 39 slices of 1.0 mm thickness, flip angle is about 80, pixel bandwidth = 1,184 Hz per pixel). During the scan, GE-EPI images with reversed phase encoding direction from experiment functional scan were collected to correct the spatial distortion of EPI images.
Anatomical data were analyzed with FreeSurfer (CorTechs Inc, Charlestown, MA, USA) and custom MATLAB codes. To reconstruct the cortical surfaces, anatomical data were further processed by FreeSurfer, including gray and white matter segmentation and identification of V1 and V2/V3 regions. SUMA and custom Python/MATLAB codes were used to generate equi-volume surfaces (https://github.com/herrlich10/mripy). For each voxel, its volume percentages of WM, CSF, and different cortical layers (deep, middle, and superficial) were calculated.
Functional data were analyzed with Analysis of Functional NeuroImages (http://afni.nimh.nih.gov), FreeSurfer, and custom MATLAB codes. Data preprocessing included the following slice-timing correction, motion correction, removal of physiological noise due to respiration and pulse signals, distortion correction using reversed-phase encoding EPI images, and intensity normalization. After preprocessing, functional data were coregistered to the anatomic images of the participants.
ROIs in V1, V2/V3, and pIPS in volume space were identified as the voxels that were responsive to the grating stimuli in the task in each cortical area labeled anatomically by FreeSurfer. Each ROI was subsequently defined as a set of continuous voxels showing significantly greater responses to the grating stimuli than the baseline within the anatomical label (P < 0.01, uncorrected).
In cortical layer analysis, the ROIs were identified in the surface space and then projected back to the volume space. The voxels in the ROI were classified into deep, middle, and superficial layers based on their dominant volume percentages generated from the anatomical data analysis.
With the current setup of the scan sequence, the PPC and early visual cortex (V1, V2/V3) could be covered simultaneously in all participants, but the other visual areas such as MT+ and hV4 were outside the FOV in many participants, making it inaccessible to analyze the neural responses in other visual areas.
Two analyses were conducted to remove vasculature-related signals. First, with General Linear Model analysis, the beta values of stimulus-evoked responses were calculated for each voxel, and the distribution of the beta values in each ROI was fitted by two Gaussian distributions. The voxels that fell into the higher response Gaussian distribution were excluded from further analyses (48). Second, the mean EPI signal was calculated for each voxel. The voxel clusters with low EPI signal were excluded from further analyses.
A 12-s fMRI signal time course starting from the grating onset was extracted for each trial. To average the time courses of fMRI signal across different participants scanned with different TRs (2- or 2.4-s), a linear interpolation with steps of 0.4 s was applied to each time course in each participant.
Cross-validation has been used to estimate the fMRI responses in each ROI. Specifically, the top 50 voxels for each ROI were selected with the data from half of the runs, and the BOLD signals were estimated with the data from the other half of the runs.
As shown in the averaged time courses in Fig. 1C, the fMRI responses peaked between 4 and 6 s in each trial and were extracted as the single-trial neural responses, and the fMRI responses at 0 s in each trial were subtracted as baseline from the peak fMRI responses. Then, the correlation of neural responses between a pair of voxels, separately across attended trials and across unattended trials, was calculated to estimate the covariability between these two voxels in the attended and unattended conditions respectively. To balance the voxel numbers in different regions, in each ROI, the top 50 voxels that showed most response boost by attention were selected, and the noise correlations of all possible voxel pairs within these voxels were calculated. The voxel pairs that were spatially adjoining (i.e., spatial distance less than 3 mm) were excluded, and the noise correlations of the remaining pairs were averaged to estimate the covariability in this ROI.
The noise correlations in the left and right hemispheres were estimated separately and then averaged in each individual. To balance the base correlation amplitudes between hemispheres, the noise correlations in each hemisphere were normalized before averaging with the Eq. shown below, in which the NC indicates the Noise Correlation and the subscript i indicates the attention states (attended or unattended).
To examine the significance of the noise correlation between different conditions, r values were transformed to z values using Fisher Transformation prior to statistical testing.
To test the relationship between the intervoxel spatial distance and the covariability, voxel pairs were grouped based on their intervoxel distances in the volume space (i.e., distance < 3; 3 <= distance < 6; 6 <= distance < 9; 9 <= distance < 12; distance >= 12 mm;).
To examine in which timescale the neural response variance is mostly driving the attentional modulation of covariability, different temporal frequency filters were applied to the neural response sequences before the noise correlation analysis. First, for each voxel, peak responses of different trials (single value for each trial) were concatenated based on their temporal order to generate the neural response sequence. Next, the neural response sequences of attended and unattended conditions were generated, with the missing data points (e.g., temporal locations of attended trials in the unattended neural response sequence) filled with linear interpolation. Therefore, the temporal fluctuations for each attentional state were sampled, on average, once every 2 trials, which allowed us to examine the frequency of about 4 trials per cycle with sufficient data. Since we focused more on trial-to-trial variation, the 4 trials per cycle was chosen as the cutoff in the current study. Then, high- and low-pass filters (designed in MATLAB, order = 8; for high-pass filter, passband period was 40 s, stopband period was 48 s; for low-pass filter, passband period was 60 s, stopband period was 48 s) were applied to the neural response sequences, and the outputs (attended high frequency, attended low frequency, unattended high frequency, unattended low frequency) were used to calculate the noise correlations at different timescales.
To decode grating orientation (CW vs. CCW), linear classifier was trained for each ROI and each participant using the latent biconstraint SVM (LBSVM) in MATLAB. Supervised learning and a leave-one-run-out cross-validation approach were used to estimate the decoding performances. For each pair of orientations, the classifier was trained with the dataset of seven neural response patterns (from seven runs), and tested with the neural response pattern from the remaining run. This procedure was performed over eight iterations, and the classification accuracies were averaged to obtain the mean accuracy of orientation decoding. For V1 in each hemisphere, 200 most responsive voxels within the V1 were selected, and significant decoding performance was observed (P = 0.04). For the left pIPS, using top 200 voxels yielded marginal significance in decoding performance (P = 0.08), and using top 100 voxels generated significant performance (P = 0.02, SI Appendix, Fig. S7). Similar results in the shuffling analysis were obtained with either voxel numbers (Fig. 4D and SI Appendix, Fig. S8). Here, the results with top 100 voxels in the pIPS were reported in the manuscript (Fig. 4D).
The classifier was also used to estimate the discriminative information in each trial in each ROI. In each attended trial, in the high-dimensional space of multivoxel responses, the distance from the spatial location of the neural response pattern to the classification hyperplane was used to estimate the discriminative information of that trial (SI Appendix, Fig. S3A).
For each ROI in each participant, a two-dimensional (T by n) data matrix was generated for either CW or CCW trials in the attended condition. T was the trial number and n was the voxel number. To remove the covariability in the voxel population, the trial order (T) was shuffled, independently for each voxel (n), thus eliminating the correlation among voxels but retaining the mean and SD of the neural responses for each stimulus. SVM was used to estimate the decoding accuracy of the shuffled dataset. Such procedure was repeated 1,000 times to generate the decoding accuracy distribution of shuffled datasets. To generate the decoding accuracy distribution for the dataset with the original covariability structure, the similar shuffling procedure was used, with the only difference that all the voxels used the same shuffled order in each repetition, thus preserving the covariability structure in the datasets.
To reveal the dimension properties of neural representations that was modulated by attention, PCA and CCA were used to characterize the information flow between two brain regions. First, with the same data used to calculate the noise correlation in each ROI, PCA was applied to cluster and transfer voxels into different principal components. The top 40 principal components that explained most variances were reserved for further analyses. Then, the CCA was used to estimate the attentional modulation on information flow in different dimensions between two regions. For instance, to estimate the information flow between K1 dimensional neural representation in V1 and K2 dimensional neural representation in the pIPS, neural responses in V1 voxels were projected onto first K1 principal components of V1 and neural responses in the pIPS were projected onto first K2 principal components of the pIPS. Then CCA (MATLAB function canoncorr) was used to estimate the maximal correlation of subdimensional neural representations between V1 and pIPS, and the first pair of canonical dimensions was selected to estimate the correlation. To estimate the attentional modulation, the correlations were estimated for attended and unattended trials separately, and the difference between the two correlations was used to evaluate the attentional modulation on information flow. Such procedure was repeated for all the dimension combinations (K1 and K2) between two regions.
To identify the dimensions of each region between which the correlation was significantly modulated by attention, a cluster-based permutation test was applied to the CCA result matrix. The correlation coefficients in both attended and unattended conditions were pooled and shuffled in each participant, and the largest cluster of significant difference (P < 0.05) between attended and unattended correlation coefficient matrices was identified in the group level. Such procedure was repeated for 10,000 times, and the distribution of the largest cluster size was estimated. The threshold of significance was the cluster size corresponding to the top 1% probability in the distribution.
This work was supported by the Ministry of Science and Technology of China (Grant Nos. 2021ZD0204200 and 2021ZD0203800); Key Research Program of Frontier Sciences, Chinese Academy of Science (Grant No. KJZD-SW-L08); and CAS Project for Young Scientists in Basic Research (Grant No. YSBR-071).
Y.J., S.H., and J.Z. designed research; Y.J. and J.Z. performed research; Y.J. and J.Z. contributed new reagents/analytic tools; Y.J. and J.Z. analyzed data; and Y.J., S.H., and J.Z. wrote the paper.
The authors declare no competing interest.
Sheng He, Email: hes@ibp.ac.cn.
Jiedong Zhang, Email: zhangjiedong@gmail.com.
fMRI data have been deposited in figshare (10.6084/m9.figshare.c.6230499) (49). All other data are included in the manuscript and/or SI Appendix.
fMRI data have been deposited in figshare (10.6084/m9.figshare.c.6230499) (49). All other data are included in the manuscript and/or SI Appendix.