Authors: Michael Wibral, Christoph Bledowski, Axel Kohler, Wolf Singer, Lars Muckli
Categories: Articles, apparent motion, electroencephalography, event-related potential, feedback, visual illusion
Source: Cerebral Cortex (New York, NY)
When 2 visual stimuli are presented one after another in different locations, they are often perceived as one, but moving object. Feedback from area human motion complex hMT/V5+ to V1 has been hypothesized to play an important role in this illusory perception of motion. We measured event-related responses to illusory motion stimuli of varying apparent motion (AM) content and retinal location using Electroencephalography. Detectable cortical stimulus processing started around 60-ms poststimulus in area V1. This component was insensitive to AM content and sequential stimulus presentation. Sensitivity to AM content was observed starting around 90 ms post the second stimulus of a sequence and most likely originated in area hMT/V5+. This AM sensitive response was insensitive to retinal stimulus position. The stimulus sequence related response started to be sensitive to retinal stimulus position at a longer latency of 110 ms. We interpret our findings as evidence for feedback from area hMT/V5+ or a related motion processing area to early visual cortices (V1, V2, V3).
Keywords: apparent motion, electroencephalography, event-related potential, feedback, visual illusion
Sometimes we perceive 2 stimuli that are presented in a temporal sequence in distinct locations of the visual field as just one, but moving, object despite the fact that none of the presented stimuli actually moved. This phenomenon has been termed apparent motion (AM) (Wertheimer 1912; Newsome et al. 1986). This illusory percept can be present even when stimulus locations are separated by distances that are many times the size of receptive fields (RFs) of direction selective neurons in V1 (long-range AM; Larsen et al. 1983). Several studies found evidence for an involvement of feedback processes in the perception of AM (Seghier et al. 2000; Pascual-Leone and Walsh 2001; Antal et al. 2003; Muckli et al. 2005; Silvanto et al. 2005; Larsson et al. 2006; Sterzer et al. 2006). A functional magnetic resonance imaging (fMRI) study from our group demonstrated that area V1 was activated in locations that were not directly stimulated by the individual stimuli but corresponded to locations on the perceived illusory motion path (Muckli et al. 2005). Sterzer et al. (2006) used dynamic causal modeling (DCM) of fMRI data to show that feedback from area hMT/V5+ to area V1 was present when illusory motion was perceived. A transcranial magnetic stimulation (TMS) study by Silvanto et al. (2005) found that it was possible to induce the percept of moving light flashes by subthreshold stimulation of hMT/V5+ when area V1 was stimulated subsequently above threshold. The inverted sequence of stimuli resulted in a static percept.
These studies, together with evidence from animal experiments (Hupé et al. 2001) suggest that feedback from motion processing areas like hMT/V5+ and V3A to V1 is part of the processes necessary for the perception of AM. The timing of these feedback processes in the human brain is still unknown. Although DCM can provide evidence for the presence of feedback processes, its temporal resolution is not sufficient to pinpoint these processes in time.
Measurements of visual-evoked potentials (VEPs) by Electroencephalography (EEG) provide sufficient temporal resolution to address the question when feedback processes take place.
In order to localize feedback processes in time several questions have to be
Here, we measured the VEPs evoked by stimuli that did or did not elicit the percept of AM while manipulating the retinal positions of the stimuli to identify potential generators of the components of these VEPs. Our choice of stimulus positions and manipulations was based on results of earlier studies that used a similar strategy to find the cortical generators of the C1, N1, and P2 VEP components for pattern onset stimuli (Di Russo et al. 2002) and of the N125/P135 component for pattern reversal stimuli (Di Russo et al. 2005). Here, we asked specifically, whether we could find feedback components in the VEP. Moreover we were interested in whether identified feedback components subserve a functional role. We tested this by systematically changing the subjects’ percepts and observing the presence or absence of feedback components.
This study comprised 2 EEG experiments and one fMRI In the first EEG experiment (“experiment I”) we recorded the EEG from subjects while they were viewing illusory downward motion stimuli with varied intensities of the AM percept. In the second experiment (“experiment II”) we recorded EEG while subjects were viewing illusory motion stimuli at varied positions in the visual field. In the third experiment (“fMRI experiment”) the cortical representations of the stimuli from experiment II were mapped with fMRI.
The question of sequence sensitivity was addressed by analysing the difference in the VEPs evoked by a stimulus sequence eliciting a motion percept and the sum of VEPs of the single stimuli used in this respective sequence (this difference is further on called the “sequence sensitive difference wave” or SSDW). This analysis was performed in both EEG experiments (I, II). To answer the question of AM specificity we manipulated the strength of the motion percept elicited by the stimulus sequence and analyzed which parts of the previously observed SSDW were changed by this manipulation (experiment I). We thus obtained information on existence and timing of VEP components related to the AM content. In a third step (retinotopic specificity, experiment II) we tried to localize the generators of the various parts of the SSDW using variations of the retinal positions of the stimuli. Parts of the SSDW originating in cortices with a clear retinotopic organization should be altered by this procedure while those coming from cortices without a retinotopic organization, or one that is not detectable using EEG would remain unaltered. If we found partially independent parts of the SSDW that were either sensitive to AM or sensitive to the retinal stimulus position the question of feedback could be addressed. An occurrence of retinotopy specific components of the SSDW after the onset of AM specific components of the difference wave would be a strong indicator for feedback. The onset asynchrony of these components would then tell about the timing of putative evoked feedback processes.
In both experiments subjects were students and teachers recruited from campus. All subjects had normal or corrected to normal visual acuity. In experiment I VEPs were recorded from 20 subjects (13 males). Of these, 4 had to be excluded. This was due to anatomical irregularities (early childhood hydrocephalus) in one case, to artifactual recordings with large muscle artifacts in one case and exceedingly high impedances in 2 cases. The mean age of the remaining 16 (10 males, 4 left handed [4 males]) subjects was 29.4 years. In experiment II VEPs were recorded from 19 subjects (13 males, 1 left handed, mean 28.5 years). Valid recordings were obtained in all subjects, behavioral responses were recorded in 11 subjects. In 10 subjects mappings of stimuli with fMRI were obtained to verify our assumptions on the cortical representation of the stimuli. Four subjects from EEG experiment I also participated in EEG experiment II.
All stimuli in the EEG experiments I and II were presented on a CRT screen with a frame rate of 100 Hz. For the fMRI experiment stimuli were presented at a frame rate of 75 Hz using an fMRI compatible video goggle system with 2 LED displays (MR Vision 2000, Resonance Technology, Northridge, CA). Stimuli were presented on a dark background (approximately 0 cd/m^2^; brightness was below the resolution of our photometer) while subjects had to fixate a bright fixation cross (83.9 cd/m^2^, 0.3° visual angle) in the screen center. Stimuli consisted of white (83.9 cd/m^2^) squares with a size of 2° visual angle that were presented (flashed) for 200 ms. All stimuli were presented in the right visual field.
The stimuli used in the 2 EEG experiments were grouped into several experimental conditions (Figs 1 and 2). Each experimental condition in turn always consisted of 3 stimulus one stimulus condition containing a temporal sequence of 2 stimuli that elicited a stronger or weaker percept of AM and 2 control stimulus conditions with a single stimulus each (used for later subtraction). In these latter control stimulus conditions the 2 stimuli of the AM stimulus condition were presented in isolation to obtain the respective single stimulus driven VEPs. By comparing the sum of these 2 VEPs in the control stimulus conditions to the VEP in the AM stimulus condition we tried to disentangle sequence or AM related processing from simple single stimulus perception (Fig. 3). To help the reader with the rather large number of different stimulus conditions used here we have developed a consistent naming scheme, presented in Table 1.
Figure 1. Stimuli used in experiment I. (a) Stimulus timing and Each square denotes presentation of the corresponding screen for 200 ms. Black screens with white presentation of the fixation cross alone. Black screen with white presentation of a white square in the corresponding location (details of the stimulus geometry can be found in (b)). The red frame denotes the corresponding baseline interval used in the evaluation of the VEPs. All stimulus timestamps are aligned to the onset of the second stimulus in the corresponding AM stimulus condition (I-AM or I-AMweak) to allow a later comparison of the cortical events following the second stimulus in each AM stimulus condition. From top to (I-AM) AM inducing stimulus condition with presentation of a white square for 200 ms in the upper visual field (position I-U) followed by 200 ms of blank screen and presentation of a second white square (S2) in the lower visual field for 200 ms. (I-U-S1) control stimulus condition corresponding to the timing and location of the first stimulus in the AM stimulus condition I-AM. (I-L-S2) control stimulus condition corresponding to the timing and location of the second stimulus in the AM stimulus condition I-AM. Stimulus conditions I-AM, I-U-S1, and I-L-S2 together formed the “AM” experimental condition. (I-AMweak) stimulus condition consisting of the presentation of a white square for 200 ms in the upper visual field (position I-U) followed by 400 ms of blank screen and presentation of a second white square (S2) in the lower visual field for 200 ms. I-AMweak did induce a weak or no AM percept. (I-U-S1) control stimulus condition corresponding to the timing and location of the first stimulus in the AM stimulus condition I-AM. Note that this was exactly the same stimulus condition as in the second row, however the timestamps and the baseline were shifted to accommodate the need for a comparison to the stimulus condition I-AMweak. (I-L-S2(AMweak)) control stimulus condition corresponding to the timing and location of the second stimulus in the AM stimulus condition I-AMweak . Stimulus conditions I-AMweak, I-U-S1 and I-L-S2(AMweak) together formed the experimental condition “weak AM.” (b) Stimulus stimuli consisted of white squares of 2° visual angle presented at an eccentricity of 4° visual angle at 25° degrees above the mid line or at 45° below the mid line. (c) Sequence of stimulus conditions in an experimental run. All stimulus conditions were presented in blocks of 6. Interstimulus condition intervals (ITI) within a block and the stimulation free IBIs were chosen at random from one of 4 1000, 1500, 2500, or 3000 ms. These intervals included the baseline of the subsequent stimulus condition. The order of the different stimulus blocks (I-AM, I-AM(weak),…) was also pseudo randomized.
Figure 2. Stimuli used in experiment II to test for a retinotopic modulation of the SSDWs. Each black frame corresponds to the presentation of the corresponding stimulus display for 200 ms. A red border indicates the use of this frame/time interval as a baseline interval in VEP analysis. The time axis is aligned to the onset of second stimulus in the respective AM stimulus conditions (t = 0). (a) From top to (II-AMupper) AM inducing stimulus condition with a motion path that lies predominantly in the upper visual field. (II-U-S1) Control stimulus condition that matches the first stimulus of II-AMupper in timing and location. (II-M-S2) Control stimulus condition that matches the second stimulus in II-AMupper in timing and location. (II-AMlower) AM inducing stimulus condition with a motion path that lies exclusively in the lower visual field. (II-M-S1) Control stimulus condition that matches the first stimulus of II-AMlower in timing and location. (II-L-S2) Control Stimulus condition that matches the second stimulus in II-AMlower in timing and location. (b) Stimulus all stimuli were located on a circle of an eccentricity of 4° visual angle; (II-U) first stimulus in the AM inducing condition II-AMupper, located 45° above the horizontal meridian, (II-M) second stimulus in the AM inducing condition II-AMupper or first stimulus in the AM inducing condition II-AMlower, located 15° below the horizontal meridian. (II-L) Second stimulus in the AM inducing condition II-AMlower, located 75° below the horizontal meridian. Red arrows symbolize the AM paths in the upper and lower condition. The stimulus geometry is chosen such that the representation of the lower motion path between the 2 stimuli (in condition II-AMlower) lies on the upper bank of the calcarine sulcus whereas the representation of the upper motion path between the 2 stimuli (in condition II-AMupper) lies at an opposing position on the lower bank of the calcarine sulcus. (c) Sequence of stimulus conditions in an experimental run. All stimulus condition types were presented in blocks of 6. ITIs within a block were randomized (1800–1920 ms, uniform distribution), each block of 6 was followed by a stimulus free IBI that had a randomized duration (1800–1920 ms, uniform distribution). The order of the different stimulus blocks (I-AM, I-AM(weak),…) was also randomized.
Figure 3. Algorithm used for identification of SSDW components of the VEP. In order to identify those parts of the VEP that differ when stimuli are presented in close temporal proximity (200–400 ms ISI) we first added the VEPs evoked in the control stimulus conditions (here: I-U-S1, red; I-L-S2; magenta) to obtain a composite VEP (here: I-composite(AM), green) that represents the VEP that is expected when no interaction or sequence effects between the stimuli in the AM sequence are present. This composite VEP is then subtracted from the true VEP elicited by the AM stimulus sequence (I-AM, black). The resulting difference wave reflects sequence effects due to either simple nonlinear addition of the underlying neuronal processes (e.g., due to refractive effects) and or processes related to motion and gestalt (motion path) perception. The graph exemplifies this algorithm with grand average VEPs recorded at electrode POz. The actual analysis was performed at the individual subject level for later cluster randomization analysis using within subjects permutation of conditions. The analysis of differences between AM VEPs (“AM”) and the corresponding sum of the single VEPs (“composite”) focused on the interval from 0 to 200 ms, as indicated by the horizontal black bar in the Diff(AM) display. This interval captured all major early peaks in the SSDW. The horizontal dashed lines indicate the minimum to maximum range of fluctuations in the SSDW that occur before the onset of S2. (Insert): Quality of the VEP VEP responses to the first stimulus in the AM condition (I-AM) and to the corresponding control stimulus (I-U-S1) with identical position and timing. VEPs in both conditions show a very high degree of similarity and no significant differences were found, indicating a good overall reliability of the measurements. To help the reader, VEP component C1 has been marked on the single stimulus VEP traces and VEP component N1 has been marked on the trace of stimulus I-L-S2. Note that electrode POz is not optimum for the display of P1 components. Note the expected dependence on retinal stimulus position (I-U-S1 vs. I-U-S2), both at early time intervals (C1, 60- to 90-ms post actual stimulus onset, originating in V1, compare Fig. 4) and later time intervals (P1, N1 originating either in dorsal (I-U-S2) or ventral (I-U-S1) extrastriate areas; also compare Figure 9.
The stimulus conditions for experiment I are displayed in Figure 1. The stimuli formed 2 experimental conditions. The main difference between these experimental conditions was a variation of the strength of the AM percept. This was accomplished by manipulating the interstimulus interval (ISI) in the 2 respective AM stimulus conditions (I-AM in the first experimental condition and I-AMweak in the second experimental condition).
The first experimental condition (strong AM condition) consisted of the AM stimulus condition I-AM and the 2 control stimulus conditions I-U-S1 and I-L-S2: In stimulus condition I-AM 2 stimuli were flashed for 200 ms each, separated by 200 ms of blank screen with the fixation cross (Fig. 1a, top row). The first stimulus appeared at an eccentricity of 4° visual angle and 25° of rotation (in the focal plane) above the horizontal meridian. The second stimulus appeared at the same eccentricity but 45° of rotation below the horizontal meridian (Fig. 1b). These stimulus positions have been proposed in (Di Russo et al. 2002) to stimulate cortical patches at opposing positions across the calcarine sulcus.
The center-to-center distance between the 2 stimuli was 4.6° visual angle. Duncan and colleagues (Duncan and Boynton 2003) estimated the RF size in human V1 to be 0.03 x δ^1.1^ where δ was the eccentricity in degrees of visual angle. Hence, RFs for the eccentricities used here (4°) would be roughly 0.14°. Mikami et al. (1986a, 1986b) found RF sizes of direction selective V1 neurons in the macaque in their study to be roughly 1° at an eccentricity of 4° (Fig. 4). Thus, the separation of the stimuli exceeded the RF size by a factor of at least 4. This stimulus condition robustly elicited the percept of just one, but moving, stimulus. The speed of AM was 11.5°/s when calculated using stimulus onset asynchrony. The timing of this experimental condition corresponded to a frequency of 1.25 Hz of continuous AM stimulation, known to elicit an AM percept (Finlay and von Grünau 1987).
Figure 4. Detection of VEP components generated in V1 (C1 component). (a) Geometry of the control upper row—upper control stimulus (I-U-S1), 25° above the horizontal meridian, bottom row—lower control stimulus (I-L-S1), 45° below the horizontal meridian, both stimuli are presented at 4° eccentricity (conf. Fig. 1). Stimulus (I-L-S1) corresponds to the stimulus condition I-L-S2 as it was introduced in Figure 1. Here, however, the baseline for computation of the VEP was 200 ms immediately preceeding the stimulus to enable a comparison to the literature. (b) Scalp topographies at a latency of 84 ms for both stimulus conditions. Although results from previous studies (Di Russo et al. 2002) predict a full reversal of scalp polarity between the stimuli we only see a partial rotation of the topography, perhaps indicating that these stimuli did not activate exactly opposing patches of cortex in V1 in our sample of subjects. (c) VEP responses at electrode POz to the presentations of control stimuli at 25° above the vertical meridian (blue, I-U-S1) and 45° below the vertical meridian (red, I-L-S1). Note the polarity reversal between responses evoked by the 2 stimuli at this electrode with a peak difference at a latency of 84 ms after stimulus onset. The last common data point of the 2 signals before this peak difference was found at 62 ms.
In the first control stimulus condition I-U-S1 only the first stimulus of condition I-AM was presented (Fig. 1a, second row). In second control stimulus condition I-L-S2 only the second stimulus of condition I-AM was presented (Fig. 1a, third row).
Thus, the expected VEP in stimulus condition I-AM was the sum of the VEPs in stimulus conditions I-U-S1 and I-L-S2 with additional (sequence sensitive) components that arose for one of the following Either the close temporal proximity lead to a simple nonlinear addition of neuronal responses (e.g., due to refractive or adaptive effects) or the additional components arose due to additional perceptual processing that gave rise to the perception of AM. These alternatives were not mutually exclusive.
The second experimental condition (weak AM condition) of the first experiment consisted of stimuli I-AMweak, I-U-S1, and I-L-S2(AMweak): In stimulus condition I-AMweak again a sequence of 2 stimuli was presented. The stimuli were flashed for 200 ms each, but in contrast to condition I-AM the stimuli this time were separated by a blank screen (with fixation cross) lasting 400 ms (Fig. 1a, fourth row). The stimulus locations were identical to stimulus condition I-AM. Stimulus condition I-AMweak elicited only a weak or no percept of AM. The hypothetical speed of AM was 7.7°/s. The timing of this experimental condition corresponded to a frequency of 0.63 Hz of continuous AM stimulation, a frequency that did not elicit a robust AM percept in previous studies (Finlay and von Grünau 1987).
The first control stimulus condition (I-U-S1) for this weak AM stimulus condition I-AMweak was identical to that for the AM stimulus condition I-AM: The timing and location of the stimulus in condition I-U-S1 also matched that of the first stimulus in I-AMweak (Fig. 1a, second and fifth row).
In the second control stimulus condition I-L-S2(AMweak) only the second stimulus of condition I-AMweak was presented. Note that this control stimulus was identical to the corresponding control in experimental condition “AM” (i.e., I-L-S2, cf. Fig. 1a, third row), but the baseline interval was shifted to be compatible with the prolonged ISI of stimulus condition I-AMweak (Fig. 1a, sixth row). Neuronal responses were therefore recorded only once and the baseline shift was performed subsequently during data analysis.
Each stimulus condition was repeated 6 times in a block (Fig. 1c) in order to allow for a comparison with previous fMRI results from our group (Muckli et al. 2005). The intertrial interval (ITI) between repeated conditions was taken at random from one of 4 values (1000, 1500, 2500, or 3000 ms; including the 200-ms prestimulus baseline of the next stimulus). The blocks were separated by interblock intervals (IBIs) chosen at random from the same set of values.
To control for attentional effects we fixed the subjects’ attention at the screen center using a center task. This procedure was similar to the one used in the study of Muckli and colleagues (Muckli et al. 2005). Subjects had to detect an infrequent (2.7% of trials) rapid (200 ms) change (−30%) in the brightness of the fixation cross and indicate it via a button press. These changes were presented at pseudo random time points throughout the stimulus presentation and did neither correlate with stimulus nor with baseline presentation. We used in house software written in DirectX(C) (Microsoft Corporation, Redmond, Washington, USA) for the creation and presentation of the stimuli and for recording of the behavioral responses.
The stimuli used in experiment II are displayed in Figure 2. Again stimulus conditions were grouped into 2 experimental conditions, each consisting of one AM stimulus condition and 2 control stimulus conditions. The timing of the stimuli in the AM stimulus conditions (II-AMupper and II-AMlower) was identical to the timing used in stimulus condition I-AM of experiment I. The difference distinguishing the 2 experimental conditions used in experiment II from each other and from the condition I-AM (experiment I) was a shift of the retinal position of the stimuli (Fig. 2b).
The first experimental condition (condition with upper visual field AM path) of experiment II consisted of stimulus conditions II-AMupper, II-U-S1 and II-M-S2. For the AM stimulus condition (II-AMupper) the first stimulus was presented at 45° above the horizontal meridian and the second stimulus at 15° below the horizontal meridian (Fig. 2a, top row). Both stimuli were presented at an eccentricity of 4° visual angle. In the corresponding first control stimulus condition (II-U-S1) a stimulus was presented with timing and location identical to the first stimulus of the AM stimulus condition, II-AMupper (Fig. 2a, second row). In the corresponding second control stimulus condition (II-M-S2) a stimulus was presented with timing and location identical to the second stimulus of the AM stimulus condition (Fig. 2a, third row). Stimulus separation was 4° of visual angle. The speed of the perceived motion was 10°/s. The timing of this experimental condition corresponded to a frequency of 1.25 Hz of continuous AM stimulation, known to elicit an AM percept (Finlay and von Grünau 1987).
The second experimental condition (condition with lower visual field AM path) of experiment II consisted of the stimulus conditions II-AMlower, II-M-S1 and II-L-S2. In the AM stimulus condition (II-AMlower) the first stimulus was presented at 15° below the horizontal meridian and the second stimulus was presented at 75° below the horizontal meridian (Fig. 2a, fourth row). In the corresponding first control stimulus condition (II-M-S1) a stimulus was presented with timing and location identical to the first stimulus of the AM sequence (II-AMlower, Fig. 2a, fifth row). In the corresponding second control stimulus condition (II-L-S2) a stimulus was presented with timing and location identical to the second stimulus of the AM stimulus condition (Fig. 2a, sixth row). Note that stimulus conditions II-M-S2 of the first experimental condition and II-M-S1 of the second experimental condition were identical except a different choice of baseline. Neuronal responses were therefore recorded only once and the baseline shift was performed subsequently during data analysis.
Each stimulus condition was repeated 6 times in a block (Fig. 2c). The ITI between repeated conditions was randomized in the interval 1800–1920 ms. Blocks were separated by IBIs of randomized duration in the interval 1800–1920 ms. The block order was randomized. To fix the attention of the subjects at the screen center as it had been done in the study by Muckli et al. (2005), subjects had to perform a center task. Subjects had to detect an infrequent (2.7% of trials) rapid (200 ms) change (−30%) in the brightness of the fixation cross and indicate it via a button press. These changes were presented at random time points throughout the stimulus presentation and did neither correlate to stimulus nor to baseline presentation.
The AM stimuli used in the fMRI mapping experiment were identical in position to the stimulus conditions II-AMupper and II-AMlower (confer Fig. 2). However, for fMRI we chose to always present 2 stimulus pairs of the same AM stimulus condition in rapid succession, that is, the 2 identical conditions were separated by the same time interval (200 ms) that was used between stimuli within one condition. This resulted in the percept of 1.5 cycles of down and up AM for a doubled condition; in contrast, subjects could only perceive downward AM in the EEG experiments I and II. We chose this approach to triple the amount of perceived AM episodes. This was necessary to obtain a sufficient signal to noise ratio for fMRI data analysis at the single subject level. The doubled stimulus conditions were presented in blocks of 6. Each block lasted 11 s. Blocks were separated by a fixation baseline of 11 s. We also presented mapping stimuli at each of the locations of the stimuli used in the AM stimulus conditions. These mapping stimuli consisted of flashing white squares with a 200 ms on, 200 ms off cycle. The flashing mapping stimuli were presented in blocks of 11 s. Each mapping block was followed by a fixation baseline of 11 s. The order of the different blocks (AM upper, AM lower, mapping upper position, mapping middle position, mapping lower position) was fully randomized. In total 35 blocks (7 repetitions × 5 stimulus conditions) were presented per fMRI run and 3 runs were performed in each subject. Subjects had to perform the same center task as in EEG experiments I and II.
In both, the EEG experiment II and the fMRI experiment, stimuli were created and presented and responses were recorded using the Presentation software package (version 9.3; Neurobehavioral Systems, Inc., CA).
In the EEG experiments (I, II) subjects were seated in a dark room. Stimuli were presented binocularly on a 19” CRT screen at a viewing distance of 110 cm. In experiment I stimuli were presented in 3 sessions of 12 min each (240 stimulus repetitions per condition in total, including repetitions with the attention task). After each session subjects had a 5-min break with full daylight. Subjects gave their responses in the behavioral task (see stimuli) via an optic response pad with their right hand.
In experiment II stimuli were presented in 8 sessions of 7 min each (325 stimulus repetitions per condition with an additional 9 repetitions including the attention task on average). After each session subjects had a short break of 5 min with the light switched on; after each third session subjects had a longer break with daylight. Subjects gave their responses in the behavioral task (see stimuli) via a computer mouse button, switching between left and right hand from session to session.
In both EEG experiments subjects were not given any instructions with respect to blinking as we did not want any contamination of the trials by preparatory scalp potentials towards the end of the trials. We therefore had to accept a relatively high rejection rate (confer below).
EEG was recorded from 62 electrodes that were placed according to the 10-10 system without F1, F2, F7, F8, FC3, FC4, C1, C2, CP5, and CP6. The electrode positions in the individual subjects were digitized using an ultrasonic digitizer (ELPOS, Zebris Medical GmbH, Isny im Allgäu, Germany) to allow for a correct coregistration of scalp topographies over subjects. All scalp channels were measured against a ground electrode at position FPz and referenced against FCz of the international 10-10 system. To record vertical eye movements and eye blinks an additional electrode was placed below the right eye. All impedances were kept below 5 kOhm. If this value could not be obtained the data were discarded. EEG was recorded at a sampling rate of 2.5 kHz (experiment I) or 1 kHz (experiment II). The data were filtered digitally with a bandpass of 0.5–100 Hz and a notch filter at 50 Hz. The segments that contained the infrequent responses of the behavioral task were discarded. In addition the subsequent segment was also discarded to avoid any contamination of the VEP by delayed motor or somatosensory activity. Artifact rejection was then performed using the artifact scanning tool of the BESA software package (MEGIS Software GmbH, Gräfelfing, Germany) prior to signal averaging. Thresholds used to discard artifactual epochs were either a signal level exceeding 75 μV from the segment baseline or a slew rate exceeding 75 μV/ms. On average 35% of the (non response-contaminated) trials were discarded by this procedure in experiment I and 46% in experiment II. However, due to relatively high numbers of trials all conditions were always left with at least 100 valid trials in all subjects (except 1 subject in experiment II, where only 90 valid trials were obtained). After averaging, data were exported to BESA's standard 81 electrode system. This allowed to account for interindividual differences in head shape and electrode placements when performing group level statistical tests (confer below). The VEPs evoked by the 2 control stimulus conditions of an experimental condition were then summed (names of summed VEPs all contain the part “composite” for easier identification) and subtracted from the corresponding VEP of the respective AM/sequence stimulus condition (Fig. 3). The corresponding difference waves were the parts of the VEP response in the AM stimulus conditions that were sensitive to sequential versus single stimulus presentation (SSDW). These SSDWs should at least partly represent motion perception processes. To test whether differences between the summed control conditions and the AM stimulus condition were statistically significant, we used cluster randomization analysis as described below. To test for sensitivity of the SSDWs for the retinal position of the stimuli we also compared the 2 SSDWs from experiment II with each other forming a “difference-of-differences” wave using cluster randomization analysis.
Table 1 lists the names of stimulus conditions and corresponding VEP responses, the calculated composite VEPs, SSDWs, and differences of difference waves.
All statistical tests were performed using cluster level randomization analysis as implemented in the Fieldtrip Toolbox (www.ru.nl/fcdonders/fieldtrip/). This modified version of randomization testing of 2 trial types for significant differences has been proposed by Maris (2004). Maris’ procedure accounts for the problem of multiple comparisons that arises when comparing VEP data at multiple electrodes and time points. An additional advantage of this approach is that no arbitrary or post hoc definition of time windows for peak detection or average amplitude calculation has to be performed.
To compare the ERP of an AM stimulus condition with the summed ERPs of the control stimulus conditions, data were averaged over trials separately for each subject. Next, a 2-tailed Student's t-test for the difference between the 2 conditions over subjects was computed separately for the amplitudes obtained at each electrode/time point. The analyzed time interval was 0–200 ms after the onset of S2 for experiment I, and, based on these results, 0–180 ms for experiment II. The resulting set of t-values on the electrode/time grid was then thresholded (P < 0.05, 2 tailed). Next, all significant electrode/time data points were identified which were connected with each other such that they formed clusters which satisfied certain minimum requirements. We chose to accept clusters with at least 4 connected electrodes showing an effect at each single time point and which consisted of at least 10 connected electrode/time data points in the cluster. These values reflect that the scalp ERP is inherently smooth in space and time. The sum of t-values over a cluster formed the final test statistic. It was tested against the distribution of the maximum cluster sums of t-values under the null hypothesis. This null distribution was derived by randomly reassigning the responses to stimulus conditions (ERP in the AM stimulus condition or summed of ERPs in the control stimulus conditions) within each subject, subsequently performing the above steps (initial t-test, identification of clusters, summation of t-values) and picking the maximum cluster sum of t-values of this randomization. By repeating this process many times we thus obtained the randomization null distribution for the maximum cluster sums of t-values. By comparing the experimental cluster-sum t-statistic for each cluster to this null distribution, P values for each cluster were obtained (so called Monte Carlo P values). For a large enough number of randomizations Monte Carlo P values accurately estimate the true P values. In our test we used 4000 random draws. As only one test statistic (cluster sum of t-values) is tested, the problem of multiple comparisons is effectively taken care of by this method. The cluster level statistic also accounts for interelectrode dependencies in the signal. For assigning significance we used a threshold of (Monte Carlo) P < 0.05, 2 tailed.
fMRI scanning was performed on a 3T head scanner (MAGNETOM Allegra, Siemens AG, Erlangen, Germany) equipped with the standard head coil (diameter: 26 cm). Functional data were acquired using a single shot echo-planar imaging (EPI) sequence with built in motion correction in k-space. EPI sequence parameters 37 slices, covering the whole brain, oriented parallel to the calcarine sulcus, slice thickness 2 mm, in plane resolution 3.2 × 3.2mm^2^, repetition time TR 2000 ms, echo time TE 30 ms, flip angle 77°. In addition we mapped the point spread function of the sequence before each individual run and used these data to correct distortion induced by magnetic field inhomogeneities (Zeng and Constable 2002). The image reconstruction program performing this task was written by MRDAC, Freiburg, Germany (Zaitsev et al. 2004). The use of distortion correction improves the localization of activated gray matter patches on the reconstructed brain surface by improving the match between the (undistorted) T1-weighted anatomical scans and (distorted) EPI slices. For each subject we acquired a high resolution (1 × 1 × 1 mm^3^) T1 weighted anatomical scan using a magnetization prepared rapid gradient echo sequence. Stimuli were presented via an fMRI compatible video goggle system with 2 LED (light-emitting diode) displays (MR Vision 2000, Resonance Technology, Northridge, CA). Responses were recorded using a fiber optic response pad.
Each run consisted of a randomized presentation of 7 blocks of each of the 5 stimulus types (3 mapping stimuli and 2 AM stimulus conditions, see section on stimuli, 35 blocks in total). Stimulation blocks lasted 11 s and were separated by a fixation baseline of 11 s. The total duration of one run was 764 s, equivalent to the acquisition of 382 functional MRI volumes. The first 4 volumes (8 s) of each run were discarded before analysis to avoid effects of non equilibrium magnetization. Four runs were acquired per subject.
fMRI data analysis was performed with BrainVoyagerQX 1.6.6 (Brain Innovation B.V., Maastricht, the Netherlands, www.brainvoyager.com) using standard slice scan time correction with sinc interpolation, 3D motion correction and high pass filtering at 0.004 Hz. fMRI data were then coregistered to the corresponding T1-weighted anatomical data set. Statistical analyses were performed using a general linear model where stimulus conditions were first dummy coded into boxcar predictors and then convolved with a standard hemodynamic response function (Boynton et al. 1996).
As we wanted to use pre-existing mappings of the visual areas in several subjects that had been obtained by retinotopic mapping (Engel et al. 1994; Goebel et al. 1998; Kriegeskorte and Goebel 2001; Linden et al. 1999; Sereno et al. 1995) in a previous study (Muckli et al. 2005), we analyzed fMRI data at the single subject level. Functional activations were identified by thresholding the activation maps at t > 4.5 for the individual subjects (except for 3 subjects where t-map thresholds had to be lowered to t > 3.8 (2 subjects) and t > 2.5 (1 subject) to obtain above threshold clusters). These relatively low thresholds were necessary due to the rather small size and short duration of the stimuli used. For display of the data we reconstructed the gray/white matter boundary of the cortical sheet of the left hemisphere of each subject. We then projected the significant voxels of the functional volume data onto inflated representations of the reconstructed gray/white matter boundary.
For group level analysis fMRI data were transformed to Talairach space and subjected to a group general linear model. Data were thresholded at P < 0.001 (Bonferroni corrected for multiple comparisons). Siginificant activation clusters were then projected onto the reconstructed boundary surface between gray and white matter of the Talairach transformed anatomical data of one subject (A.K.). Note that an exact assignment of activated clusters to visual areas was not possible for group data due to interindividual differences in the layout of the areas and in cortical folding.
To corroborate our analysis of the generators of the SSDW we used fMRI-constrained source analysis. Details of this method have been described in (Bledowski et al. 2004, 2007). In short we chose fMRI activation clusters from condition conditions II-L-S2 and II-AMlower at the group level as seed points for dipolar sources. We assigned one cluster of condition II-L-S2 to each of the visual areas V1, V2, V3/V3A and assigned 2 clusters found in condition II-AMlower at lateral occipital positions to ipsi and contralateral area hMT/V5+. The assignment of clusters to the early visual areas was based on anatomical proximity. As a precise assignment of fMRI cluster to visual areas at the group level is not possible (see section on fMRI scanning and data analysis) we termed these sources locations V1/V2+, V2/V3+, V3/V3A+ to reflect this uncertainty. Then a dipolar source was placed at each cluster position. We only placed 3 sources in primary visual areas because the placement of a fourth source in primary visual cortex resulted in excessive cross-talk between source waves. For the source analysis we used ERP data from the 10 subjects with fMRI data from the stimulus condition II-AMlower. The seeded dipoles were first oriented based on the group level ERP of condition II-AMlower in the interval 0–200 ms (postonset of S2). Then we projected the SSDW relevant to condition II-AMlower (i.e., Diff(AMlower)) for each individual subject onto the dipole model and performed bootstrap statistics on the resulting source waves to obtain the 95% confidence interval for the SSDW being different from zero at a given time and source.
Detection rates in the center task used for the control of attentional effects of AM when compared with no-motion in EEG experiment I did not differ significantly (n = 10; Wilcoxon paired sample test; P = 0.81). Detection rates were 76% for the weak or no AM stimulus condition (I-AMweak) and 77% for the strong AM stimulus condition. In EEG experiment II we tested the effects of single versus sequential stimulus presentation. Detection rates in the center task used for control of attentional effects in EEG experiment II did not differ significantly (n = 11; Wilcoxon paired sample test; P = 0.07) between control single stimulus conditions (II-U-S1, II-M-S1, II-M-S2, II-L-S2; mean detection rate 84%) and the AM conditions (II-AMupper, II-AMlower; mean detection rate 78%).
Figure 4 displays the VEPs at electrode POz for the first 200 ms after onset of the 2 control stimuli I-U-S1 and I-L-S1 (positions are given schematically in the figure) and the scalp topographies at a latency of 84 ms (with respect to the appearance of the stimuli on the screen). A visual C1 component with an onset around 60 ms (last common point of the 2 data curves at electrode POz) and a peak latency of 84 ms for both stimuli was clearly visible. This component exhibited a polarity reversal at electrode POz between the 2 stimulus conditions as it was described for the identical stimulus geometry (albeit with different stimuli) by Di Russo et al. (2002) . The corresponding scalp topographies at the peak latency showed a clear rotation of a dipolar occipital pattern. However, this rotation did not result in a full polarity reversal on all occipital electrodes as described previously, perhaps indicating an overlap between the C1 component and the onset of the early contralateral P1 components of the VEP at the peak latency.
The VEPs following the onset of the second stimulus of the AM sequence are shown in Figure 5 for the AM stimulus condition (I-AM) and the sum of the control conditions (I-composite AM), obtained in experiment I. In addition, the SSDW between the AM stimulus condition and the sum of control stimulus conditions is displayed (Diff(AM), green)). This difference was computed as sketched in Figure 3. The insert in Figure 3 demonstrates the almost perfect overlap of the VEPs in reaction to the first stimulus of the AM sequence and to its isolated presentation in the control stimulus condition. We did not observe a significant SSDW on any electrode before the onset of S2 (t-tests on single electrodes, an interval of 2 standard deviations over subjects that did not include zero was taken as threshold for significance, data not shown). Hence, only the VEPs for the relevant time interval from 0 to 200 ms following the onset of the second stimulus in the AM sequence are presented further on. The VEPs are displayed as grand averages over all subjects (n = 16). A significant SSDW (cluster randomization statistics, P < 0.05, corrected) was first observed starting around 90-ms post the onset of S2 (Figs 5 and 6). This first component of the SSDW peaked at 103 ms at electrode POz (confer Fig. %, left insert, “d103”). For this interval the VEP evoked by the AM stimulus condition was more positive on occipital electrodes than the summed VEPs evoked by the control stimuli, resulting in a positive SSDW. Following this first peak the SSDW remained significant up to the end of our analysis interval (200 ms post S2). This latter part, however, had a more complex spatial and temporal structure (Fig. 7, top row) with a consistent negative focus over parietal occipital electrodes, indicating that the VEP evoked by the AM stimulus was more negative at these electrodes than the summed VEPs of the control stimuli. This negative parietal focus of the SSDW was modulated in amplitude with time (refer to the insert in Fig. 5 for its time course on electrode POz). It was accompanied by time-varying significant positive foci over left temporal, bilateral central and right temporal sensors (Fig. 7, top row and data of electrode C3 bottom insert).
Figure 5. Grand Average VEPs in experiment I: I-AM condition and sum of VEPs from the corresponding control conditions. VEPs are displayed using BESA's standard 81 electrode system in a pseudo topographical arrangement. The grand average was performed after resampling the interpolated scalp voltage topographies of individual subjects’ 63 electrode recordings to BESA standard 81 electrodes. VEPs were filtered between 0.5 and 100 Hz and are presented after onset of the second stimulus in the AM sequence (t = 0 ms). (I-AM, black)—VEP evoked by the AM stimulus. (I-composite(AM), red)—the composite VEP in the control stimulus conditions. (Diff(AM), green)—the SSDW. (Small insert, upper left) enlargement of these VEPs at electrode POz, which showed the largest peak in the SSDW in the time window from 90 to 110 ms (denoted d103). (Small insert, upper right) enlargement of the VEPs at electrode C3, which showed the highest peak amplitude in the later bilateral positivity of the difference wave (denoted bpos).
Figure 6. VEP topographies in the AM stimulus condition, the composite VEP from the summed control stimulus conditions, the SSDW and the corresponding cluster level statistics over time after the onset of S2 (t = 0 ms). Scalp voltage topographies represent values at the indicated time points. Statistical maps are averaged over an interval of ±12.5 ms around the indicated time points. (Upper Panel) upper scalp topographies of the VEP evoked in the AM stimulus condition (I-AM); middle scalp topographies of the summed VEPs in the control stimulus conditions (I-composite(AM)); bottom scalp topographies of the SSDW. Lower Panel: Scalp topographies of the SSDW masked by membership in a statistically significant electrode/time pair cluster as revealed by cluster level randomization analysis (P < 0.05). The nonsignificant electrode/time pairs are marked in green (n.s.). Significantly different electrode/time pair clusters are shown with their corresponding scalp voltage topography.
Figure 7. Comparison of the SSDWs for the AM and the AMweak experimental condition. (a) Scalp maps of statistically significant electrode/time pair clusters resulting from cluster level randomization analysis (P < 0.05) for the SSDWs for the AM sequence in experiment I (Diff(AM), upper row) and the slowed down stimulus sequence (Diff(AMweak), bottom row) which elicited only a weak or no motion percept. Nonsignificant electrode time pairs are masked in green; statistically significant electrode/time cluster are shown with their corresponding scalp voltage topography. Although a negative going significant cluster appeared for both the weak and the strong AM stimulus condition around 100–110 ms, the positive going earlier (90–120 ms, dashed red line) cluster existed only in the strong AM stimulus condition (I-AM). The interval from 90 to 100 ms where a significant part of the SSDW was only found for the strong AM condition is marked by a solid red line. This early positive going difference of the 2 SSDWs (Diff(AM) - Diff(AMweak)) was a classified as a strong trend (P < 0.06) when using cluster randomization analysis with 4 connected electrodes. This positive going difference of the 2 SSDWs was significant at the single electrode level (t-test, P < 0.05) at P1, Pz, P2, P3, P6, P8, POz, PO4, PO8, PO10, Oz, O2. (b) Difference waves for the 2 conditions Diff(AM) (green) and Diff(AMweak) (blue) at 2 electrode electrode POz where the peak of the early (90–120 ms), motion sensitive part of the difference wave was found in condition Diff(AM) (green) but not in condition Diff(AMweak) (blue); electrode C3 that was close to one of the peaks of a later (150–200 ms) bilateral positive difference wave common to both conditions. The scalp map to the right displays the electrode locations and data averaged over the interval 160–190 ms in condition Diff(AM).
To learn more about the origin and function of the components of the SSDW we next increased the interstimulus interval of the AM sequence from 200 ms (I-AM) to 400 ms (I-AMweak). This went along with a decrease in the strength of the motion percept or a complete breakdown of perceived motion, as reported by the subjects—in line with results from earlier studies (Finlay and von Grünau 1987). Figure 7 displays the statistically significant components of the corresponding SSDWs for the short ISI (200 ms, Diff(AM)) and the long ISI (400 ms, Diff(AMweak)). We found significant early (90–100 ms) components of the SSDW only in the condition with a strong motion percept (Diff(AM), upper row, also refer to Fig. 6) but not in the slowed down sequence (Diff(AMweak)) - this interval is marked with a solid red line in Figure 7. Although a negative going significant cluster appeared for both the slowed down sequence and the strong AM stimulus condition around 100–110 ms, the earlier positive going cluster existed only in the strong AM stimulus condition (Diff(AM)). This difference of the 2 SSDWs (Diff(AM)-Diff(AMweak)) showed a strong trend (P < 0.06, cluster P value) when computed with the parameters given in the methods section. This positive going difference of the 2 SSDWs (Diff(AM)-Diff(AMweak)) was significant at the single electrode level (t-test, P < 0.05) at P1, Pz, P2, P3, P6, P8, POz, PO4, PO8, PO10,Oz,O2 (data not shown). At a stricter single electrode criterion of P < 0.01 the positive difference of the 2 difference waves (Diff(AM) − Diff(AMweak)) was significant at P2, POz, PO4, PO8, O2 (data not shown). It was also significant at the cluster level when choosing to investigate effects at 5 or more connected electrodes instead of 4 (which was, however, not the parameter set chosen for the analysis in this study). The full interval containing the positive cluster ranged from 90 to 120 ms (dashed red line in Fig. 7).
In sum, the presence of this early positive cluster activity in the SSDW qualitatively correlated best with the perceived motion strength.
To identify the origin of the observed SSDW components we next varied the retinal position of the stimuli in the fast stimulus sequence. The stimulus conditions used in experiment II II-AMupper, with an AM path predominantly in the upper visual field and II-AMlower with an AM path exclusively lying in the lower visual field and their matching control stimulus conditions. Components of the SSDW that are generated in retinotopically organized cortices (V1, V2, V3/VP, V4) should be highly susceptible to this manipulation and thus differ for upper and lower visual field stimulus conditions. In contrast, components that arise from cortices without a retinotopic organization that is clearly separable with scalp EEG should remain more or less unaltered. Candidate areas for the generation of components that are not sensitive to retinal stimulus position are those with coarse representations of a full hemifield like area V3a and area hMT/V5+. These latter areas are, however, likely to be generators of the AM specific aspect of the SSDW, due to their known involvement in AM processing (Muckli et al. 2005).
Statistical maps of the SSDWs for the 2 AM stimulus conditions in experiment II (Diff(AMupper), Diff(AMlower)) are presented in Figure 8. In addition, Figure 8 displays the results of a statistical test for the difference in topographies between the 2 SSDWs (Diff**). For the 2 SSDWs Diff(AMupper) and Diff(AMlower) we observed the earliest significant components around 90–100 ms. These onset times of the SSDW were comparable to those observed in experiment I with different subjects and different stimulus positions. We furthermore found sustained significant differences for the whole interval up to 200 ms, a finding also consistent with the results from experiment I.
Figure 8. Scalp topographies of the SSDWs and the effect of a shifted retinal position of the stimuli in the interval 0- to 180-ms post S2. (a) Nonsignificant electrode/time pairs are marked in green, significant electrode/time pair clusters (P < 0.05) are displayed according to their corresponding scalp voltage. From top to Diff(AMupper)—Difference between VEP in the condition II-AMupper and the summed VEP of the corresponding control stimuli (II-U-S1 + II-M-S2). Diff(AMlower)—Difference between VEP in the condition II-AMlower and the summed VEPs of the corresponding control stimuli (II-M-S1 + II-U-S2). Diff**—Difference of the SSDWs: Diff**=(Diff(AMupper) − Diff(AMlower)). Significant electrode/time pair clusters of this difference Diff** indicate that a part of Diff(AMupper) and Diff(AMlower) is susceptible to shifts in the retinal position of the stimuli and that this part of the SSDWs arises from retinotopically organized cortex, presumably with quarter field representations (confer Figs 2, 9). Note that this significant difference, especially its positive peak were well localized in time between 110 and 150 ms indicating a transition of processing through retinotopically organized cortices, that started to cease at the later stages of our analysis interval. The red box indicates the time interval where SSDWs had been susceptible to the manipulation of motion energy in experiment I (conf. Fig. 7). Note that this part of the difference waves was not susceptible to a retinal shift of stimulus position, indicating that the early, motion energy sensitive part of the SSDW arises from a piece of cortex with no or a nondiscernible retinotopic organization (like hMT/V5+). (b) Difference waves for the 2 conditions Diff(AMupper) (blue), Diff(AMlower) (red), the difference of differences Diff** (green) and the cluster corrected t-statistics (black, dashed) at 2 electrode locations over early visual electrode POz and PO7. The time range (80–180 ms) is identical to the one presented in (a). The tvalues (dashed black line) are masked by statistical significance in the same way as the maps in (a). The inserted map shows the sample electrode locations and data from condition Diff** averaged from 130 to 160 ms.
Only the later components of the SSDWs, post 110 ms, showed a susceptibility to the retinal stimulus position (see Fig. 8, Diff**, bottom row). This modulation of the SSDWs by retinal stimulus position was localized on occipital electrodes close to or on the midline. This was compatible with generators of these modulations in early visual areas. In the early time interval (90-100ms), we did not find a significant difference of the difference waves for retinotopically differing stimuli. Note, that the AM specific components of the SSDW had been found in this interval.
An interpretation of the above results in terms of susceptibility to retinal stimulus position however relied on the correctness of our assumptions about the cortical representations of our stimuli in experiment II.
In our fMRI experiment we tried to verify these assumptions. Activations elicited by stimuli used in EEG experiment II strongly differed in visual areas V1,V2v/V2d,V3D,Vp but largely overlapped in area hMT/V5+ (Fig. 9).
Figure 9. fMRI activations evoked by the AM stimuli II-AMupper and II-AMlower. (a) Schematic drawing of an inflated left cortical hemisphere seen from a posterior/medial viewpoint. The extent of retinotopically organized cortex relevant for this study is highlighted in light gray. Representations of the horizontal meridian of the visual field are depicted in blue; representations of the vertical meridian are depicted in green. Activations in response to the inducing single stimuli that were expected based on the known retinotopy of early visual areas and previous studies with similar stimuli (Muckli et al. 2005) are depicted as little colored dots (II-U = II-U-S1; II-M = II-MS1 and II-M-S2; II-L = II-L-S2). Note that the expected centers of gravity are depicted, whereas no indication of the actual size is intended here. (b) Actual fMRI activations evoked by the AM stimulus conditions II-AMupper (yellow) and II-AMlower (red) and their overlap (orange). These AM stimulus conditions consisted of the single inducing stimuli II-U-S1/II-M-S2 (II-AMupper) and II-MS1/II-L-S2 (II-AMlower). Activations in a sample subject are depicted on the inflated left cortical hemisphere of this subject. The visual areas V1/V2/V3 V3a and hMT/V5+ were marked based on results from a previous retinotopic mapping experiment (Muckli et al. 2005). Activations in both conditions strongly overlap in visual area hMT/V5+, whereas the ventral visual areas V1v, V2v, Vp, and V4v are only activated by condition II-AMupper. The expected overlap of the 2 conditions due to the shared single stimulus II-M (being either II-M-S1 or II-M-S2) is found exclusively in the dorsal visual areas V1d, V2d, V3d because this stimulus was presented below the horizontal meridian. The positions of stimuli II-M-S1 and II-M-S2 had been chosen to place them in the fundus of the calcarine sulcus in the average subject (Di Russo et al. 2002).
Group level data (Fig. 10) were consistent with those obtained at the single subject level. An exact assignment of the group level activations to individual early visual areas was not possible because of interindividual differences in the layout of the areas and in cortical folding. Activations in conditions II-AMupper and II-AMlower were largely overlapping in area hMT/V5+, but differed strongly in early visual areas.
Figure 10. Group level fMRI results. Group level fMRI activations in conditions II-AMupper (yellow), II-AMlower (red), and their overlap (orange) projected onto the gray-white matter boundary of the Talairach transformed brain of one subject (A.K.). All activated clusters were thresholded at P < 0.001 (Bonferroni corrected). (a) Mesial view of the left hemisphere. (b) Lateral view of the left hemisphere. (c) Occipital view of the left hemisphere. (d) Inflated gray-white matter boundary, occipital view. Positions of the sulci are indicated in dark gray.
In early visual areas we assigned 3 dipolar sources to fMRI activation clusters for the mapping condition (II-L-S2, corresponding to the second stimulus in the II-AMlower stimulus sequence) significant at the group level (P < 0.001, Bonferroni corrected). As the group level activations in early visual areas formed one large cluster, even at this conservative threshold, we additionally used proximity to anatomical landmarks to assign the sources to visual areas. These sources in early visual areas were termed V1/V2+, V2/V3+, and V3/V3A+ to indicate the uncertainty about the exact contribution of each subject to these clusters at the group level. Inclusion of an additional source (e.g., for V3A alone) would have lead to considerable cross-talk because of the close proximity of the sources. In contrast hMT/V5+ sources could bilaterally be assigned to single clusters of activations (Fig. 11a) obtained from the AM stimulus II-AMlower. Dipolar sources were then oriented according to the group ERP in condition II-AMlower using BESA (Fig. 11b). Bootstrap statistics based on the source waves from individual subjects for each source indicated an onset of significant (95% bootstrap confidence interval not containing zero) SSDW activity at 88 ms in both, contra- and ispilateral area hMT/V5+. This was followed by an onset of SSDW activity of the sources V1/V2+ and V2/V3+ at 100 ms and at 115 ms at the source for V3/V3A+.
Figure 11. fMRI-constrained source analysis. (a) Significant fMRI activation (P < 0.001, Bonferroni correction) in condition II-AMlower used for dipole seeding and seeded dipolar sources. From top to (top) Coronal section through primary visual areas with dipoles V1/V2+ (cyan), V2/V3+ (red), V3/V3A+ (blue) display oriented in radiological convention (“R” indicates right). (middle) Transversal section through both hMT/V5+ hMT/V5+ ipsilateral (magenta) and hMT/V5+ contralateral (green), oriented in radiological convention. The dipole symbol for hMT/V5+ ipsilateral (magenta) covers the small activation cluster for ipsilateral hMT/V5+. (bottom) 3-D view of cross-section through the head oriented in natural coordinates (“R” indicates right). (b) View of dipole positions and orientations in glass head model. All plots are oriented in radiological convention. Dipoles have the same colors as in (a). (c) Dipole source waves obtained by projecting the SSDW for condition II-AMlower onto the dipole model. Colors correspond to the respective dipole sources in (a, b). Dark center line indicates average signal; shaded areas indicate the 95% confidence interval obtained via bootstrap stastistics over 11 subjects. Dashed lines indicate the onset of activation in contralateral hMT/V5+ (at 88 ms), in V1/V2+ (100 ms), and the onset of consistent activity in the source assigned to V3/V3A+ (115 ms). Several sources (hMT/V5+ ipsilateral, V2/V3+, V3/V3A+) exhibit significant but very tiny activations before 60 ms. As stimulus processing in V1 was first observed at the scalp level around 60 ms we attribute these events to the increased noise of the SSDW signal.
We investigated VEPs evoked by sequences of 2 stimuli that elicited a percept of AM and compared them to summed VEPs evoked by isolated stimuli. Geometry and timing of these stimuli were chosen such that no direction selective responses from V1 neurons were expected. Stimulus locked processing started as early as 60 ms after the onset of a stimulus (onset of C1). In this early time interval we did not find any indication of sensitivity to the context of a stimulus in a sequential presentation. Subtracting VEPs in the control stimulus conditions from VEPs in the AM stimulus conditions we found a significant SSDW in the interval from 90 to 200 ms after the second stimulus of the sequence. This SSDW consisted of several components that were differentially modulated by AM content and stimulus geometry. The onset of the SSDW was considerably later than the onset of the VEP component C1 (Fig. 5). Specificity for AM was observed already at this early SSWD onset stage (Fig. 7). In contrast, a susceptibility of the SSDW to retinal stimulus position was found only at later stages beginning around 110 ms, peaking at 120–130 ms after the second stimulus and remaining significant up to 150 ms (Fig. 8). The scalp pattern of modulations of this late part of the SSWD was compatible with generators of these changes in early visual cortices.
Thus, we infer the following sequence of 1) Early cortical processing in area V1 (C1 onset, 60 ms), without an EEG-detectable sensitivity to long-range AM and sequence context for the stimuli used here. 2) Onset of AM processing starting at 90 ms, possibly originating in area hMT/V5+, as indicated by the onset of the SSDW at this latency and its modulation by AM strength. 3) A reactivation of retinotopically organized early visual cortices, possibly including area V1, between 110 and 150 ms. At this latency, processing in early visual cortices was sensitive to the context of the stimuli in their respective sequence, as indicated by a significant SSDW. The generation of this late part of the SSDW in early retinotopically organized visual cortices like V1/V2/V3 was suggested by its sensitivity to retinal stimulus position and the scalp topography of the modulation of the SSWD. Interestingly enough, Vanni et al. (2004) also found second peaks of the contributions from area V1+ (including contributions from V1 and V2 sources) and V3/V3A to the pattern onset VEP in this time interval.
Our above interpretation of the late part of the SSWD is dependent on the assumption that stimulus position dependent changes in SSDW components from areas V1/V2/V3 dominate those from hMT/V5+. Indeed, the early parts of the SSDW, which were susceptible to changes of AM strength and therefore most likely originated in area hMT/V5+ did not show a modulation by changes in retinal stimulus position (Diff**, red frame, Fig. 8). These findings suggest that the changes in retinal stimulus position used in our study did not lead to detectable changes in VEP components originating in area hMT/V5+, despite the fact that area MT has a coarse retinotopic organization that is detectable with fMRI (Huk et al. 2002) but mostly for peripheral stimulation. This notion is compatible with the mapping results from fMRI (Fig. 9). We found a strong overlap of activated patches in area hMT/V5+ for the 2 conditions with differing retinal positions of the AM path.
The late parts of the SSDW were modulated by changes in retinal stimulus position. Moving stimuli from upper to lower visual field, as done in our study, will shift generators between opposing banks of the calcarine sulcus in V1 and from the ventral portions of V2/V3 to the dorsal ones (Di Russo et al. 2002). Using fMRI we found that BOLD activations in the early retinotopic areas (V1, V2, V3/Vp) did not overlap in the ventral portions of these areas and differed strongly in the dorsal portions when AM stimuli with different AM paths were presented (Fig. 9). Hence, shifts in retinal stimulus position were expected to alter most those parts of the VEP that were generated in early visual areas. The trivial part of these VEP changes due to single stimulus position was removed by subtracting the control stimuli. The resulting SSDW, however, remained susceptible to retinal stimulus position (Fig. 8). Hence, we attribute the changes in the later part of the SSDW to generators in early visual areas like V1, V2, and V3/Vp.
The above interpretation would be wrong if all of the retinotopy-specific part of the SSDW was generated exclusively in area V3A. This cannot be fully excluded because area V3A also was differentially activated by the 2 AM stimuli of experiment II (Fig. 9). However, we think that an exclusive generation of the retinotopy-specific part of the SSDW in V3A was unlikely for 2 reasons. First, we would have expected retinotopy-specific contributions from area V3A as a part of an early SSDW ((Vanni et al. 2004) report a first activation of V3/V3A at 75 ms). This early V3A generated SSDW would have been both, motion and retinotopy specific. However, such parts of the SSDW were not found. Second, in the light of existing evidence for AM related feedback from hMT/V5+ to V1 (Sterzer et al. 2006) it seems unlikely that all of the retinotopy-specific part of the SSDW would have been exclusively generated in area V3A. In addition fMRI-constrained source analysis supported an important role of hMT/V5+ in the generation of the early motion sensitive part of the SSDW.
Our interpretation also assigned a functional significance to the observed SSDW. We assumed that the changes in brain signal were functionally related to the perceptual changes (AM perception, perception of the AM path as a gestalt), induced by the changes in the stimulation context (i.e., sequential presentation). However, evoked responses to stimuli presented in rapid succession need not necessarily add linearly. Thus, a SSDW could have been observed because the VEP evoked by the trailing, second stimulus is changed due to nonlinear addition of the VEP of a leading, first stimulus, but this SSDW would bear no functional role. This kind of interference should be highest immediately after presentation of the leading stimulus, decaying with time. Hence the earliest components in response to the trailing stimulus should be affected most. However, the C1 component which started at 60 ms and peaked at 84 ms after this trailing stimulus was unaffected, that is, there was no SSDW in this time interval. The first observable SSDW started around 90 ms post the onset of the trailing stimulus, which was 290 ms (or more) after the offset of the leading stimulus. More importantly yet, the clear modulation of the SSDW by both, changes in AM strength and retinal stimulus position suggested that the SSDW was related to changes in function rather than to a simple nonlinear addition of VEPs.
Timing differences in cortical processing can arise due to the retinal position of stimuli despite identical timing of stimuli (e.g., see Vanni et al. 2004, Fig. 3). Hence, the observed differences when changing the AM path could be due to intracortical timing differences instead of changes in the location of cortical generators. Timing differences without a change in generator geometry would result in a biphasic difference per electrode. This is because in the presence of timing differences the leading response will initially be of larger absolute amplitude on a given electrode whereas the trailing response will be larger later on. In contrast, the observed difference of SSDWs (Diff**, Fig. 8) was monophasic. Therefore, we exclude the possibility that cortical timing differences were the cause of the observed effects.
Our interpretation of these results with respect to the generators of the SSDW were supported by the results of the fMRI seeded source analysis (Fig. 11). The projections of the SSDW in the lower motion path condition onto our fMRI-constrained source model were significantly different from zero first in sources assigned to contralateral area hMT/V5+ at a latency of 88 ms. This onset of the SSWD in contralateral area hMT/V5+ was followed by a significant SSDW in the source assigned to visual areas V1/V2 and V2/V3 at 100 ms. The source assigned to V3/V3A became significant at 115 ms. This supported our interpretation that early motion sensitive part of the SSDW was generated in area hMT/V5+. We chose to only analyze the SSDW for condition II-AMlower as the sources of the ERP in this condition were closest to the EEG sensors. Thus, this condition was expected to yield the highest signal to noise ratio (SNR) which is crucial for a valid source analysis.
A more precise analysis at the level of individual early visual areas without contamination from neighboring ones would be desirable. These analyses, however usually require high SNR at the individual subject level and, therefore, approximately 10 times the number repetitions per condition as it was used here. This is possible for simple stimuli like pattern onsets and flashed stimuli that can be rapidly repeated (Di Russo et al. 2002) but not for complex stimuli that require a significant number of control stimuli and that are extended in time as the stimuli used here. Even in the optimal case a separation of source activities in the early visual areas may not be possible due to fundamental physical limitations (Vanni et al. 2004). We also had to project the SSDW onto a model derived from condition II-AMlower. This is because it is not possible to directly use a difference wave for fitting of dipolar sources. The reason for this is that the fitting procedure is a nonlinear operation and the fit of the difference is not necessarily equal to the difference of 2 fitted models. In addition the uncertainty about the number of sources is exacerbated when one tries to fit a difference wave. As an example one may think of left and right visual field stimulation and the resulting difference wave. The number of sources for the difference wave in this case will be close to but not exactly twice the number of sources in each single condition.
As we chose condition II-AMlower of the fMRI experiment to determine the location of area hMT/V5+ bilaterally and in addition determined the precise location of activations due to the second stimulus in this sequence based on mapping in fMRI condition II-L-S2 we are confident that our fMRI analysis did not miss a source for other reasons than those discussed in (Bledowski et al. 2004, 2006, 2007; Schicke et al. 2006).
It is well possible however, that by assigning single sources to visual areas we actually summed over several individual sources per area that could be assumed to be present due to the presentation of stimuli in different locations. In an AM condition sources for both single stimuli and, potentially, for path-related activity (Muckli et al. 2005) should be considered, yielding up to 3 sources per area and a total model size of approximately 14 sources in occipital cortex. Analysis of such a large number of sources at very small cortical distances is clearly beyond the resolution of fMRI-constrained source analysis. However, although several fundamental uncertainties exist when using source analysis alone to elucidate complex processing in early visual areas, the identical results obtained by fMRI-constrained source analysis and experimental manipulation of the SSDW support the model of AM processing proposed here.
Given that the final percept for our AM stimuli was that of one, but moving object the questions arises to what extent direction selective motion responses from area V1 may have contributed to our results. We think that this contribution to the early (onset at 90 ms) AM selective SSDW is negligible for several
First, Mikami and colleagues (Mikami et al. 1986a, 1986b) tested the dependence of direction selectivity of V1 neurons on their RF sizes in the macaque. For V1 neurons of RF sizes below 1°, as they were stimulated in our study, they found that the maximum speed for the observation of directionally selective responses was well below 10°/s, which was below the apparent speeds used in our study (11.5°/s). However, it is unclear to what extend these values apply to neurons in human V1.
Second, single stimulus dynamics were identical across conditions (200 ms ON flash) and stimulus pairs were presented at locations separated by distances at least 4 times the RF size of the V1 neurons involved. The cortical separation of our stimuli was approximately 13–15 mm using the values for the cortical magnification factor in human V1 provided in (Duncan and Boynton 2003). Using a dominant conduction velocity of 0.1–0.2 mm/ms for horizontal connections (Bringuier et al. 1999; Girard et al. 2001) the time necessary to connect the 2 stimulus locations amounts to approximately 70–150 ms. This time interval is longer than the observed difference (20–30 ms) between first activation in V1 (C1, 60 ms) and the onset of an AM selective SSDW (90 ms). Note that any unspecifically spreading activity on horizontal connections due to the first stimulus (with an onset at –400 ms) would have been removed by the subtraction procedure. Faster stimulus interactions in the far surround (spatial summation field) of the RFs of V1 cells are conceivable. However, in line with our interpretation, these are usually thought to be mediated by feedback from higher tier visual areas (Angelucci et al. 2002).
Third, direction selective lateral interactions in V1 should start concurrently with the arrival of the second stimulus in V1 (latency 60 ms) and then spread. Hence, we would expect to see a retinotopically specific SSDW starting at 60 ms. In contrast, we did not find any retinotopically specific SSDW in the time interval between 60 and 110 ms. In EEG, this failure to observe a retinotopically specific SSDW between 60 and 110 ms may be due to an insufficient signal to noise ratio. However, we were able to observe a retinotopically specific SSDW at longer latencies, post 110 ms. Hence, we think that a low signal to noise ratio is an unlikely explanation for the missing retinotopically specific SSDW between 60 and 110 ms.
We cannot rule out the possibility that, in addition to feedback, lateral interactions in V1 contribute to the SSDW at later intervals. A rough estimate of the possible onset time of these effects would be the sum of the onset latency of the C1 component (60 ms) and the conduction time of horizontal connections for a cortical distance of 13–15 mm (70–150 ms). The resulting estimate of 130–210 ms overlaps partially with the observed retinotopically sensitive part of the SSDW but is decidedly later than its onset (110 ms).
Given that object motion can act as a salient stimulus for bottom up attentional processing care must be taken when comparing static and apparently moving stimuli. As in an earlier study (Muckli et al. 2005) we used a center task to control for attentional effects. The behavioral data obtained for this task did neither reveal a significant difference between the detection rates in strong and weak AM stimulus conditions (EEG experiment I) nor between AM and control stimulus conditions (EEG experiment II). This suggests that the effects we observed were not simply due to shifted attention for the apparently moving stimuli.
We found the onset of AM processing around 90 ms and the appearance of a retinotopy-specific SSDW component around 110 ms. Hence, the onset of hMT/V5+ or V3A processing preceded putative feedback activity in early visual areas (V1/V2/V3,Vp) by roughly 20 ms. This temporal sequence fits well with the one observed by Silvanto and colleagues in a TMS study on the induction of moving phosphenes (Silvanto et al. 2005). Silvanto and colleagues applied subthreshold stimulation to hMT/V5+ between 10 and 50 ms before a suprathreshold stimulation of area V1 to elicit the percept of a moving phosphene. When using a lag between the 2 TMS pulses that was outside this range, subjects perceived only static phosphenes. The timing of putative reactions in V1 we observed was consistent with the timing sequence observed for motion onset stimuli using magnetoencephalography (Prieto et al. 2007).
Using fMRI seeded dipole modeling Di Russo and colleagues (Di Russo et al. 2005, 2007) located the generators of the pattern reversal N75/P85 in visual area V1 and found this area to be reactivated roughly 50 ms later. The retinotopically specific part of the SSDW observed in our study showed an onset latency (approx. 100 ms) that was consistent with this earlier observation. We add to these previous finding by demonstrating that part of this reactivation depended on sequential stimulus context and is most likely related to motion processing in our study. This reactivation was also unlikely to be due to lateral interactions within V1 as stimuli were separated by distances that were by far larger than the RF size of the respective V1 neurons, thus providing a strong support for true feedback activity as the underlying cause of reactivation. In addition, our study demonstrated a reactivation of early visual cortices for pattern onset stimuli as opposed to the pattern reversal stimuli used by Di Russo and colleagues (Di Russo et al. 2005).
Our data fit with the study of Sterzer and colleagues who observed feedback between hMT/V5+ and area V1 by means of DCM analysis of fMRI data from an AM experiment (Sterzer et al. 2006). This correspondence of results between their study and ours is not fully conclusive, however, given that fMRI/DCM has not yet been shown to pick up on timing differences as small as 20 ms. In addition, our study did not provide direct evidence for generators of the retinotopy-specific part of the SSDW in V1. Our results were compatible with generators in all early retinotopic areas (V1/V2/V3,Vp). fMRI-constrained dipole analysis, however, indeed supported the hypothesis that the target area to receive the earliest feedback activity from hMT/V5+ was area V1or V2 (Fig. 11).
Direction selective neurons in area V1 have been hypothesized to play an important role in preprocessing of motion signals before these are fed to area hMT/V5+ (Mikami et al. 1986a, 1986b; Newsome et al. 1986; Pack et al. 2003; Pack et al. 2006). These studies report timing and tuning characteristics for V1 neurons that would enable them to subserve this purpose. Stimuli in our study were chosen in a way that they should not have elicited direction selective responses in V1 neurons directly (Newsome et al. 1986). Hence, we did not expect relevant motion preprocessing in area V1 in this study. Nevertheless, we observed evoked neuronal responses in early visual areas V1/V2/V3 roughly 20 ms after the onset of motion processing. These responses were selective for the sequential context of the presented stimuli. Thus, our data suggest a stimulus-context dependent reactivation of areas V1/V2/V3 approximately 20 ms after the onset of motion processing in hMT/V5+ or V3a. We add to previous findings by demonstrating this effect without relying on subjects’ responses as in TMS studies where cortical processing is disrupted (Sack et al. 2006) and without injecting artificial neural activity into the system as in TMS studies that used TMS pulses to generate motion percepts (Silvanto et al. 2005). The overlap of our results with those from “percept-generating” TMS studies may serve as an indication that timing of cortical processing may not be strongly altered for neural activity injected by TMS. We add to previous reports of feedback activity from hMT/V5+ to V1 by locating these feedback processes in time.
It is unclear at present what purpose these putative feedback responses may serve. One possibility is that results of higher order specialized areas are fed back to V1 to be integrated and then passed on to consciousness (Bullier 2001). Another possibility is that a set of areas that perform overlapping computations, for example, the set of areas that contain direction selective neurons, V1, hMT/V5+, V3a mutually update each other to give rise to consistent information across the network even in cases where only one of these areas can fully perform the desired computation for a particular stimulus (like area hMT/V5+ or V3A for long range AM stimuli). This would help to guide further perceptual processes by establishing a consistent expectation across the network.
Supplementary material can be found http://www.cercor.oxfordjournals.org/.
Federal Ministry of Education and Research (BMBF/DLR 01GO0203) and the Max Planck Society supported Brain Imaging Center Frankfurt. Funding to pay the Open Access charges for this article was provided by the Max Planck Society.
We thank Cerisa Stawowsky for preparing Figure 9 and Peter Wibral for proofreading part of the manuscript. We are grateful to Markus Staudinger, Tam Ho and Carolyn van Veen for their help with the EEG measurements. We thank the members of the Laboratory for Clinical Neurophysiology and Neuroimaging at the Department of Psychiatry of the University Clinic, Frankfurt and Dr Corinna Haenschel for their kind support in performing these experiments. Conflict of Interest: None declared.