Authors: Kayla R. Donaldson (1Minneapolis VA Health Care System; 2University of Minnesota, Department of Psychiatry and Behavioral Sciences), Julia E. Hanson (2University of Minnesota, Department of Psychiatry and Behavioral Sciences), Victor Pokorny (3Northwestern University, Department of Psychology), Chen Shen (4Mayo Clinic College of Medicine and Science, Arizona Campus), Samuel Klein (5University of California Los Angeles Department of Psychiatry and Biobehavioral Sciences; 6Semel Institute for Neuroscience and Human Behavior), Holly K. Hamilton (1Minneapolis VA Health Care System; 2University of Minnesota, Department of Psychiatry and Behavioral Sciences), Scott R. Sponheim (1Minneapolis VA Health Care System; 2University of Minnesota, Department of Psychiatry and Behavioral Sciences; 7University of Minnesota, Department of Psychology)
Categories: Article, Schizophrenia, Bipolar Disorder, EEG, Perception, Genetic Risk
Source: Journal of psychopathology and clinical science
Doi: 10.1037/abn0001128
Authors: Kayla R. Donaldson, Julia E. Hanson, Victor Pokorny, Chen Shen, Samuel Klein, Holly K. Hamilton, Scott R. Sponheim
Though schizophrenia is associated with disturbances in processing auditory and visual stimuli, it is unclear whether these anomalies reflect a shared abnormality in neural functions or modality-specific sensory impairments. It is also unclear to what degree these anomalies are associated with psychotic symptomatology or genetic risk for schizophrenia. We examined electrophysiological responses during auditory and visual target-detection tasks in probands with schizophrenia (SCZ, N=107) and bipolar disorder (BP, N=62), first-degree biological relatives of SCZ (SREL, N=62), and controls (CON, N=85). Principal components analyses of event-related potentials (ERPs) were used to evaluate (1) the covariation of ERPs across sensory modalities, (2) relationships with clinical and cognitive symptoms, and (3) associations with polygenic risk for schizophrenia. The covariance structure amongst ERPs for auditory and visual stimuli in CON revealed early, mid-latency, and late components that were associated across sensory modalities. In SCZ this covariation was disrupted with only mid-latency components consistently covarying across modalities. SCZ also showed reduced ERP amplitudes across stages of visual processing and in late auditory processing. Reduced early and mid-latency visual ERP components were associated with psychotic symptoms and polygenic risk, while reduced late auditory components were associated with lower cognitive performance. Neural responses reflecting separable stages of processing sensory stimuli covary across visual and auditory modalities. In schizophrenia, early sensory and late neural responses are independently disrupted within auditory and visual modalities, while reductions in mid-latency neural responses are shared across modalities. Targeting mid-latency neural responses may be advantageous in generalizing beneficial treatment effects across sensory modalities.
Schizophrenia and other psychotic disorders are characterized by anomalous perceptual experiences that can occur in any sensory modality. It is also well established that schizophrenia is associated with compromised sensory processing of auditory (Dondé et al., 2023) and visual (Adámek, Langová, & Horáček, 2022) stimuli. During acute psychotic episodes perceptual disturbances are often severe, but they are also evident during remission (Chieffi, 2019; Cutting & Dunne, 1986), before disorder onset during prodromal and clinical high risk periods (Hamilton et al., 2022; Kalkstein, Hurford, & Gur, 2010; Oribe et al., 2013), and in unaffected individuals who carry genetic liability for psychosis (Donaldson et al., 2021; Hébert et al., 2010; Loughland, Williams, & Harris, 2004). These perceptual disruptions are observable in performance on experimental laboratory tasks (Butler, Silverstein, & Dakin, 2008; Dondé et al., 2023; Javitt & Sweet, 2015) as well as in neural markers of sensory processing manifest in electroencephalography (EEG) and event-related potentials (ERPs; Hamilton et al., 2018; Javitt & Sweet, 2015; Näätänen & Kähkönen, 2009; Urban, Kremláček, Masopust, & Libiger, 2008; Wynn, Sugar, Horan, Kern, & Green, 2010; Yeap et al., 2008). Yet, despite many studies examining perceptual processes in schizophrenia, the relationships amongst neural responses across sensory domains of perceptual processing are relatively unexplored. For example, it is unclear whether electrophysiological anomalies elicited by stimuli in the auditory modalities are reflective of common processes shared with the visual modality. Understanding how early (less than 200ms), middle latency (“mid-latency”, 200 to 350ms), and late (after 350ms) neural responses covary across sensory modalities may shed light on the brain abnormalities underlying aberrant perception and help isolate neural mechanisms related to the pathophysiology of schizophrenia. If neural responses reflecting early perceptual processing in sensory cortices are independent and anomalous, this would point to the need to target separate sensory cortical functions, while shared anomalies across sensory modalities would support using a single intervention. Here we report on a conjoint analysis of ERP responses during auditory and visual perception in people with schizophrenia to identify promising targets for intervention that may generalize across sensory modalities and stages of processing.
Evidence for impairments in early visual processing associated with schizophrenia come from studies of contrast sensitivity, gain control and surround suppression, motion processing, visual integration, grouping and perceptual closure (for reviews, see Adámek, Langová, & Horáček, 2022; Butler, Silverstein, & Dakin, 2008). These impairments are related to amplitude reductions in early EEG responses such as the N1 and P1 reflecting low-level processing of physical properties of stimuli. Mid-latency responses such as the N2 and P2 are also elicited in these visual experiments and reflect evaluative aspects of stimulus meaning such as novelty. Evidence for early and mid-latency auditory dysfunction in schizophrenia includes impairments in tone-matching thresholds, encoding of sensory information, distinguishing between sounds by stimulus features, spatial localization of auditory information, and novelty detection (for reviews, see Fivel, Mondino, Brunelin, & Haesebaert, 2023; Javitt, 2009; Javitt & Sweet, 2015). Like the visual domain, such impairments are associated with reductions in the auditory N1, P1, N2, P2, and mismatch negativity (MMN). While several studies have localized early sensory components to auditory and visual cortices depending on stimulus modality (Di Russo, Martínez, Sereno, Pitzalis, & Hillyard, 2002; Foxe & Simpson, 2002; Godey, Schwartz, De Graaf, Chauvel, & Liegeois-Chauvel, 2001; Javitt, 2000, 2009; Näätänen & Picton, 1987), generators have additionally been localized to parietal and frontal regions (Di Russo et al., 2002; Foxe & Simpson, 2002; Godey et al., 2001). Localization of these neurophysiological signatures to regions beyond sensory cortices, and the influence of cognitive processes such as attention and memory (for a review, see Ruff, 2013), supports the notion that high level (e.g., top-down) mechanisms affect lower level functions during perception and reflect processes shared across sensory modalities.
There is ample evidence for schizophrenia also being associated with impairments in the later stages of perceptual processing for both auditory and visual stimuli which involve cognitive functions beyond simple attention. These deficits include working memory (Fuller, Luck, Braun, Robinson, McMahon, & Gold, 2009; Gold, Carpenter, Randolph, Goldberg, & Weinberger, 1997), scene analysis and pattern recognition (Bestelmeyer, Tatler, Phillips, Fraser, Benson, & Clair, 2006; Javitt, Shelley, Silipo, & Lieberman, 2000; Matthews, Todd, Mannion, Finnigan, Catts, & Michie, 2013), object and speech recognition (Doniger, Silipo, Rabinowicz, Snodgrass, & Javitt, 2001; Wu, Wang, & Li, 2018), and emotion perception (Hoekert, Kahn, Pijnenborg, & Aleman, 2007). The P300, indexing attention and working memory, and the LPP, indexing arousal and engagement with stimuli, are both reduced in schizophrenia (Ergen, Marbach, Brand, Başar-Eroğlu, & Demiralp, 2008; Ford, 1999; Kotov et al., 2024; Strauss et al., 2015; Vianin et al., 2002). The P3 and LPP have been localized to brain areas throughout frontal, temporal, and parietal cortices depending on task demands and stimuli (Bledowski et al., 2004; Masuda, Sumi, Takahashi, Kadotani, Yamada, & Matsuo, 2018; Scharmüller, Leutgeb, Schäfer, Köchel, & Schienle, 2011; Tarkka, Stokić, Basile, & Papanicolaou, 1995). Thus, while it is apparent that schizophrenia is related to impaired neurophysiological responding from sensory registration to cognitive functions during auditory and visual perception, whether such aberrations reflect a common neural circuit dysfunction or unique neural origins is unknown.
Researchers have demonstrated that early and mid-latency sensory responses as well as later-stage components are associated with reports of real-world perceptual disturbances in schizophrenia, including both frank hallucinations and more subtle anomalous perceptual experiences (Debruille, Schneider-Schmid, Dann, King, Laporta, & Bicu, 2005; Donaldson et al., 2020; Ford et al., 2012; Kayser et al., 2012; Spironelli, Romeo, Maffei, & Angrilli, 2019). Although abnormal functioning of modality-specific sensory and heteromodal cortices have been associated with auditory and visual hallucinatory experiences, recent reviews and meta-analyses have pointed to the need for a more unified understanding of the full range of hallucinatory experiences given inconsistency across studies (Allen, Larøi, McGuire, & Aleman, 2008; Erickson, Bansal, Li, Waltz, Corlett, & Gold, 2025; Sterzer et al., 2018; Thakkar, Mathalon, & Ford, 2021; Zmigrod, Garrison, Carr, & Simons, 2016). An important consideration is that such experiences often occur in individuals with various forms of psychopathology outside the schizophrenia spectrum (Cardella & Gangemi, 2019). Individuals with bipolar disorder often report perceptual symptoms (Toh, Thomas, & Rossell, 2015) and are sometimes (Donaldson et al., 2020; Maekawa et al., 2013) shown to exhibit reductions in neural sensory responses similar to those reported in schizophrenia. As such, it is important to determine whether the relationships between auditory and visual perceptual processing in bipolar disorder are similar to those seen in schizophrenia and possibly suggestive of an overlapping neuropathology for the two disorders.
Individuals who carry genetic liability for schizophrenia also experience unusually frequent perceptual disturbances. First-degree biological relatives of individuals with a schizophrenia-spectrum disorder are approximately eight times more likely to develop a psychotic disorder (Lo, Kaur, Meiser, & Green, 2020) and experience perceptual symptoms at considerably higher rates than the general population (Martín-Reyes, Quiñones, de Villalvilla, & Sosa, 2010). Polygenic risk scores for schizophrenia have been shown to predict anomalous perception (Alloza et al., 2020) and could possibly hold potential as a tool for forecasting the course of a psychotic disorder (Jonas et al., 2019). Furthermore, auditory and visual processing deficits seen in first-degree relatives (Martín-Reyes et al., 2010; Yeap et al., 2006) are associated with attenuated neural responses during perception (Earls, Curran, & Mittal, 2016; Winterer et al., 2003; Yeap et al., 2006). Understanding how anomalies in perceptual processing mark genetic risk and whether select anomalies are tied to the actual experience of psychotic symptoms will allow better differentiation of the neural expression of genetic liability from the actual development of psychotic psychopathology.
In the present study we sought to first clarify whether neural abnormalities in people with schizophrenia that are evident during auditory and visual perception are shared or unique to a sensory modality. By testing whether these mechanisms are shared or specific, we may better understand effective means of intervening to address perceptual disturbances. Second, we aimed to evaluate whether the relationships between neural abnormalities during auditory and visual perception observed in schizophrenia extend to individuals with bipolar disorder. Similarities and differences in neural responding across the two disorders will point to which perceptual anomalies are most distinguishable between primary mood and psychotic psychopathology and have implications for the diagnostic specificity of intervention. Finally, we examined relationships between neural abnormalities in auditory and visual perception and genetic liability for schizophrenia in order to investigate the neural manifestation of genetic risk and how neural responses during perception might be affected by the development of psychotic psychopathology. In order to accomplish these aims, we recruited probands with schizophrenia-spectrum and bipolar disorders, their first-degree biological relatives, and healthy comparison subjects. Participants completed target detection tasks in both the auditory and visual modality while EEG data were collected. We also collected measures of perceptual disturbances and other psychosis-spectrum symptoms, as well as biospecimens from which we calculated polygenic risk scores (PRS) for schizophrenia to afford precision in characterizing genetic liability. We compared covariation of neural measures across sensory modalities and stages of processing across groups and investigated how neural responses during perception were associated with clinical symptoms, genetic risk, and behavioral performance. We hypothesized that healthy control participants would have early sensory components that covaried less across sensory modalities than components occurring later during perceptual processing. We expected the pattern of covariation amongst auditory and visual neural responses to be different between people with schizophrenia and healthy controls.
Participants were recruited as part of three separate family studies at the Minneapolis VA Medical Center. Participants were identified through records at the medical center and fliers posted at local mental health centers and at locations in the community (e.g., public libraries). Enrolled participants included probands (PRO) with schizophrenia-spectrum (SCZ; N=107), bipolar disorder (BP; N=62), or other psychosis (OP; N=4), their first-degree biological relatives (REL; N=85, including 6 relatives of probands with bipolar disorder, 62 relatives of probands with schizophrenia, and 17 relatives of probands with schizoaffective disorder), and healthy comparison subjects (CON; N=86). As the goal was to characterize ERPs in a manner that generalized across individuals, all available data were included in first-level principal component analyses models (described in principal component analyses section below). Individuals with OP, relatives of BP, and relatives of probands with schizoaffective disorder were excluded from subsequent statistical analyses due to small sample sizes in these groups, leaving the final sample used in statistical analyses to be 107 SCZ, 62 BP, 62 SREL (exclusively relatives of individuals with schizophrenia), and 85 CON.
All probands were stable outpatients at the time of participation. Exclusion criteria for PRO and CON included intellectual disability as indicated by an IQ<70, recent (last 6 months) drug or alcohol dependence, history of central nervous system conditions, epilepsy, electroconvulsive therapy, head injury with skull fracture or loss of consciousness greater than 30 minutes, or age below 18 and above 60 years. CON were also excluded if they had a history of psychotic illness, depressive or manic episodes, or a family history of depression, mania, or psychosis. Due to the unique nature of the sample, SREL were excluded only for the presence of a medical condition which rendered safe completion of the study impossible.
This study was approved by the Minneapolis VA Institutional Review Board (VA IRB protocols 2511-A, 3187-B, and 4549-B, initially dated 01/26/1998, 09/16/2002, and 10/28/2014) and complied with ethical standards of relevant national and institutional committees on human subjects research. Informed consent was obtained from all participants. Previous publications have reported on a subset of this sample’s performance and electrophysiological responses on the auditory oddball task (Pokorny & Sponheim, 2021), described below. However, the present analyses differ in that (1) we include additional subjects, (2) we explore relationships between EEG measures and polygenic risk for schizophrenia, and (3) the focus of the present study is on drawing comparisons between auditory and visual processing, rather than exploring task effects in each modality.
Clinical symptoms were assessed via the Brief Psychiatric Rating Scale (BPRS; Ventura, Nuechterlein, Subotnik, Gutkind, & Gilbert, 2000), Schizotypal Personality Questionnaire (SPQ; Raine, 1991), and Scales for the Assessment of Positive and Negative Symptoms (SANS, SAPS; Andreasen, 1981, 1984). Diagnoses were obtained using the Structured Clinical Interview for DSM-IV (First, Spitzer, Gibbon, & Williams, 2001) followed by a consensus review of all available study and medical record information by two trained diagnosticians who were advanced doctoral students in clinical psychology, postdoctoral researchers, or licensed doctoral-level psychologists. IQ was estimated using the Wechsler Adult Intelligence Scale III Vocabulary and Block Design subscales (Silva, 2008). In addition to assessments of cognitive functioning and current and past psychopathology, each participant completed the auditory and visual perception tasks while EEG was recorded. These tasks are described below.
Visual perception was assessed using a sustained-attention target detection task, specifically the degraded stimulus continuous performance task (DSCPT), administered via Neurobehavioral Systems Presentation software. This task has been fully described previously by our group and others (Klein, Shekels, McGuire, & Sponheim, 2020; Nuechterlein & Asarnow, 1999). In brief, task stimuli consisted of numerals between “0” and “9”, presented on the screen at 4.3 × 3.4 degree visual angle in size for 29 ms followed by a 971ms white display. Task stimuli and backgrounds were degraded in image clarity, with 40% of white numeral pixels switched to black and 40% of black background pixels switched to white. Participants were instructed to respond via button-press whenever they saw a target stimulus appear on the screen. The target stimulus (“0”) was presented on 25% of trials, while non-target stimuli (“1” through “9”) were presented on the remaining 75% of trials. Participants completed two control conditions in which the same stimuli were viewed while participants were instructed to “just look” or “press every” stimulus, followed by practice trials, and finally three task blocks of 160 trials each. Refer to Figure 1A for a depiction of stimuli and trial structure. Of participants included in the present sample, all had usable visual task data. Behavioral data from the DSCPT was evaluated using the signal detection measure of sensitivity d’, calculated as z-scored hit rate minus z-scored false alarm rate.
A directed-attention auditory oddball task was administered using Neurobehavioral Systems Presentation software. This task has been described in full previously (Pokorny & Sponheim, 2021). In brief, participants completed four blocks of 200 trials each. During each trial, a tone was presented at 96 dB over a 55 dB white noise background. Four possible tones were presented, each differing in pitch, in a pseudorandom order alternating between ears. The tone had a duration of 100ms, 10ms rise/fall, and intertrial interval between 1200–1500ms. In each block, participants were instructed to click the mouse button with their right finger when they heard the target. During blocks 1 and 2, participants heard a 2400 Hz pitched tone on 20% of trials in their left ear, a 1600 Hz tone on 80% of trials in their left ear, a 1200 Hz tone on 20% of trials in their right ear, and an 800 Hz tone on 80% of trials in their right ear. During block 1, infrequent tones in the left ear were the targets (i.e., 2400 Hz). During block 2, infrequent tones in the right ear were the targets (i.e., 1200 Hz). During blocks 3 and 4, the instructions remained the same but headphones were reversed on the participants’ heads so that specific tones were presented to the opposite ear. Refer to Figure 1B for a depiction of task structure. Due to the timing of task administration during study visits, more participants completed the visual than auditory task. Of participants included in the present sample the following subset had usable auditory task 65 SCZ, 46 BP, 57 SREL, and 65 CON, following exclusions for EEG data quality such as the presence of large artifacts or significantly noisy data. Behavioral performance during the dichotic listening task was evaluated using the signal detection measure of sensitivity d’, calculated as z-scored hit rate minus z-scored false alarm rate. Behavioral performance on auditory and visual tasks is included in Supplemental Information.
In each study, scalp EEG was recorded during both auditory and visual task completion. For two studies, scalp EEG was recorded using BioSemi ActiveTwo systems with a differential amplifier and a high-density electrode cap. For earlier subjects, a 64-electrode cap was used; for later subjects, an upgraded 128-electrode cap was used. For one study, EEG was recorded using a Brain Vision actiCHamp EEG system. Both BioSemi and BrainVision montages were radially organized centered on Cz, though electrode placement is not identical across the systems. Prior work with these data compared component amplitudes measured by 64 and 128 channel arrays and found no difference in data collected between systems and a high degree of convergence between studies (Pokorny & Sponheim, 2021). Furthermore, 128 channel BioSemi and BrainVision data were interpolated to a common 64-electrode array in the present analyses for consistency using a spherical spline approach (Perrin, Pernier, Bertrand, & Echallier, 1989). To further promote consistency between montages, PCA component amplitudes (described in Principal Components Analyses section below) were utilized in analyses rather than native format ERP amplitude scores, and the study from which data were drawn was controlled for statistically (described in Statistical Analyses section below). During acquisition, all data collected on BioSemi systems were sampled at 1024 Hz and referenced to ear electrodes, while all data collected on BrainVision systems were sampled at 1000 Hz and referenced to Cz. Common mode sense (CMS) and driven right leg (DRL) ground electrodes were used for BioSemi system recordings, while Fpz was used as the ground electrode in BrainVision recordings.
Offline analysis and processing of EEG data was completed using MATLAB (Mathworks). Matlab’s resample function was used to high-pass filter data at 0.5 Hz and resample to 256 Hz with implementation of anti-aliasing lowpass filters that prevent frequencies above the Nyquist frequency from aliasing during downsampling. Noise and other artifacts were removed using a custom ICA algorithm (Kang, Sponheim, & Lim, 2022). Cleaned data were re-referenced to averaged electrodes. Subject-level average ERPs were computed for use in PCA models described in section 2.6 below. For auditory task data, a −150 to 0 ms baseline was used and averages were created for conditions varying in attention (left or right ear), odd ball (rare or frequent), and pitch (high frequency or low frequency tone). Across subjects, cleaned EEG data contained an average of 61 rare target trials, 264 unattended frequent nontargets, 267 attended frequent nontargets, and 66 unattended rare nontarget trials. Healthy control participants had an average of 61 rare target trials, 263 unattended frequent non-target trials, 267 attended frequent non-target trials, and 66 unattended rare nontarget trials. PSZ had an average of 60 rare target trials, 261 unattended frequent non-target trials, 265 attended frequent non-target trials, and 66 unattended rare nontarget trials. Participants with bipolar disorder had an average of 60 rare target trials, 267 unattended frequent non-target trials, 270 attended frequent non-target trials, and 67 unattended rare nontarget trials. Siblings of PSZ had an average of 63 rare target trials, 267 unattended frequent non-target trials, 268 attended frequent non-target trials, and 67 unattended rare nontarget trials.
For visual task data, a −100 to 0 ms baseline was used and averages were created for conditions varying in the numeral’s similarity to the target (‘very dissimilar’ 1, 4, 7; ‘dissimilar’ 2, 3, and 5; ‘similar’ 6, 8, or 9; ‘non-targets’; and ‘targets’). Across subjects, cleaned EEG data contained an average of 59 target trials, 76 very dissimilar nontarget trials, 75 dissimilar nontarget trials, and 63 similar nontarget trials. Healthy control participants had an average of 57 target trials, 75 very dissimilar nontarget trials, 74 dissimilar nontarget trials, and 63 similar nontarget trials. PSZ had an average of 58 target trials, 73 very dissimilar nontarget trials, 72 dissimilar nontarget trials, and 61 similar nontarget trials. Participants with bipolar disorder had an average of 62 target trials, 80 very dissimilar nontarget trials, 76 dissimilar nontarget trials, and 65 similar nontarget trials. Siblings of PSZ had an average of 63 target trials, 80 very dissimilar nontarget trials, 80 dissimilar nontarget trials, and 66 similar nontarget trials. In both auditory and visual tasks, only correct trials were included in scored average waveforms. Condition averages for each participant were used in PCA models.
Event-related potentials were also scored in order to verify the accuracy of our interpretation of PCA components. In auditory task data, the N1 was scored as the 40 ms adaptive area under the peak between 75 and 150 ms at Cz; the P2/MMN was scored as the 31.25 ms adaptive area under the peak between 150–225 ms at FCz; the P300 was scored as the mean amplitude from 300–500 ms at Pz, and the LPP was scored as the mean amplitude from 600–800 ms at Pz. In visual task data, the N1 was scored as the 40ms adaptive area under the peak between 100–150ms at Oz; the P1 was scored as the 31.25 ms adaptive area under the peak between 75–110 ms at PO7/PO8; the N2 was scored as the 40ms adaptive area under the peak between 270–350 ms at Pz; the P2 was scored as the mean amplitude from 150–225 ms at Oz; and the P3/LPP was scored as the 70ms adaptive area under the peak between 350–625 ms at Pz.
A central goal of this study is to establish the similarities and differences between neural measures of auditory and visual perception. As such, participant average ERPs for each task condition were subjected to a covariance matrix based temporal promax-rotated PCA using the EPtoolkit with kappa set to 3. The number of components to retain was determined based on visual inspection of both (1) the eigenvalue scree plot and (2) the interpretability of resulting factor waveforms and topographies in relation to established ERP components. Separate PCAs were run for each study and for each task, resulting in a total of six first-level PCA datasets across the three studies with data on auditory and visual target detection tasks. The same resulting components emerged for each study’s auditory and visual task data, including auditory N1, P2/MMN, P300, and LPP components and visual P1, N1/P2, N2, and P300/LPP components. The best-fitting PCA for the visual task also included a fifth factor, which accounted for minimal residual variance and could not be identified as representing any known ERP component, which was therefore not exported for analyses. As a confirmatory analysis, we calculated these same first-level PCAs for probands and healthy controls separately, and found that they yielded the same identifiable components, confirming that data from one PCA calculated across groups was appropriate to use for further analyses. We then derived first-level PCA component variables by calculating the mean of temporal factor waveforms from 0–900 ms at the following auditory P3 & LPP at Pz, auditory N1 at Fz, auditory MMN/P2 at FCz, visual N1/P2 & N2 at Oz, visual P3/LPP at Pz, and visual P1 and PO7/PO8. PCA component topographies and waveforms are displayed in Figure 2. Scatterplots of associations between PCA components and relevant ERPs are included in Supplemental Figure 1. In all instances, PCA components appropriately captured variance associated with the scored ERPs (all PCA – ERP correlations p<.001) and were thus deemed acceptable representations of the ERPs for use in statistical analyses.
Biospecimen samples (saliva, whole blood, or fast technology for analysis (FTA) cards) were collected from participants and sent to the University of Minnesota Genomics Center (UMGC) for DNA extraction and genotyping using the Illumina Infinium Human PsychChip v1.1 (Illumina Inc., San Diego, CA). Imputation of the genotyped data was performed with the TOPMed Imputation Server (Das et al., 2016; Fuchsberger, Abecasis, & Hinds, 2014; Taliun et al., 2021). Quality control steps were performed based on established guidelines (Choi, Mak, & O’Reilly, 2020). Single Nucleotide Polymorphisms (SNPs) were removed if their minor allele frequencies < 0.05, Hardy-Weinberg Equilibrium p-values < 1e^−6^, or if they had a high fraction of missingness across subjects (geno > 0.01). Individuals with a high rate of genotype missingness (mind > 0.01) were also removed. The PRSice software package, which calculates polygenic risk scores (PRS) at any number of p-value thresholds, was used to identify PRS with the most predictive threshold based on R^2^ (Choi & O’Reilly, 2019). The Psychiatric Genomics Consortium Wave 3 Schizophrenia genome-wide association study (GWAS) was selected as the base data for this PRS calculation (Sigurðsson & Consortium, 2022).
All statistical analyses and data visualization were carried out using RStudio (Team, 2022) with support from the following the tidyverse (Wickham et al., 2019), scatterplot3d (Ligges & Mächler, 2003), corrplot (Wei & Simko, 2017), nlme (Pinheiro, 2011), FactoMineR (Lê, Josse, & Husson, 2008), reshape2 (Wickham, 2007), psych (Revelle, 2020), plotrix (Lemon, 2006), ggpubr (Kassambara, 2018), GGally (Schloerke et al., 2021), ggrepel (Slowikowski et al., 2018), lmerTest (Kuznetsova, Brockhoff, & Christensen, 2017), car (Fox & Weisberg, 2018), and data.table (Dowle, Srinivasan, Gorecki, Chirico, Stetsenko, & Short, 2019). R syntax needed to reproduce analyses presented herein is included as supplementary information, as it the associated output with statistical values. As a re-analysis of archival data, this study was not preregistered. Associations between first-level PCA components across modalities were examined using bivariate Pearson correlations between average neural responses to all task stimuli and were not separated by task condition. Group differences in the covariance amongst these first-level PCA components were evaluated by comparing the correlation matrices of first-level PCA components across groups using the Jenrich test (Jennrich, 1970). Based on differences between groups, these auditory and visual components were then subjected to groupwise second-level three-factor (based on Eigenvalue >1 and interpretability of factors in the CON group) varimax-rotated PCAs to establish shared vs unique loadings across sensory modalities and stages of processing in each group for qualitative comparison. Data used as inputs to the second-level PCA were averaged across task conditions, and thus one score per component per subject was entered into the model. A varimax rotation was selected to yield a structure where higher order factors summarizing neurophysiological responses during auditory and visual perception could be interpreted as independent. Promax rotations were also performed which resulted in the same factor structure as varimax rotation; these are included in Supplemental Information.
All subsequent analyses were completed using neural responses to target stimuli only because they include late component responses of the P3 and LPP which were largely absent for nontarget stimuli. We examined associations between first-level PCA components and relevant ERPs, as well as between first-level components across auditory and visual modalities using bivariate Pearson correlations. Diagnostic group differences in each first-level component amplitude and second-level component score were queried using one-way ANOVAs. In order to confirm that the study from which the data were drawn did not impact any of these effects, we used multiple linear regression models controlling for study (i.e., which of the three possible studies each participant’s data was drawn from). Similarly, in order to evaluate whether task performance differed as a function of diagnostic group and study, linear mixed-effects models were fit, again including study as a covariate. Next, bivariate Pearson correlations were used to examine associations of first-level PCA components with task performance, polygenic risk for schizophrenia, and clinical variables. Diagnostic group differences in the relationship between components and task performance were queried using multiple linear regression. For linear models using PRS, ancestry PCs were calculated with PLINK (v1.90; Purcell et al., 2007)) and the first six ancestry PCs were controlled for in analyses along with age and biological sex (Price, Patterson, Plenge, Weinblatt, Shadick, & Reich, 2006). Similar multiple linear regression models were employed to examine relationships between components, diagnostic group, and symptom measures. Finally, we examined relationships between clinical & cognitive variables and groupwise second-level PCA factors. In these analyses, because each factor represented a different combination of component loadings in each group, relationships between factors and clinical symptoms were conducted within each group independently. Furthermore, in order to examine relationships between clinical symptoms and cross-modal factors (i.e., factors onto which component scores from both auditory and visual tasks loaded) for subjects who had clinical data associated with the date of both their visual and auditory EEG, an average symptom score was taken. Across all models, significant main effects and interactions were parsed using marginal means and pairwise comparisons. Results of analyses related to task performance are presented in Supplemental Information.
Based on an average effect size of f = .22 reported in prior studies (Klein et al., 2020), our sample size of an average of 85 participants per group is associated with a power of .94 to detect group differences in ERP components derived from the visual task. Similarly, based on an average effect size of f = .19 reported in prior studies (Pokorny et al., 2021), our sample size is associated with a power of .85 to detect group differences in ERP components derived from the auditory task. Finally, we determined post-hoc power to detect clinical relationships based on all reported symptom correlations in the latter of these above referenced studies. Based on an average absolute value effect size of r = .1133, our total sample size of 340 is associated with a power of .67 to detect relationships between clinical and neural measures. It is possible that additional weaker relationships exist with clinical measures that are too small to be detected.
We have striven in our reporting to comply with transparency and openness promotion (TOP) guidelines. We report a post-hoc power analysis using our sample sizes, all data exclusions, all manipulations and measures, and all other relevant study details. We have made the syntax needed to reproduce results available in supplementary information. Output of all statistical tests is also provided as supplementary information. This study was not pre-registered.
Demographic and clinical characteristics of the sample can be found in Table 1. Differences between groups on demographic variables were queried using one-way ANOVAs and chi-squared tests. Group differences emerged in age, biological sex, estimated IQ, years of education, race, symptom burden, and schizotypy scores. The SCZ group was the youngest, had less education, the lowest IQ, greater antipsychotic use, the highest symptom scores, and was less racially white and more male than other groups. The BP group had lower antipsychotic usage and symptom severity and more years of education than the SCZ group, and lower IQ than the CON group. Age, race, and sex, which are not expected to differ between groups due to factors related to clinical diagnosis, were included in covariates in a second set of omnibus group difference models to establish their impact on results. PRS models also controlled for ancestry.
First-level PCAs of both auditory and visual ERPs yielded early perceptual (auditory N1 and P1, visual N1/P2), mid-latency (auditory P2/MMN, visual N2) and late (auditory P300 and LPP, visual P3/LPP) components. Group differences in amplitude of each component elicited from first-level PCAs were first examined using one-way ANOVAs with group as the independent variable and each component as the dependent variable. Group effects emerged on visual N1/P2 (F~(3,272)~ = 7.01, p = 0.0001, η^2^ = 0.07; SCZ < CON d = −0.56, BP < CON d = −0.60), visual N2 (F~(3,272)~ = 3.02, p = 0.03, η^2^ = 0.03; SCZ < CON d = 0.31), auditory P3 (F~(3,216)~ = 9.31, p < 0.001, η^2^ = 0.11; SCZ< CON d = −0.82, SCZ < SREL d = −0.70, BP < CON d = −0.66, BP < SREL d = −0.54), and visual P3/LPP (F~(3,272)~ = 3.31, p = 0.02, η^2^ = 0.04; SCZ < SREL d = −0.51). All pairwise comparisons noted were significant at p < 0.05, and these group differences remained significant or marginally significant (p< 0.07) when controlling for age, race, sex, CPZ equivalent, and study. When including these demographic factors as covariates, an additional group effect also emerged on the visual P1 (F~(3,265)~ = 2.98, p = 0.03). This pattern demonstrates impairments across early (visual N1/P2), mid-latency (visual N2), and late (auditory P3, visual P3/LPP) stages of processing in SCZ, where component amplitudes were reduced compared to CON. Auditory P3 and visual P3/LPP amplitudes in SCZ were also reduced compared to SREL, who did not differ from CON, suggesting intact processing in relatives of probands with psychosis. Only early visual and late auditory components were reduced in BP compared to CON. Group differences in overall component amplitude are shown in Figure 1.
Significant bivariate Pearson correlations emerged between components across sensory modalities at each stage of processing. Specifically, cross-modal associations were evident between visual N1/P2 with auditory N1 and P2/MMN; visual N2 with auditory N1 and P2/MMN; and visual P3/LPP with auditory P3 and LPP. Primarily, when examined in the full sample, early and mid-latency components across modalities tended to be related to one another but not with late components. Similarly, late components across modalities were related to one another, but not to early and mid-latency components. Graphical illustration of the associations between components in the full sample with associated statistical values are presented in Supplementary Information. We also sought to confirm that these relationships were not due to effects of study from which the data were drawn. To do this, eight multiple linear regression models were run, with each of the eight auditory or visual components as the dependent variable and all cross-modal components, group, and study as predictors. In only two of these eight models was study a significant predictor of component amplitude, with study predicting amplitude on the auditory P2/MMN (F~(2,171)~ = 3.67, p = 0.03) and the auditory P3 (F~(2,171)~ = 3.18, p = 0.04). In each instance, cross modal relationships (i.e., relationships between auditory and visual components) remained significant (p < 0.05) or marginally significant (p < 0.08, referring only to the relationship between auditory P2/MMN and visual N2) after controlling for the effect of study.
To examine whether patterns of covariation differed across groups, a correlation matrix was computed for each group and subject to statistical comparison using the Jenrich test. We found that patterns of association differed between CON and SZ (X^2^ = 48.42, p < 0.01), but not between CON and BP (X^2^ = 33.48, p = 0.22), CON and SREL (X^2^ = 32.29, p = 0.26), BP and SCZ (X^2^ = 33.99, p = 0.20), SREL and BP (X^2^ = 36.99, p= 0.12), or SZ and SREL (X^2^ = 33.69, p = 0.21). Associations between all components in each group are presented in Figure 3.
Group-wise loadings for second-level PCAs are presented in Table 2; visualization of factors across groups is presented in Figure 4. In the CON group, factor 1 showed the highest loadings of late components for both sensory modalities, factor 2 had the highest loadings for early components for both modalities, and factor 3 had mid-latency components across modalities. In no instance did the same pattern of component loadings across factors emerge in another group as was seen in CON, suggesting that normative covariation of auditory and visual components at each stage was disrupted in people with psychotic psychopathology and individuals who carry genetic liability for schizophrenia, but significantly so only in SCZ. Factor loadings of electrophysiological components for each group are displayed in Figure 4. For singular factor structure and loadings for the entire sample composed of all groups as well as associated findings, please refer to Supplementary Information.
A subset of subjects (N = 198) had biospecimen data available from which to calculate PRS; the following analyses are restricted to this smaller group. The optimal threshold identified by PRSice was p < 0.0143, which included 17,513 SNPs. At this threshold, the PRS was significantly associated with the phenotype (p = 2.43 × 10−^5^), explaining an additional 5.98% of the variance beyond the covariates. PRS was a significant predictor of mid-latency visual components, including the visual N1P2 (F~(1,167)~ = 7.37, p = .007) and N2 (F~(1,167)~ = 6.34, p = 0.01). In each instance, increased polygenic risk for SCZ was associated with reduced amplitude (Figure 5). Relationships did not differ across groups, based on non-significant interaction terms in the linear models.
Bivariate Pearson correlations revealed varying patterns of relationships between components and clinical variables. Correlation coefficients for relationships between each component and clinical variables are presented in Table 3. Because of the lack of variability in full-threshold psychotic symptoms in healthy control subjects, relationships with positive and negative symptoms were examined only in the patient groups. All other correlations are reported across the full sample. Here, we describe those that survive correction for multiple comparisons at a Bonferroni-adjusted p < 0.004. In the visual modality, primarily early and mid-latency components were associated with dimensional schizotypy measures. Specifically, increased visual P1 (r = 0.21) was associated with increased disorganized symptoms of schizotypy. Conversely, reduction in the mid-latency visual N2 associated with increased cognitive-perceptual schizotypal symptoms (r = 0.22). In no case were auditory components associated with clinical symptoms at a level that survived correction for multiple comparisons. Conversely, regarding cognitive battery scores, associations were unique to late auditory components. Increased auditory P3 (r = 0.22) was associated with higher estimated IQ and greater digit symbol percentile (r = 0.24), while increased auditory LPP was associated with increased digit span percentile score (r = 0.23). In no instance were visual components associated with cognitive performance at a level that survived correction for multiple comparison.
Associations emerged in CON between factor 1 (late components) and total schizotypy scores (r = −0.25) as well as cognitive-perceptual schizotypy scores (r = −0.37), with a higher loading on late components being associated with lower schizotypy scores. Similar associations emerged in SREL, with greater scores on the late components factor 1 being associated with lower cognitive perceptual schizotypy scores (r = −0.41). No such associations emerged in SCZ. Similarly, in BP the only association that emerged was between factor 1 (deemed ‘late components’ in CON but containing variance from mid-latency and late components in BP) and total symptom severity (r = 0.37).
In the present study we sought to understand the degree to which electrophysiological anomalies associated with auditory and visual perception in schizophrenia reflect shared or unique neural processes. We examined early, mid-latency, and late electrophysiological responses elicited by auditory and visual target-detection tasks to determine whether the degree of shared and unique variance across modalities is dependent on the temporal stage of processing. The inclusion of individuals with schizophrenia spectrum disorders, bipolar disorder, and people with elevated genetic liability for schizophrenia allowed us to additionally examine whether the findings carried clinical specificity or possibly marked a genetic predisposition for schizophrenia. We conducted a first-level principal components analysis of average EEG responses and examined associations with clinical symptoms and polygenic risk. We then conducted a second-level group-wise PCA on these first-level EEG-derived components and examined the covariation between neurophysiological responses to auditory and visual stimuli in each group. Across our sample, first-level auditory and visual PCAs produced similar, but not identical, components, each capturing early, mid-latency, and later stages of processing. Group contrasts of correlation matrices revealed that the covariation of component scores across sensory modalities was disrupted in people with schizophrenia compared to the healthy controls. We further examined the cross-modality covariation using second-level PCA factors. In control participants, early, mid-latency, and later components covaried across auditory and visual modalities based on processing stage, while people with schizophrenia only showed comparable covariation across modalities in the mid-latency components. Furthermore, associations emerged between component scores and polygenic risk, clinical symptoms, and cognitive functioning that were dependent on sensory modality and stage of processing, such that (1) early- and mid-latency visual components were correlated with additive genetic risk for schizophrenia and clinical symptoms, and (2) late auditory components were correlated with cognition.
The emergence of similar components from auditory and visual first-level PCAs suggests that similar sensory processes contribute to perception in each modality. Second-level PCAs conducted in healthy participants confirm this, revealing covariation across modalities in early, mid-latency, and later stages of processing. The covariation of late components is logical, as they share the influence of attentional and cognitive demands than span modalities and have remarkably similar scalp topographies. Early and mid-latency first-level components have markedly different topographies in each modality (see Figure 1), in line with their sources being localized to the auditory and visual cortices (Di Russo et al., 2002; Foxe & Simpson, 2002; Godey et al., 2001; Javitt, 2000, 2009; Näätänen & Picton, 1987). Their covariation in healthy samples is therefore an intriguing finding. Covariation of these earlier components is largely consistent with the notion that early and mid-latency components of sensory processing reflect a combination of both bottom-up sensory specific information originating in sensory cortices, as well as shared top-down neural signals originating perhaps in frontal and parietal regions (Di Russo et al., 2002; Foxe & Simpson, 2002; Godey et al., 2001). Indeed, some frameworks such as the predictive coding theory posit that perception occurs through the integration of bottom-up stimulus information with top-down signals that convey prior contextual knowledge about the environment and task requirements (Friston & Kiebel, 2009; Jin, Jonas, & Mohanty, 2023; Yao et al., 2024). Both top-down predictive and bottom-up sensory signals are hypothesized to be conveyed through rhythmic cortical oscillations which underly the ERPs we report on here (Alamia & VanRullen, 2019). For instance, mechanisms such as phase-amplitude coupling between high and low frequencies appear relevant to perception. Specifically, the phase of slower oscillations is presumed to dictate the amplitude of faster oscillations to promote efficient processing of stimulus information. Evidence suggests that phase-amplitude coupling is important to both auditory and visual perception (Demiralp et al., 2007; Hirano et al., 2018). It may be that the statistical relationships we observed between unique early auditory and early visual signals is due to phase amplitude coupling of both auditory and visual low-frequency sensory signals with a common high-frequency oscillation. Future work could test this more directly. Furthermore, the influence of top-down signals may increase as time passes from the sensory registration of stimuli as is suggested by the later P300 and LPP components having markedly similar topographies and strong covariation across modalities. The observed scalp topographies of these later components are also consistent with their generators being localized primarily outside of sensory cortical areas (Bledowski et al., 2004; Masuda et al., 2018; Scharmüller et al., 2011; Tarkka et al., 1995).
Unique patterns of covariation between ERPs emerged within each of the other groups. In no instance did we observe the same second-level factor structure of components for BP, SREL, or SCZ as was seen in CON. In particular, SCZ showed a pattern of covariation amongst auditory and visual components that statistically differed from CON. People with schizophrenia lacked consistent covariation in early and late ERP components across modalities, with loadings at each stage of processing spread across the second-level three-factor structure. While covariation of mid-latency components in this group may be relatively spared, with factor 1 being composed primarily of mid-latency ERP responses, the strength and consistency of covariation was less than in healthy participants. Mid-latency components in SCZ also failed to covary with other components within the sensory modality, with each factor in SCZ comprising loadings from responses to both auditory and visual stimuli. This pattern suggests that mid-latency components may reflect a disrupted neurophysiological response during perception in schizophrenia that is shared across sensory modalities.
With regard to specific ERP component anomalies, reductions in visual components across stages of processing in SCZ is consistent with a large literature demonstrating disrupted visual processing (Ergen et al., 2008; S. C. Johnson, Lowery, Kohler, & Turetsky, 2005; Vianin et al., 2002; Yeap et al., 2008). Conversely, our finding that of auditory components only P300 amplitude was impaired in SCZ was somewhat surprising. Although an auditory ‘P2/MMN’ abnormality might be expected, it is important to note that this PCA-derived component does not reflect a true MMN because we did not compute the difference in responses for standard and deviant tones. Additionally, the auditory mid-latency component also captures variance associated with the P2 response and the neural responses were elicited by a dichotic listening task rather than a traditional MMN paradigm. Therefore, the auditory mid-latency components we characterized may differ from those reflected in traditional MMN components (Garrido, Kilner, Stephan, & Friston, 2009) and may fail to fully capture processes that blunt MMN responding in schizophrenia. Our finding that two of the eight components were reduced in bipolar disorder as compared to four components reductions in schizophrenia broadly agrees with other reports of ERP amplitude reductions during perception in people with bipolar disorder (Chun et al., 2013; Degabriele & Lagopoulos, 2009; Donaldson et al., 2020; Ford, 2018; Kiang, Kutas, Light, & Braff, 2008; Mathalon, Ford, & Pfefferbaum, 2000; O’donnell, Vohs, Hetrick, Carroll, & Shekhar, 2004). Studies have shown associations between reduced neural response amplitude and symptoms of psychosis (Donaldson et al., 2020; Ford, 2018; Kaustio, Partanen, Valkonen-Korhonen, Viinamäki, & Lehtonen, 2002), which are present to a lesser degree in bipolar disorder (Heron, Jones, Williams, Owen, Craddock, & Jones, 2003). The variable presentation of psychosis with bipolar disorder may explain inconsistent ERP findings across studies and indicate a dimensional relationship between neural response amplitude and psychotic psychopathology that spans diagnostic categories.
The associations we identified between reductions in auditory components and lower scores on cognitive tests is in agreement with a substantial literature demonstrating that auditory ERPs may provide a window into neural mechanisms of cognitive functioning (Davidson & Souza, 2024; Hamilton et al., 2018; Näätänen, 1990). That associations were primarily with late components is consistent with these neural responses reflecting aspects of cognitive functioning such as attention, working memory, and engagement with task stimuli (Cuthbert, Schupp, Bradley, Birbaumer, & Lang, 2000; Donchin & Coles, 1988; R. Johnson, 1988; Schupp, Cuthbert, Bradley, Cacioppo, Ito, & Lang, 2000). Such associations across our sample are unsurprising, as reductions were evident in both auditory P3 amplitude and cognitive functioning in probands compared to healthy comparison subjects (see Supplemental Information). Visual components, on the other hand, were associated only with clinical symptoms and not with cognitive functioning. Reductions in the visual N2 mid-latency component were predictive of more severe self-reported perceptual-cognitive schizotypal symptoms which is consistent with the mid-latency neural responses being relevant to psychotic psychopathology. Studies of schizotypal characteristics in the general population have yielded similar associations (Döring et al., 2016; Sumich, Castro, & Kumari, 2014). Conversely, the association between larger early visual responses captured by the P1 and increased disorganized symptoms is inconsistent with previous work demonstrating reductions in the visual P1 (Yeap et al., 2008). Notably, the P1 was the only visual component for which amplitude was not reduced in SCZ in the present sample.
In no instance did the first-degree relatives as a whole show alteration in the components of neural responses to auditory and visual stimuli. This is consistent with some prior reports indicating a lack of impairment in relatives of individuals with schizophrenia in sensory neural responses (Donaldson et al., 2021; Kiang, Christensen, & Zipursky, 2014; Winterer, Egan, Rädler, Coppola, & Weinberger, 2001), but contrary to others that find deficits (Davenport, Sponheim, & Stanwyck, 2006; Earls, Curran, & Mittal, 2016; Force, Venables, & Sponheim, 2008; Klein et al., 2020; Sponheim, McGuire, & Stanwyck, 2006; Turetsky et al., 2008), possibly reflecting varying levels of clinical symptomatology or degrees of genetic liability for schizophrenia. Indeed, the SREL group in the present analyses did not report greater symptomatology than CON. When we characterized the degree of genetic liability for schizophrenia using polygenic risk scores, we found that greater additive genetic predisposition was associated with reductions in mid-latency visual component amplitudes across all participants, consistent with associations between clinical symptoms and early and mid-latency visual components. Taken together, these findings support the notion that electrophysiological responses during visual perception may tap the neural underpinnings of psychotic symptomatology (Bedwell, Butler, Chan, & Trachik, 2015; Bedwell, Spencer, Chan, Butler, Sehatpour, & Schmidt, 2018; Martin et al., 2021; Salgari, Potts, Schmidt, Chan, Spencer, & Bedwell, 2021) and possibly be a manifestation of genetic liability for schizophrenia. Individuals with greater genetic liability regardless of group memberships in the current analysis may tend to exhibit diminished neural responses during visual perception and also be prone to more schizotypal and psychotic psychopathology. Future work with genetic and clinical high-risk samples may be better able to determine whether visual ERPs have utility as biomarkers of vulnerability and development of psychotic psychopathology.
While similar sensory processes contribute to auditory and visual perception across stages of processing, later ERP components show the greatest similarity, possibly indicating that targeting more global, top-down cognitive processes could result in changes that are more likely to generalize across modalities. Conversely, targeting low-level sensory processes may have both local and global effects. Furthermore, our data suggest that mid-latency components may reflect a neurophysiological process in schizophrenia which is uniquely shared across sensory modalities. It is therefore possible that processes related to mid-latency components (for example, stimulus evaluation, deviance detection, and plasticity) may make suitable targets for interventions such as neuromodulation, as effects may be more generalizable and reproducible. This is, however, speculative, and more work is needed to evaluate these ideas. Furthermore, given the high likelihood that neural origins of mid-latency responses are anatomically separate, it is possible that covariation in mid-latency components reflects the influence of top-down neural processes that similarly affect mid-latency stages of perceptual processing. Future work employing neuromodulation techniques could test whether mechanisms underlying the mid-latency components are influenced by top-down cognitive functions and have potential as treatment targets.
Strengths of the present study include our recruitment of individuals with both schizophrenia spectrum and bipolar disorders, as this allowed us to establish the clinical specificity of effects reported in schizophrenia. Inclusion of a first-degree relatives of people with schizophrenia as well as use of PRS allowed us to examine the degree to which findings were associated the degree of genetic liability for schizophrenia as compared to the development of psychotic psychopathology. Finally, PCA-based analyses provided a means for parsimonious representation of neural responses to auditory and visual stimuli and allowed contrasts of the components elicited by auditory and visual tasks in order to determine whether the covariation of these components was preserved in schizophrenia. Furthermore, traditional ERP data analysis approaches are plagued by several problems that are minimized by employing a PCA-based scoring method. First, traditional ERP scoring approaches are often subjective in the definition of component windows (Luck & Gaspelin, 2016). Use of data-driven and statistically determined methods such as PCA to quantify components reduces the impact of rater subjectivity (Scharf, Widmann, Bonmassar, & Wetzel, 2022; Dien, 2012). Second, EEG measured via electrodes placed on the scalp is composed of a multitude of underlying neural signals generated in the brain and dispersed across the scalp (Nunez & Srinivasan, 2006; Burle et al., 2015), making it a spatially and temporally imprecise measure. PCA based scoring methods reduce this imprecision by relying on statistical covariance in the data to establish component signals (Scharf, Widmann, Bonmassar, & Wetzel, 2022). Third, PCA is particularly helpful for scoring ERPs in populations where increased noise may be present in the data (as is often the case in psychiatric populations; Dien, 2012). Finally, our results suggest that PCA components utilized in the present study are valid representations of the corresponding ERP components (p’s <.001; see Supplemental Figure 1). They are also reliable across groups (with the same factors emerging in probands and healthy subjects) and across studies (with the same factors emerging across 3 independent datasets). For these reasons, use of PCA components results in dependent variables that are more objective, transparently derived, and less subject to bias than traditional scoring methods. For a thorough discussion of the advantages of using PCA in ERP analyses, see Scharf et al., 2022 and Dien, 2012.
Several limitations exist which should be addressed in future research. First, though auditory and visual perceptual responses were both collected using target detection tasks, there were differences in task difficulty. Use of PCA rather than conventional ERP component measurement facilitated contrasting responses across these two tasks. Second, data for this investigation were drawn from several studies collected at different times and using different EEG-collection systems; however, extensive precautions were taken to ensure no or minimal effect on EEG data. Third, EEG tasks were always collected in the same order, and if a task needed to be skipped due to time constraints (e.g., due to participant fatigue or tardiness) the auditory task was not collected. Data may therefore not be missing entirely at random. Fourth, these results derive from analyses that do not establish causality, and caution is therefore warranted in interpretating associations between factors. The association of neural components across modalities does not necessarily imply a singular underlying neural function. Additional work is required to determine causal influences. Finally, our examination of only ERP data from these tasks limits the ability to investigate the dynamics of neural oscillations that have been shown to be important to perception in schizophrenia (Ramsay, Pokorny, Lynn, Klein, & Sponheim, 2024).
The analyses presented in this text were completed using data drawn from three studies completed at the Minneapolis VA Health Care System. Our target population was Veterans with schizophrenia or bipolar disorder, their first-degree biological relatives, and healthy comparison subjects who were matched across demographic factors such as age, gender, and race. Though efforts were made to promote diversity in our sample, it remains generally representative of the target population in the area from which the data were drawn (i.e., skewed slightly more male, white, and middle-aged; predominantly urban, midwestern, and veterans). Replication of our results in other samples will be a crucial next step in identifying the degree to which our findings are broadly applicable to people with psychosis vs specific to veterans with schizophrenia residing in our catchment area. Conversely, the auditory and visual tasks employed and the measures we used to assess symptoms and cognitive functioning have been widely published on. To that end, utilization of other similar and well-validated measures capturing the same constructs would likely yield similar results in a similar sample.
In conclusion, the present study provides evidence for anomalies in several stages of perceptual processing in schizophrenia as indicated by disruptions in the normative covariation of brain responses to auditory and visual stimuli. While early, mid-latency, and late neural responses during target detection covaried across auditory and visual sensory modalities in healthy control participants, only mid-latency components covaried across sensory modalities in people with schizophrenia. Mid-latency responses in schizophrenia to visual stimuli were also associated with clinical symptoms and polygenic risk scores. It is possible that if mid-latency components of perceptual processing are targeted it will affect a neurophysiological contributor to distorted perception that is common to auditory and visual processing in schizophrenia and that beneficial effects of treatment might generalize across sensory modalities. Conversely, results suggest that the modality of intervention may be critical to consider if targeting basic sensory functions that represent feed-forward processes. Remarkably similar and covarying scalp topographies of late neural responses to auditory and visual stimuli imply that these mostly cognitively-mediated components may serve as direct measures of neural functions shared across modality. Therefore, interventions intended to alter these components and their associated processes – for example, through cognitive remediation or neuromodulatory techniques – are likely to have effects that also extend across sensory modalities.