Authors: Dong-Hyun Lee, Kyoung-Mi Jang, Hyun Kyoon Lim
Categories: Original Paper, Cybersickness, Electroencephalography (EEG), Visually Induced Motion Sickness (VIMS), Visual Stimuli
Source: Brain Topography
Authors: Dong-Hyun Lee, Kyoung-Mi Jang, Hyun Kyoon Lim
Virtual reality (VR) is an immersive technology capable of simulating alternate realities, however, it often leads to cybersickness, causing discomfort for users. We conducted an experiment using a group of 30 participants (aged 25 ± 2.1 years) to see the alpha and delta wave changes for three Blank, Video, and Video Pause, with electroencephalography (EEG) recordings. The experiments were repeated three times (Trial 1, Trial 2, and Trial 3). The results showed a significant increase in delta wave power for Video compared with the Blank (p < 0.05). Video Pause showed a significant decrease compared to Video. Alpha waves significantly decreased during the Video compared with Blank (p < 0.05). Alpha waves during Video Pause showed a significant increase compared to Video (p < 0.05). Our study showed consistent alterations in alpha and delta waves across various visual stimuli for inducing cybersickness, and we observed that the decrease in alpha waves may be significantly associated with cybersickness rather than visual stimuli. These findings have implications for advancing cybersickness research.
Virtual Reality (VR) is an immersive technology that leverages computer technology to simulate human sensory experiences, creating an alternate reality that closely resembles the real world. While VR offers a wide range of applications (Berntsen et al. 2016), users often experience symptoms such as nausea, dizziness, and disorientation, which are akin to motion sickness, a condition termed cybersickness (Gavgani et al. 2017; McCauley and Sharkey, 1992; Reason and Brand 1975). Cybersickness includes visually-induced motion sickness (VIMS) and virtual reality-induced symptoms and effects (VRISE) regardless of whether a head mounted display (HMD) is used. In this paper, we will collectively refer to these conditions as cybersickness (Kourtesis et al. 2019). The primary cause of cybersickness is a sensory mismatch, in which the user’s body remains stationary while visual information received through the eyes suggests motion, leading to discomfort (Cobb et al. 1999; Kolasinski 1995; Rebenitsch and Owen 2016).
Developing a quantitative and objective evaluation method of cybersickness is one of the key strategies for reducing its impact (Weech et al. 2019). To date, the main methods for assessments have relied on subjective and qualitative approaches, such as the simulator sickness questionnaire (SSQ) (Carnegie and Rhee 2015; Kennedy et al. 1993; Ujike et al. 2008) and the motion sickness susceptibility questionnaire (MSSQ) (Fowler et al. 2014; Golding 1998; Sawada et al. 2020). These methods have limitations because they rely on participants’ ability to recall sensations during sickness, making real-time quantitative measurements challenging (Fernandes and Feiner 2016; Rebenitsch and Owen 2014). Consequently, current guidelines prioritize the use of biosignal-based assessment methods to achieve an objective and quantitative measurement of cybersickness (Bando et al. 2012; Chang et al. 2023).
Quantitative measures for cybersickness include electroencephalography (EEG), electrocardiogram (ECG), heart rate variability (HRV), and skin galvanic response (GSR) (Guna et al. 2019; Kiryu et al. 2007; Naqvi et al. 2015). Among these biosignals, EEG stands out for investigating cybersickness due to its high temporal resolution, enabling real-time observation of brain activity changes in VR experiences (Chen et al. 2012; Heo and Yoon 2020; Lin et al. 2007; Naqvi et al. 2015). Previous EEG studies on cybersickness have spanned various domains and frequencies (Chen et al. 2010; Krokos and Varshney 2022). Especially, delta waves in the frontal region and alpha waves in the occipital region are commonly associated with cybersickness (Celikcan 2019; Jang et al. 2022; Lim et al. 2021; Wawrzyk et al. 2019).
Delta waves consistently appear in the literature on cybersickness, with increased delta wave activity observed in the frontal lobe regions during cybersickness episodes (Chen et al. 1989; Kim et al. 2005; Krokos and Varshney 2022). Delta waves reflect the arousal system of the brain and are linked to the autonomic nervous system and metabolic processes, including brain activity homeostasis. Prolonged exposure to intense virtual reality stimuli can modify brain wave patterns in the frontal lobe, indicating heightened states of excitement, anxiety, and overwhelm (Güntekin and Başar 2016; Knyazev 2012). These consistent prior studies enabled us to use delta waves as an indicator of whether cybersickness was induced.
Typically, alpha waves decrease with visual stimulation when the eyes are open compared to when they are closed (Adrian and Matthews 1934; Legewie et al. 1969). Previous studies have shown a reduction in alpha waves when participants focus on a cross with their eyes opened as opposed to when the eyes are closed (Li 2010). Alpha waves, unlike delta waves, don’t exhibit the same level of consistency in relation to cybersickness research. Studies involving head-mounted displays (HMD) and car driving simulators have reported an increase in alpha wave activity, whereas those using curved monitors have observed a decrease in alpha waves (Chen et al. 2010; Jang et al. 2022; Kim et al. 2019). Furthermore, studies involving the viewing of 2D and 3D images using smartphones and Cardboard glasses (Google Inc., USA) have indicated that 3D images lead to a more pronounced decrease in alpha waves in the frontal and occipital regions (Xu and Sui 2021). Consequently, these prior studies suggest the need for further investigation to determine whether cybersickness affects alpha waves directly or if alterations in visual stimuli play a significant role.
In this study, we used a video clip designed to induce cybersickness to investigate the alpha wave and delta wave responses, to show whether they stemmed from visual stimuli or cybersickness itself. Changes in delta waves in the frontal lobe were used as an indicator to assess whether the video clip effectively induced cybersickness. We measured changes in alpha waves in the occipital lobe to explore whether they were caused by cybersickness or visual stimuli.
The experiment involved 30 males in their 20s (25 ± 2.1 years). Initially, EEG data were recorded from 31 subjects. The excluded participant did not follow our test instructions and moved his hands and head frequently, resulting in considerable noise in the EEG data, which led to his exclusion from the final analysis. Therefore, the final analysis included 30 participants. To induce cybersickness, participants were selected on the basis of their susceptibility to motion sickness and assessed using the MSSQ. Out of 858 subjects who participated in the MSSQ test, 30 individuals scored above 75% of 20-year-old males (with a score of 21.6) in a previous study, surpassing this threshold with an average score of (31.0 ± 6.5). Given the reported gender-based disparities in some cybersickness studies, which are attributed to factors such as visual field variances and hormonal effects of the menstrual cycle, this study exclusively recruited male participants to mitigate hormonal effects and account for visual field differences (Clemes and Howarth 2005; Stanney et al. 2020). All participants possessed good vision, were in sound health, and were not using any medications that could impact brain function or emotion. Before their involvement, informed consent was obtained from all participants, and the ethics committee of the Korea Research Institute for Standards and Science (KRISS-IRB-2021-03 & KRISS-IRB-2022-03) provided formal approval for this study.
The motion sickness susceptibility questionnaire (MSSQ) identifies motion sickness susceptibility using nine sources of motion sickness, including cars, buses, trains, airplanes, small boats, ships, and playground swings. It assesses motion sickness experienced in childhood before age 12 (referred to as A) and in adulthood over the last 10 years (referred to as B), covering the frequency of motion sickness on a scale of 0 (never experienced) to 3 (experienced very often).
The simulation sickness questionnaire (SSQ) evaluates cybersickness through 16 symptoms on a four-point scale ranging from 0 to 3, with higher scores suggesting more severe symptoms. The SSQ is divided into three nausea, oculomotor, and disorientation. This experiment involved the administration of a pre-experiment SSQ (Pre-SSQ), three SSQs (SSQ1, SSQ2, and SSQ3) after watching each video, and a post-experiment SSQ (Post-SSQ) upon recovery from cybersickness (Table 1).
Table 1Summary of related works using SSQPre-SSQSSQ1SSQ2SSQ3Post-SSQ SSQ total score
0
16.58(17.95)
21.69(20.13)
24.56(21.63)
0.13(0.70) Nausea08.90(14.35)11.13(13.95)13.04(15.86)0Oculomotor015.16(15.78)21.98(18.29)24.00(18.26)0.3(1.4)Disorientation020.88(22.66)24.13(26.64)28.30(29.96)0Note. SSQ: Simulation sickness questionnaire; ( ): standard deviation
Participants completed the Pre-SSQ before the experiment. The Pre-SSQ was conducted to ensure that subjects were not affected by cybersickness prior to the experiment. Participants were informed that the experiment would begin with the video and were instructed to focus on the visual stimuli. The experimental process was repeated three times (Trial 1, Trial 2, and Trial 3). The experiments comprised Blank, where participants stared at a black screen for 30 s; Video, where participants stared at a cybersickness-inducing video for 60 s; and Video Pause, where participants stared at a screen with frozen Video for 30 s. Between each of the three repetitions, participants took a 5-min break to recover from motion sickness and completed the SSQ orally during the break. Following the viewing of the third video, participants had a sufficient break to recover from cybersickness and completed the Post-SSQ once they felt they had recuperated from cybersickness (Fig. 1).
Fig. 1Experimental protocols
In this experiment, we used the Unity game engine program (Unity Technology) to create VR content. The VR content is displayed on a curved monitor (3,840 × 1,080 pixels, 1,196 × 336 mm, Samsung). The VR content comprised three rotations (pitch, yaw, and roll), zoom in, and zoom out, with a movement speed of 8 m/s and a rotation angle speed of 60º/s. To simplify the content, we excluded vertical elements such as buildings and trees, which contribute to cybersickness. This simple composition with the sky and ground only was to focus on the screen movements (rotation, zoom–in, and–out) simplify the motion content eliminating vertical components such as trees and buildings (Lim et al. 2021). Given that VR content could influence cybersickness through various variables, such as rotation speed, linear movement speed, axis of motion, background complexity, and scene fidelity, we aimed to keep the conditions as simple as possible to minimize the impact of other variables. The content duration was 2 min, and participants were informed that the content was about to begin. The content started with a 30-second black screen, played the video for 1 min, and ended with a 30-second pause (Fig. 2). The participants’ eyes were instructed to maintain 65 cm away from the 49-inch curved monitor displaying the VR content. A black screen fence obstructed the view behind the curved monitor, preventing the observation of other objects.
Fig. 2EEG recording condition. (a) Blank, (b) Video, and (c) Video Pause
We conducted EEG recording in an electrically shielded and soundproofed room using a 256-channel HydroCel Geodesic Sensor (Net Amp 400; Electrical Geodesics Incorporated, Eugene, OR), with Cz serving as the reference electrode and impedance maintained below 50 kΩ. The EEG signals were recorded within a 0.01–200 Hz bandpass range at a sampling rate of 1000 Hz and preprocessed using a 0.1–50 Hz bandpass filter as well as digital filtering. These data underwent independent component analysis (ICA) using the EEGlab toolbox in Matlab to eliminate noise artifacts such as eye movements, blinks, and ECG interference. For all participants, we segmented the full 2-min EEG measurement, allocating 30 s from the beginning for the Blank condition, 60 s into the beginning part video for the Video condition, and 30 s after the Video stopped for the Video Pause condition (Fig. 3). Subsequently, we down-sampled these continuous EEG data to 250 Hz and utilized the Fast Fourier transform (FFT) to analyze the absolute power of the EEG by spectrally classifying the delta (1–4 Hz) in the frontal region and the alpha (8–13 Hz) in the occipital region (Fig. 4).
Fig. 3EEG experimental environment and extraction area
Fig. 4256ch Sensor Net single fitting image of Frontal (pink) area and Occipital (green) area
Statistical analyses were conducted using SPSS 27 software. We employed a repeated measures analysis of variance, incorporating three factors at the time (Trial 1, Trial 2, and Trial 3) to represent the repeated sequence of EEG measurements and three factors at the condition (Blank, Video, and Video Pause) to account for the differences in visual stimuli. These factors were considered to be within-subjects factors. To address any violations of sphericity, we applied the Greenhouse–Geisser correction and reported the adjusted p-values. In addition, we examined correlations between EEG data, serving as objective measures, and both SSQ total scores and SSQ subscores, which serve as subjective indicators of cybersickness. These correlations were calculated using the Pearson correlation coefficient.
For the analysis of variance for delta wave power, a significant difference was observed in the condition (F = 4.49, p < 0.05), whereas no significant difference was found in time (F = 1.08, p = 0.32). The mean delta power in the frontal region across conditions was as Video (10.08 ± 1.23) > Video Pause (7.19 ± 1.36) > Blank (6.20 ± 1.44). Compared with Blank, Video showed significant increases (p < 0.05). When comparing Video and Video Pause, Video Pause showed a significant decrease (p < 0.05) (Figs. 5 and 6). No significant difference was observed between the Blank and Video Pause (p = 1.0), and no interaction effect of time x condition was observed (F = 0.35, p = 0.74) (Table 2). Concerning the SSQ and delta wave relationship, the SSQ1 subscales nausea (r = 0.37, p < 0.05) exhibited a positive correlation. However, the SSQ total scores and other SSQ subscales did not show significant relationships with delta waves (Table 3).
Fig. 5Comparison of delta wave absolute power averages across visual stimulus ( ** P < 0.05 error Standard error)
Fig. 6Topographic distribution of the three conditions (Blank, Video, and Video Pause) in delta (frontal lobe) and alpha (occipital lobe); Red square box (Frontal region), and blue square box (Occipital region)
Analysis of variance revealed a significant difference in alpha wave power based on condition (F = 10.6, p < 0.05), and a significant difference in time (F = 7.66, p < 0.05) and no interaction effect of the time x condition was observed (F = 1.42, p = 0.24) (Table 2). The mean alpha power in the occipital lobe region across conditions was as Blank (4.38 ± 0.79) > Video Pause (3.35 ± 0.91) > Video (1.6 ± 0.33). When compared with Blank, Video showed significant decrease (p < 0.05), and compared to Video Pause, Video also showed a significant decrease (p < 0.05). However, no significant difference was found between Blank and Video Pause (p = 0.19) (Figs. 6 and 7). The mean alpha power in the occipital lobe area showed the following order across Trial 3 (3.86 ± 0.76) > Trial 2 (3.01 ± 0.7) > Trial 1 (2.45 ± 0.52). A significant decrease was observed in Trial 1 when comparing it to Trial 3 (p < 0.05) and a significant decrease in Trial 2 when comparing it to Trial 3 (p < 0.05). However, SSQ total and subscale scores did not show a significant relationship with alpha wave power (Table 3).
Fig. 7Comparison of alpha wave absolute power averages across visual stimulus ( ** P < 0.05 error Standard error)
Table 2RM ANOVA results with factors time X conditionDeltaAlphaFdfFdfTime1.081.157.662Condition4.491.5310.6**2Time x Condition0.352.361.422.91Note. **p < 0.05; df, degrees of freedom; p-values adjusted using Greenhouse–Geisser correction
Table 3Pearson correlation between Delta and SSQ scores, alpha and SSQ scoresTotalNauseaOculomotorDisorientationTrial1Trial2Trial3Trial1Trial2Trial3Trial1Trial2Trial3Trial1Trial2Trial3Delta0.27−0.080.050.370.300.08−0.04−0.03−0.170.170.000.00Alpha0.210.110.060.210.190.180.160.110.050.120.050.01 P < 0.05
In this study, we presented VR content designed to induce cybersickness in order to investigate whether the EEG responses stemmed from visual stimuli or cybersickness itself. We used a video designed to induce cybersickness to investigate whether cybersickness or visual stimulation caused changes in alpha waves in the occipital lobe, and measured delta waves in the frontal lobe to assess whether the video effectively induced cybersickness. For this purpose, we measured EEG during Blank, where subjects looked at a black screen after recognizing that the VR content was playing, Video, where subjects looked at a cybersickness-inducing video, and Video Pause, where subjects looked at a paused segment in the cybersickness video. The results demonstrated a notable increase in delta power and a decrease in alpha power during Video when compared to Blank and Video Pause.
Similar findings align with this increase in delta power, particularly in the context of cybersickness (Chen et al. 2010; Kim et al. 2005; Min et al. 2004). An object-finding VR experiment using a rear-projected cave display system identified an increase in delta power as an indicator of cybersickness (Kim et al. 2005). To induce motion sickness, they built a driving simulator built with a motion platform within a 360° rear-projection display setup, which provided both visual and vestibular stimulation, aiming to confirm the increase in delta waves (Chen et al. 2010). During the assessment of simulator-induced motion sickness triggered by simulated car driving, an increase in delta power was observed, particularly in the frontal lobe (Min et al. 2004). Delta waves play a role in the autonomic nervous system, affecting brain arousal system activity and causing drowsiness. Given that the frontal lobe of the brain is involved in and responsible for motor function and spontaneity, changes in brain wave patterns are commonly reported in response to VR stimuli (Güntekin and Başar 2016; Knyazev 2012). Consistent with previous studies, this study found a significant difference in the main effect of delta waves. An increase in delta waves during Video can be used as a measure of cybersickness. A decrease in delta power during Video Pause can be interpreted as a recovery from the cybersickness induced by Video. These results suggest that the frontal delta wave activity experienced a significant rise during exposure to the Video, indicating a direct correlation between this Video and cybersickness.
Varying findings have been reported regarding the influence of VR on alpha-wave activity. An increase in alpha waves during VR has been reported (Chen et al. 2010; Krokos and Varshney 2022), while a decrease in alpha waves was observed when participants experienced a virtual roller coaster on a monitor screen (Baumgartner et al. 2006). In addition, a separate study reported a significant reduction in alpha waves compared with baseline levels when subjects viewed cybersickness-inducing videos in a test-retest (Lim et al. 2021). They also observed a significant declinedecrease in alpha wave activity from baseline levels in both sensitive and non-sensitive groups when participants engaged with cybersickness-inducing VR content (Jang et al. 2022). Furthermore, alpha waves exhibited higher levels in 3D videos that incorporated rest frames than in videos lacking such intermissions (Chang et al. 2013). The main effect in our study was significantly different in the alpha wave condition. In comparison to Blank, Video exhibited significant reductions in alpha wave activity. In addition, the cybersickness inducing Video exhibited a significant decrease in alpha wave activity compared with Video Pause. These findings are consistent with previous research that has reported a decline in alpha power as a manifestation of motion sickness. Typically, a decrease in alpha power is indicative of heightened neural activity, particularly associated with outward-directed attention in the visual domain (Mo et al. 2013; Sadaghiani and Kleinschmidt 2016).
However, distinguishing whether this decrease in alpha power is primarily a consequence of alterations in visual stimuli or the induction of cybersickness posed a challenge. To address this, our study analyzed different types of visual stimuli. While some conditions did not yield significant differences, the results showed a pattern as Blank > Video Pause > Video. In simpler terms, the Video condition, featuring cybersickness-inducing elements, resulted in the most pronounced reduction in alpha power. The Video condition showed a significant decrease in alpha power when compared with the Blank and Video Pause. In other words, the alpha waves were significantly lower in the complexly designed Video intended to induce cybersickness than in Blank and Video Pause, which showed simple and paused images. These results show the same trend as delta waves measured to assess cybersickness, but the increase and decrease are reversed. These results suggest that the decrease in alpha power primarily stems from cybersickness, rather than being influenced by visual stimuli.
In this study, the correlation between SSQ and EEG was only positive for delta waves in the SSQ1 subscale nausea. However, we did not find any significant associations between EEG and SSQ. Previous studies have shown a positive correlation between delta and SSQ-disorientation scores in the frontal lobe (r = 0.49, p < 0.05) (Jang et al. 2022). In another study, no significant correlation was found between EEG and SSQ scores except for the correlation between temporal lobe and alpha (r = −0.5) (Lim et al. 2021). Chen et al. reported a positive correlation between alpha waves and questionnaire scores (r = 0.5) (Chen et al. 2010), and another study reported no significant correlation between EEG relative power (Chuang et al. 2016). This study focused on Delta and Alpha waves, so further research on various EEG frequencies is needed. Alpha waves could be subdivided into specific frequencies such as Lower-1 Alpha, Lower-2 Alpha, and Upper Alpha frequencies (Klimesch 1999).
The study exclusively involved men in their twenties, which limits the generalizability of the findings. In future research, we plan to conduct gender- and age-specific studies. Although the term “cybersickness” may not fully apply due to the use of a curved monitor instead of a complete VR setup, prior studies have classified VIMS as a form of cybersickness, supporting our terminology choice (Kourtesis et al. 2019; Lim et al. 2021). Nevertheless, with VR development primarily centered on HMD devices, research tailored to these requirements is essential. To address this, the authors are preparing a comparative study on curved monitors and HMD. Due to the composition of the VR content, the presentation of visual stimuli could not be randomized, and the cybersickness induced by the EEG during the Video may have been affect the EEG during Video Pause. Previous research suggested that a rest period of 11.5 ± 7.1 min can alleviate cybersickness, underscoring the importance of exploring varying rest period durations in subsequent studies (Woo et al. 2023). Although we did not consider mental workload and attention, which could help evaluate the psychological status using evaluation tools such as the Profile of Mood States (POMS) and Social Avoidance and Distress (SAD), these evaluations are highly recommended for future studies (Ronca et al. 2024). It is worth noting that this study excluded the vertical element in content compositing to minimize the complexity of VR content and reduce cybersickness. Future research will examine the impact of content complexity.
In this study, we presented VR content designed to induce cybersickness to investigate the EEG responses whether they stemmed from visual stimulation or cybersickness itself. By inducing cybersickness through visual stimuli, we observed an increase in delta waves and a decrease in alpha waves. These findings hold significant importance for the field of cybersickness research, particularly in the context of using EEG measurements alongside VR content.