Authors: Chuan Liu, Liangcheng Qu, Qixue Li, Yiting Cao, Jingping Shi, Chao Yu, Weiguo Liu, Kuiying Yin
Categories: Aging Neuroscience, EEG, MRI, Parkinson’s disease, minor hallucination, visual reconstruction
Source: Frontiers in Aging Neuroscience
Visual hallucination is a prevalent psychiatric disorder characterized by the occurrence of false visual perceptions due to misinterpretation in the brain. Individuals with Parkinson’s disease often experience both minor and complex visual hallucinations. The underlying mechanism of complex visual hallucinations in Parkinson’s patients is commonly attributed to dysfunction in the visual pathway and attention network. However, there is limited research on the mechanism of minor hallucinations.
To address this gap, we conducted an experiment involving 13 Parkinson’s patients with minor hallucinations, 13 Parkinson’s patients without hallucinations, and 13 healthy elderly individuals. We collected and analyzed EEG and MRI data. Furthermore, we utilized EEG data from abnormal brain regions to train a machine learning model to determine whether the abnormal EEG data were associated with minor hallucinations.
Our findings revealed that Parkinson’s patients with minor hallucinations exhibited excessive activation of cortical excitability, an imbalanced interaction between the attention network and the default network, and disruption in the connection between these networks. These findings is similar to the mechanism observed in complex visual hallucinations. The visual reconstruction of one patient experiencing hallucinations yields results that differ from those observed in subjects without such symptoms.
The visual reconstruction results demonstrated significant differences between Parkinson’s patients with hallucinations and healthy subjects. This suggests that visual reconstruction techniques may offer a means of evaluating hallucinations.
Keywords: minor hallucination, Parkinson’s disease, EEG, MRI, visual reconstruction
Visual hallucination (VH) is a perceptual phenomenon characterized by the occurrence of false visual experiences while an individual is awake (Shine et al., 2014), resulting from erroneous neural coding in the brain. One of the common neurological diseases related to VH is Parkinson’s disease (PD) (Barnes and David, 2001; Diederich et al., 2009). In the past, complex VH has been roughly attributed to levodopa’s effects, thought to be a side effect, but as research progresses, it is now thought to be a symptom of a disconnect between the dorsal, ventral, and default-mode networks, mediated by the quality of visual input (Muller et al., 2014). As the disease progresses, CVH become more pronounced, evolving from simple visual disturbances such as flashes or geometric patterns to vivid depictions of colorful animals and figures. This progression can lead to a gradual loss of the ability to differentiate between CVH and reality (Fénelon et al., 2000). However, research has indicated that even in the early stages of PD, prior to medication treatment, some patients may exhibit mild VH symptoms, which are referred to as Minor Hallucinations (MH). Minor hallucinations include three existential hallucinations, through sexual hallucinations, illusions (Lenka et al., 2019).
In order to analyze the data and reconstruct the vision, it is necessary to understand the mechanism of visual hallucination. It is a good way to study CVH from a model-building point of view. For example, Collerton et al. (2023) provide eight models to explain the mechanism of CVH. With the development of research, the current mainstream consensus on the mechanism of Parkinson’s disease (PD). CVH is the theory that bottom-up and top-down information flow transmission is impaired, bottom-up: impaired transmission of visual information through the optic nerve-optic chiasm-lateral geniculate body-primary visual cortex; Top-down: impaired transmission of visual information through the ventral and dorsal advanced processing pathways of visual information; At present, we prefer to study the processing of advanced information from the aspect of brain network. Recruiting patients is a difficult task, and as hallucinations become more severe, information gathering becomes more difficult. We try to recruit a certain number of patients with minor hallucinations. A study investigating structural and functional changes associated with MH found that disruption of the internal organization of the DMN and its precise balance with the attentional control network is responsible for minor hallucinations in patients with Parkinson’s disease (Bejr-kasem et al., 2019). This is consistent with the attention network hypothesis for CVH proposed by Shine et al. (2015) and Caviness (2014). It is concluded that MH and CVH have similar structural and functional correlations. It is reliable to study the mechanism of visual hallucination in patients with minor hallucinations.
In order to study the mechanism of MH subjects with PD, a total of 13 PD patients with MH, 13 PD patients without hallucinations, and 13 healthy individuals were recruited for the present study. EEG and MRI data were collected and analyzed across all brain regions to investigate the neural correlates of MH in PD. The aberrant brain regions were found to contain neural coding information related to the experience of hallucinations. Furthermore, visual reconstruction techniques were employed to validate the association between MH and EEG data derived from these abnormal brain regions.
We recruited 13 PD patients with minor hallucinations (PD-MH), 13 PD Patients without hallucinations (PD), and 13 healthy controls (HC), respectively. PD patients were grouped into hallucinations and non-hallucinations according to the Unified Parkinson’s Disease Rating Scale (UPDRS), Part I, Item 2(Goetz et al., 2003). The participants who got score of 2 or more were grouped into hallucinations, the others were grouped into non-hallucinations. All subjects were recruited at Nanjing Medical University Affiliated Brain Hospital and all experimental procedures were approved by the Ethics Committee of Affiliated Brain Hospital of Nanjing Medical University (Ethics approval 2019-KY018-01). The inclusion criteria is Mini-Mental State Examination (MMSE) score of more than 17. Exclusion criteria included patients with eye diseases; limitations in MRI scanning, such as claustrophobia or pacemaker implantation; a history of cerebral infarction or brain tumor.
The demographic and clinical characteristics of all participants are presented in Table 1. There were no significant differences in age, gender, education level, cognitive ability, disease duration, and H-Y staging between the PD-MH group and the PD group. It is important to note that all PD patients had not previously received levodopa treatment, thus ruling out any potential drug effects.
In this study, both resting-state and task-state data were collected from all participants. The resting-state data acquisition lasted for a duration of 60 s with eyes closed. For the task-state data collection, participants were instructed to sit in a room where objects and people were arranged in a complex manner. The task required participants to verbally describe the objects and people they observed in order to assess their visual orientation and content. During the task, a camera was positioned directly behind the participants’ eyes, capturing the visual scenes they were observing in real-time. In terms of MRI data collection, a sequence of resting-state data was obtained. During the MRI session, participants were in a resting state, characterized by a clear and relaxed state of mind, with unknown eye closure. No task-state magnetic resonance data was collected, and no specific gaze point was recorded.
The EEG recording was taken with the 64-channel EGI GES systems using AgCl electrode (10–10 standard mapping). The data was sampled at a frequency of 1,024 Hz, and filtered between 0.1 and 50 Hz by FIR filter. Noisy or unusable channels was removed according to the Nina rule (Bigdely-Shamlo et al., 2015). The data was processed using eeglab toolkit in MatLab (MathWorks). Independent Component Analysis (ICA) was used to separate the noise components and the data source components. Then the bad components of EEG data are manually removed, and finally restore the sequence EEG data.
MRI was obtained using a Siemens 3.0 T singer scanner (Siemens, Verio, Germany) and an 8-channel radio frequency coil at the Nanjing Medical University Affiliated Brain Hospital. The 3D-SPGR sequence was used to obtain three-dimensional T1-weighted images in the sagittal plane with the following TE = 3.34 ms; TR = 2,530 ms; flip angle = 7°; 128 sagittal surface slices; 1.33 mm slice thickness; matrix = 256 × 256. Functional images were collected using T2-weighted single-shot EPI 240 time points; TE = 30 ms; TR = 2000 ms; FOV = 240 × 240 mm 2; matrix = 64 × 64; flip angle = 90°; 30 axes 3.0 mm thick; section gap = 0 mm. The MRI data adopts the resting-state fMRI data processing assistant toolkit DPARSF based on the statistical parameter mapping software SPM8 to be processed. To ensure data quality, the first 10 moments of each subject’s data were discarded. We performed slicing timing and motion calibration on the rest of the images. Based on head motion recordings, all participants had a maximum displacement of less than 2.0 mm on the x, y, or z axis, and a maximum angular rotation of less than 2° on each axis. After spatial normalization of the T1 space, all images were resampled to 3 × 3 × 3 mm-sized voxels and spatially smoothed using a Gaussian filter with a half maximum width of 4 mm. The fMRI data were then subjected to temporal bandpass filtering (0.01–0.08 Hz) to remove low-frequency drift and physiological high-frequency noise.
x2 ).
In the processing of MRI data, the ICA method was employed using functional magnetic resonance imaging (fMRI) data. Seven brain functional sub-networks were identified based on the AAL template for intra-network comparison. The Shapiro–Wilk test was used to assess the normality of the data, and the ANOVA test was conducted to analyze the significance of the data across the three groups. Multiple comparisons were then performed to examine the differences between groups. A significance level of p < 0.05 was considered statistically significant.
Additionally, indices of fractional anisotropy (FA) and mean diffusivity (MD) were calculated, and whole-brain fiber tract tracking was conducted using diffusion tensor imaging (DTI) based on the MRI data (Le Bihan et al., 2001; Alexander et al., 2007; Thomason and Thompson, 2011). This analysis was performed on the DAN and VAN networks. The fiber bundle orientation map of specific brain regions was determined using the ICBM-DTI-81 template provided by Johns Hopkins University.
A deep learning network is capable of computing features of input information through multiple iterative processes. In the context of vision-based reconstruction technology, deep network technology is employed to simultaneously calculate two sets of deep network structures in parallel. One network is responsible for computing features of the input EEG signals, while the other network supervises the original stimuli. The iterative EEG features are then transmitted to the feature decoder, resulting in the output of a reconstructed visual stimulus image (Qu et al., 2021). The algorithm network flow chart depicting this process is illustrated in Figure 1.
Figure 1 Visual reconstruction algorithm flow chart.
In our study, visual stimuli in the form of pictures were utilized for our subjects. We collected EEG data for a duration of 500 ms following the presentation of the stimuli. Subsequently, the EEG signal underwent offline preprocessing as described earlier. The preprocessed signal was then fed into the first group of Long Short-Term Memory (LSTM) recurrent neural networks to extract the features present in the EEG data. Simultaneously, the stimulation image associated with the EEG signal was inputted into the second group of neural network VGG, which performed calculations for encoding and decoding models of the stimuli images, thereby extracting the features of the image coding process. These features were then transmitted to the first group of LSTM networks, resulting in the acquisition of EEG features related to the stimulus image features. At this stage, the EEG features associated with the stimulus image features were sent to the constructed VGG model, where image decoding and reconstruction were carried out, ultimately producing the reconstructed image.
LSTM is a recurrent neural network (RNN) that incorporates hidden computational gates. RNNs are designed to handle data with semantic associations, such as text sentences, semantics, reasoning, and stocks, which are logically related. EEG signals, in particular, are suitable for recurrent neural networks as they contain human physiological behaviors and logical patterns. However, when dealing with large amounts of data, the iterative calculation of RNNs may result in gradient explosion. To address this issue, a special type of RNN called LSTM, which includes hidden gate settings, is utilized. The network structure model of LSTM is depicted in Figure 2.
Figure 2 LSTM network model.
In Figure 2, a hidden gate structure called the tan gate is added to the input data h. This hidden gate structure performs long-short-term memory sequence calculations each time the input data h is processed. During each calculation, a parameter related to the forward calculation is randomly omitted, and the input parameter is repeated twice. The resulting tan calculation is then fed into the next hidden gate h. By iteratively calculating the LSTM recurrent neural network, the features of the EEG signals fed into the neural network can be outputted as fE .
Simultaneously, the stimulation image associated with the EEG signal is sent to a second neural network called VGG. This network is responsible for encoding and decoding models of stimuli images, extracting features from the image coding process. VGG is a deep encoding and decoding computing network and is considered a classic representative network structure of convolutional neural networks (CNNs). It possesses a deep network structure and can perform numerous iterative calculations on input images through operations such as convolution, pooling, and activation. This allows VGG to compute the underlying features of the input image and construct an encoding and decoding model for similar images. The deeper network structure of VGG enables it to encode and compress images to a higher degree. However, one drawback is that the reconstructed images after decoding may exhibit lower accuracy.
The encoding and decoding model of the input images was constructed using the VGG model. The forward structure of the network was then utilized to generate the encoding results of the stimulus image corresponding to the input’s EEH time series. The results fP were sent to the first group of network LSTM as the time-stamp for image encoding results and for calculating the supervision loss of the EEG feature extraction network. The supervision mode employed the Archirid loss Formula (1).
After the supervision of Equation (1) and the stimulus image, the EEG signal sent to the LSTM network would eventually output EEG features fE related to the stimulus image feature.
At this time, the EEG features fE associated with the stimulus image features were sent to the constructed VGG model, and the decoding calculation of this model was regarded as image features, fE and image decoding and reconstruction were performed on them.
The original image was used as supervision, and MES loss was applied to the reconstructed image as decoding supervision, as shown in Formula (2):
Then, the decoded image with original image supervision was used as the output of the decoder, that is, the reconstructed image is obtained.
For the purpose of our study, we collected EEG data spanning 500 ms from the PD-MH subject during approximately 1,200 experiments, which encompassed instances where the partner either faced the subject or did not face the subject. Out of these experiments, 1,000 were used as a training set and 200 were allocated as a test set.
The structural similarity index (SSIM) was employed to assess the accuracy of the reconstruction as shown in Formula (3).
where, μx is the average value of x,
σx2
is the variance of x, μy is the average value of y,
σy2
is the variance of y,
σxy
is the covariance of x and y,
c1,c2
are constants.
Figure 3 depicts the EEG mapping results for 13 patients with PD-MH, 13 patients with PD, and 13 HCs across various frequency ranges. The color scheme in the figure represents the average intensity, with red indicating high intensity and blue indicating low intensity.
Figure 3 (A) δ frequency band (B) θ frequency band (C) α frequency band (D) β frequency band (E) γ frequency band.
In comparison to the healthy controls, the PD-MH patients exhibited an elevated power ratio in the α and θ frequency bands, particularly at the junction of the frontal lobe, occipital lobe, fronto-parietal junction, and occipital-parietal junction. Conversely, a decrease in power was observed in the γ frequency band, specifically in the frontal lobe, parietal lobe, and occipital lobes.
Table 2 presents the results of a multiple-comparison test conducted on three groups of subjects in the analysis of MRI functional network. The table records the connections between brain regions, denoted as Node-a and Node-b. The term “excellent” indicates that the functional connectivity of the PD-MH group has a significant advantage, while the absence of this term suggests a significant disadvantage in functional connectivity for the PD-MH group.
The PD-MH group exhibited significantly enhanced connectivity between the frontal gyrus and middle frontal gyrus compared to the HC group. These connections also involved the Calcarine, and enhancements were observed in the parietal and occipital lobes. In comparison to the PD group, the PD-MH group showed enhancements in a limited number of brain regions, such as Occipital gyrus.
Table 2 displays complex network nodes that require further investigation to establish their functional connections. To this end, seven functional networks were selected for significance analysis, and the results are presented in Table 3. The second and third columns of Table 3 indicate the p-values associated with the significance of the functional networks.
Table 3 reveals substantial differences among the DMN, DAN, and VAN networks for the three groups of subjects, prompting a specific investigation into these networks.
In the analysis of the DMN, the functional connections of the posterior cingulum, medial prefrontal cortex, angular gyrus center dorsal medial subsystem, and medial temporal subsystem were compared among the three groups of subjects. The results are displayed in Table 4. It was observed that the DMN network in the PD-MH group exhibited significant enhancement, particularly in the dorsal medial subunit centered on the medial prefrontal cortex.
Given the connection strength is close between the PD-MH group and the HC and PD groups, a connection strength above 0.8 in functional connectivity was considered an effective threshold. Consequently, the average DMN of the PD-MH group with hallucinations and the HC group was determined. The network connection traces are depicted in Figures 4A–D. Figures 4A,B represent the network diagrams of the dorsal-medial subsystems, while Figures 4C,D represent the temporal-medial subsystems. The default network of PD-MH patients exhibited greater connection complexity than that of the HC group, particularly in the dorsal subsystem.
Figure 4 (A) dorsal medial subsystem network of PD-MH group (B) dorsal medial subsystem network of HC group (C) temporal -medial subsystem network of PD-MH group (D) temporal -medial subsystem network of HC group.
For the functional sub-networks, DAN, and VAN of visual information processing, the average connection result with DMN also used 0.8 as the threshold, and the network connection is depicted in Figures 5A–D. This figure only includes the connection between the visual information processing sub-network and the DMN network, not includes the connection of the DMN network itself. As can be seen from the figure, the VAN network was more closely connected with the DMN network than the DAN network, while in the connection with the DMN network, the dorsal-medial system was more closely connected with the attention network, namely PD - MH In the network connection of group patients, the connection between DMN network and DAN network was weakened, and the connection with VAN network was more intimate, and it was more dependent on DAN.
Figure 5 DMN subsystem network connection. (A) Dorsal attention network and dorsal medial subsystem. (B) Ventral attention network and dorsomedial subsystem. (C) Dorsal attentional network and temporal - medial subsystem. (D) Ventral attention network and temporal - medial subsystem.
In the analysis of functional sub-networks involved in visual information processing, the threshold of 0.8 was utilized to determine the average connection strength with the DMN. The resulting network connections are illustrated in Figures 5A–D, which specifically focuses on the functional sub-networks associated with visual information processing. It is important to note that the connections shown in the figure do not include the connections within the DMN network itself. The figure reveals that the VAN exhibited a stronger connection with the DMN network compared to the DAN. Furthermore, within the connection to the DMN network, the dorsal-medial system demonstrated a closer association with the attention network, specifically the posterior-dorsal and medial-hemispheric regions. In the network connections observed in the group of patients, the connection between the DMN network and DAN network was weakened, while the connection with the VAN network became more prominent, indicating a greater reliance on the DAN network.
Fiber tracts were delineated based on the findings of brain regions exhibiting significant differences in certain functional brain networks, as presented in Table 2. The visualization figures were generated using Mrtrix 3 software on a Linux ubuntu 16.04 platform. A total of 5,000 fiber tracts were sampled for both groups. Figure 6 illustrates that, in comparison to the HC group, the PD-MH group displayed a lower density and more dispersed arrangement of fiber bundles, whereas the HC group exhibited a higher density and more compact configuration.
Figure 6 (A) dorsolateral superior frontal gyrus-cuneus of PD-MH group (B) suboccipital gyrus-insula of PD-MH group (C) middle temporal gyrus-amygdala of PD-MH group (D) dorsolateral superior frontal gyrus-cuneus of HC group. (E) suboccipital gyrus-insula of HC group (F) middle temporal gyrus-amygdala of HC group.
Based on the findings from the analysis of fMRI and EEG data, we identified specific channels within the DMN that are located near the dorsal subsystem. Additionally, we selected electrodes in the frontal lobe region associated with the VAN, electrodes in the occipital lobe region situated in the inferior occipital gyrus, and electrodes in the temporal lobe region linked to memory recall. These channels were utilized for the visual reconstruction of a PD-MH subject who experienced persistent hallucinations of being stared at. To minimize any potential disturbance to the subject, we opted for a slightly complex scene consisting of one person at a close distance and another person at a greater distance. Throughout the experiment, the person would either face the subject directly or have their side face visible. The actual experimental scenes are depicted in Figure 7A (subject facing) and Figure 7B (side face).
Figure 7 Real experimental scene of visual reconstruction. (A) Partner in the distance faces the PD-MH subject. (B) Partner in the distance does not face the PD-MH subject.
To classify the two scenarios (i.e., whether the partner’s face was directed toward the subject or not), we employed a deep learning network, as illustrated in Figure 8. The SSIM of Figure 8 is 0.747. However, the findings indicate that the reconstructed images consistently depict the subject facing forward and are unable to differentiate between the two scenarios.
Figure 8 Reconstructed images of hallucinating subjects.
In the case of healthy subjects, when playing a previously recorded video of the same scenarios, we were able to reconstruct the two scenarios using a deep learning network. Figure 9 displays the reconstructed image of the side face scenario. Although the facial image of the distant collaborator is not highly distinct, it is evident that the subject’s face is not facing forward. The classification of the two scenarios (face or not face subject) can be achieved by utilizing EEG data from HC subjects. The SSIM of Figure 9 is 0.759, indicating that the results obtained from HC subjects are superior to those obtained from individuals with PD-MH.
Figure 9 Reconstructed images of healthy subjects.
After conducting an analysis of EEG and fMRI data, significant findings were observed in the activity of different frequency bands among the three groups of subjects. Specifically, the α and θ bands exhibited enhanced activity, while the power of the Gamma γ band was weakened.
The attentional networks model proposes that CVH in PD patients are associated with impaired activation of the DAN when interpreting ambiguous percepts. This finding aligns with our own results in MH, which indicate that the DAN is weakened in individuals with MH.
The analysis of DTI data shows that MH has a more dispersed arrangement of fiber bundles in the DAN related brain regions compared to HC. The location of abnormal fiber bundles related to DAN provide a basis for choosing EEG channels containing hallucinatory information.
To establish a correlation between abnormal EEG signal and MH, we conducted an study on a PD-MH participant who experienced hallucinations of being stared at during the experiment. The EEG data from this PD-MH participant were unable to differentiate between two scenarios (presence or absence of a face) using visual reconstruction techniques. In contrast, the EEG data from a HC participant successfully distinguished between these two scenarios. This suggests that visual reconstruction techniques may offer a means of evaluating hallucinations.
Future research should explore the relationship between MH and more complex visual hallucinations. Further analysis of MH could potentially facilitate early identification of PD patients who are at risk of developing hallucinations.
In the visual reconstruction experiment, only one subject with PD-MH was included due to the challenges associated with obtaining data on ongoing hallucinations. As a result of several participants’ inability to recall the specifics of their hallucinations, their MH characteristics are documented as unknown in Table 1. The analysis of different types of MH is limited by the small number of patients in each category.
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.
The studies involving humans were approved by the Ethics Committee of Affiliated Brain Hospital of Nanjing Medical University. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any identifiable images or data included in this article.
CL and LQ: study design, data collection, data analysis, and manuscript writing. KY: funding acquisition and supervision. WL: data collection and supervision. QL: data analysis. YC: data collection. JS and CY: manuscript editing. All authors contributed to the article and approved the submitted version.
This work is supported by the National Natural Science Foundation of China(NSFC Grant No. 62293552 and NSFC Grant No. U19B2018), and the Natural Science Foundation of Jiangsu Province of China (Grant No. BK20200161).
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.