Eye-tracking in adult protocol for a systematic review and meta-analysis

Authors: Blake Noyes, Aleks Biorac, Gustavo Vazquez, Sarosh Khalid-Khan, Douglas Munoz, Linda Booij

Abstract

Introduction

In recent years, eye-tracking has been proposed as a promising tool to identify potential biomarkers for mental disorders, including major depression. We will conduct an updated systematic review and meta-analysis on eye-tracking research in adults with major depressive disorder or other clinically diagnosed depressive disorders.

Methods and analysis

This protocol follows all reporting items in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Protocol extension. We will conduct a systematic search of PubMed, PsycINFO, Google Scholar and EMBASE for sources published up until March 2023. Abstract and full-text review will be completed independently by two reviewers. Non-randomised studies using eye movement tasks in individuals with a depressive disorder versus controls will be included. Eye movement tasks of interest include, but are not limited to, saccade, smooth pursuit, fixation, free-viewing, attentional disengagement, visual search and attentional blink tasks. Results will be categorised by eye movement task. Risk of bias will be assessed using the National Institutes of Health Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies and confidence in cumulative evidence will be assessed using Grading of Recommendations, Assessment, Development and Evaluation criteria.

Ethics and dissemination

Ethics approval is not required due to the nature of the proposed analysis. Results will be disseminated through a journal article, conference presentations and/or dissertations.

Introduction

Major depressive disorder (MDD) is the leading cause of disability worldwide.1 According to the most recent edition of the Diagnostic Statistical Manual of Mental Disorders (DSM-5), MDD is characterised by having low mood and/or low interest/pleasure all day, nearly every day for at least 2 weeks, as well as having a total of five out of nine listed symptoms.2 Similarly, in the latest edition of the International Classification of Diseases (ICD-11), a depressive episode requires a total of 5 out of 10 listed symptoms to be present, with one being either low mood or low interest/pleasure.3 The presence of MDD has been found to significantly impact life satisfaction and functioning, with associations to marital dissatisfaction, lower income, missed work and cognitive impairment,4 as well as increased risk for comorbid mental disorders.5 In comparison to single major depressive episodes, recurrent depression is associated with impaired psychosocial functioning in adulthood, including lower positive and higher negative emotionality, more antisocial behaviour, increased use of mental health services and increased suicidal ideation.6

Despite continued efforts to identify biomarkers (eg, candidate genes, blood-based markers, structural or functional neuroimaging markers) for depression, research has been unsuccessful at identifying a reliable biomarker to integrate into clinical practice.7 However, the growing field of eye-tracking in mental disorders may be able to address this issue. Unlike other methods, such as neuroimaging, eye-tracking is comparatively cost-effective and non-invasive, making it an accessible and attractive option for researchers, study participants and patients. Eye movement behaviour, such as saccades or pupil responses, has been previously investigated and linked to specific brain structures and networks, and therefore, may be used to identify impaired functioning in specific brain areas and networks.8–12 For example, eye movement tasks can be designed to target areas responsible for motor control, cognitive processes (eg, inhibitory control) and emotion.13 14 In mental disorders such as depression, eye-tracking has emerged as a useful tool to understand alterations in cognitive and emotional processes.15 Thus, eye-tracking is a promising method to improve psychological models of disorders, study progression of illness and identify neural markers, which may lead to improved interventions and early identification in at-risk individuals.

A variety of tasks or paradigms have been used in eye-tracking research to probe specific brain functions. Commonly used are saccade tasks, which require participants to make eye movements to different areas on a computer monitor. Variations of saccade tasks include the pro-saccade (requiring participants to look towards a visible target), anti-saccade (requiring participants to look away from a visible target), predictive/anticipatory saccade (requiring participants to make a saccade to a location before a target appears) and memory-guided saccade (requiring participants to make a saccade to the remembered position of a target after it has disappeared). Fixation tasks (requiring participants to keep their gaze on one target) and smooth pursuit tasks (requiring participants to keep their eyes on a moving target) have also been employed. Finally, unstructured free-viewing tasks (allowing participants to look wherever they please) of varying stimuli, attentional disengagement tasks and visual search tasks have become increasingly popular in eye-tracking research. Multiple types of outcomes can be examined from these tasks, including saccade metrics (eg, saccadic reaction time), pupil metrics (eg, baseline pupil size), blink metrics (eg, blink rate) and performance metrics (eg, error rate).

Eye-tracking research in clinical populations, including in individuals with MDD, has grown substantially in the last decade (figure 1). To the authors’ knowledge, only three reviews exist so far on eye-tracking in depression. Two of these are systematic reviews and meta-analyses on free-viewing tasks in depression—Suslow et al,14 which included 16 studies, and Armstrong and Olatunji,16 which included nine studies. The third review is a systematic review on various eye movement tasks in depression, which included 43 studies, and had a broader scope.15

In their review, Carvalho et al15 identified several differences in eye movement behaviour between individuals with MDD and controls. In comparison to controls, those with MDD had increased reaction time during the pro-saccade and anti-saccade tasks, decreased accuracy in the pro-saccade task, and increased error rate during the anti-saccade task. They also had decreased accuracy during the predictive saccade task, decreased reaction time during the memory-guided saccade task, and lower pursuit gain and greater catch-up-saccade rates in smooth pursuit tasks.15 For free-viewing and visual search tasks, those with MDD, as well as those with bipolar disorder in depressive phase, exhibited increased attention to negative (anger, dysphoric, sadness, anxiety related, depression related) stimuli and decreased attention to positive stimuli.15 Finally, the authors reviewed the effects of medication (such as benzodiazepines, anti-depressants and anti-psychotic drugs) on oculomotor behaviour, and noted the need to control for medication when examining eye movements in depression and bipolar disorder.15

In their meta-analyses, Armstrong and Olatunji16 and Suslow et al14 quantified differences between eye-tracking of attention in those with MDD compared with healthy individuals. Suslow et al found no group differences in initial orientation of attention but found moderate to large effect sizes when looking at maintenance of gaze to various types of stimuli during free viewing of images. Across studies, individuals with depression looked less at positive images (Hedge’s g=−0.51) and happy faces (g=−0.54) and longer for dysphoric images (g=0.66) and sad faces (g=0.58) than healthy individuals.14 The previous meta-analysis by Armstrong and Olatunji16 reported that, for orientation, there was no difference between depressed individuals and healthy participants for threatening (g=−0.01) or dysphoric images (g=0.18). However, they found individuals with depression oriented less towards positive stimuli (g=−0.24).16 For maintenance of gaze, there was again no difference for threatening stimuli (g=0.08), but an increase in gaze maintenance for depressed individuals for dysphoric stimuli (g=0.46) and a decrease in positive stimuli (g=−0.80) compared with healthy participants.16

Although much of the research in this area includes the use of one patient group in comparison to controls, some studies have looked at performance on eye-tracking tasks in relation to differential features of depression. For example, non-melancholic depressed patients have been found to perform more similarly to control participants than melancholic depressed patients for most saccade tasks.17 Sears et al18 found that while currently dysphoric participants exhibited initial attention orientation biases for both depression-related and positive images, previously depressed participants only exhibited a bias for depression-related images. This pattern of attention to happy faces was also identified by Isaac et al,19 where currently depressed individuals looked at happy faces less than healthy controls, but there was no difference between individuals with remitted depression and healthy controls in looking time. Severity of depressive scores has also correlated with more abnormal performance on their free-viewing task.20 Different symptom groupings of depression, patient history and severity of depression have all been found to impact eye-tracking measures, and thus need to be considered when synthesising results.

Objectives

The proposed systematic review and meta-analysis will discuss and quantify results from eye-tracking literature with a focus on adults with MDD in comparison to healthy controls, as well as other clinically rated depressive disorders (ie, persistent depressive disorder/dysthymia, dysphoria, bipolar disorder in depressive phase, disruptive mood dysregulation disorder, premenstrual dysphoric disorder, seasonal affective disorder, minor/subclinical/subthreshold depression). Although excluded in prior literature, inclusion of conditions such as subthreshold depression in research is warranted due to the clinical similarity to MDD.21 We will include data from saccade, smooth pursuit, fixation, free-viewing, attentional disengagement and visual search tasks. Additionally, we will cover tasks not previously examined by other reviews (eg, attentional blink task). Eye-tracking outcomes include, but are not limited to, saccades (number, rate, types), saccadic amplitude and velocity, reaction time, error rate and accuracy (qualitative or quantitative), eye position, fixation count and fixation duration, blinks, and pupil dilation/constriction. As the last systematic review on many of these eye movement tasks was published in 2015, an update on adult eye-tracking in depression is necessary given the growth in literature over the past 8 years (see figure 1). To the authors’ knowledge, this will also be the first meta-analysis to include saccade, smooth pursuit and fixation tasks, as well as blink and pupil measures, which is important as these tasks implicate different neural networks. As a result, this paper may aid in identifying consistent eye movement differences in depression, which may advance the research of biomarker discovery.

Method

This protocol covers all items outlined under the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) Protocols.22

Eligibility criteria

Types of studies

Experimental studies performed in a controlled laboratory environment will be included. Any experimental sessions, including longitudinal work, that include eye-tracking measures and eye movement outcomes will be included. Case reports and case series will be excluded.

Types of participants

We will review studies involving study participants with a mean age between 18 and 60 years. Participants must have a current or remitted primary depressive diagnosis, confirmed via a standardised diagnostic instrument or clinical judgement (not reliant on self-report measures). Included diagnoses will be MDD, persistent depressive disorder/dysthymia, dysphoria, disruptive mood dysregulation disorder, premenstrual dysphoric disorder, seasonal affective disorder, and minor/subclinical/subthreshold depression, according to established criteria of the Diagnostic and Statistical Manual of Mental Disorders (DSM; any version) or of the ICD (any version). Studies involving individuals with diagnosed bipolar disorder in depressive phase will also be included, and studies involving mixed samples of bipolar phases will be included if depressive phase data can be isolated. Studies will not be included if they use experimental mood induction procedures in otherwise healthy participants.

Type of control

Studies must also have a control participant group with a mean age between 18 and 60 years. Though screened controls (no history of mental disorders or psychiatric medication usage) would be preferred, we will also include studies that do not screen their controls and take this into consideration on analysis and discussion of results.

Types of interventions

The tasks/measures reviewed by Carvalho et al15 will be pro-saccades, anti-saccades, predictive saccades, memory-guided saccades, smooth pursuit eye movements, fixation tasks and free-viewing/attentional disengagement/search tasks, as well as the attentional blink task (see table 1 for definitions). Additional tasks that are found not to fit in these categories will be included if sufficient data exist. Tasks must have performance (eg, reaction time, error rate) saccade, pupil or blink data to be included in analyses.

Type of outcomes

Eye movement outcomes will include, but are not limited

Performance metrics

Accuracy (qualitative or quantitative), correction factors, error rates, reaction time, response search score.

Saccade/fixation metrics

Saccade type (back-up, catch-up, corrective, intrusive micro, macro), saccade amplitude, saccade count/rate, saccadic duration, saccadic peak velocity*, saccade position error, intersaccadic interval, direction of initial eye movement, eye scanning length (mean, total), pursuit (gain, latency), first/initial fixation (location, duration, latency), fixation bias, fixation duration, fixation frequencies, fixation number, percentage of time fixating, scan path length, maintenance of gaze, mean gaze duration, final eye position, root mean square, root mean square error, square wave jerk, time-weight average gain, target waveform.

Pupil metrics

Baseline pupil size, pupil constriction, pupil dilation.

Blink metrics

Blink duration, blink rate, inter blink interval.

*Because of the saccade main sequence relationship, we will scrutinise any velocity comparisons that are not explicitly amplitude matched.

See table 2 for an overview of patient, intervention, comparison, outcome and study design (PICOs) criteria for inclusion.

Information sources

Journal articles in English, French, German, Dutch and Portuguese published online up until March 2023 will be reviewed. Included studies will not be limited by geographical location. The electronic databases searched will include PubMed, PsycINFO, EMBASE and Google Scholar. Reference lists of highly relevant articles will also be examined. If full text of papers cannot be accessed, attempts to contact authors will be made. Conference papers, posters, abstracts and student theses/dissertations will not be included.

Search strategy

The following is the proposed search strategy for PubMed. All searches will be completed for title/abstract and filtered for journal articles and full text availability. The same search strategy will be repeated across databases with minor changes to adjust for database search tools.

Search terms

• depress* OR mood AND eye tracking OR saccad* OR pupil OR blink OR smooth pursuit
• dysthymi* AND eye tracking OR saccad* OR pupil OR blink OR smooth pursuit
• dysphori* AND eye tracking OR saccad* OR pupil OR blink OR smooth pursuit
• seasonal affective disorder AND eye tracking OR saccad* OR pupil OR blink OR smooth pursuit
• bipolar AND eye tracking OR saccad* OR pupil OR blink OR smooth pursuit

Search strategies for all databases were determined via consensus among the authors.

Study records

Data management

Excel spreadsheets will be used to manage records at the level of abstract screening and full text review. BN will perform the initial search and copy relevant information (authors, publication year, title, link to access) from all search hits and will send the completed spreadsheet to AB.

Selection process

BN and AB will extract abstracts, screen them for eligibility and conduct the full-text review independently. Kappa will be used to quantify the level of inter-rater agreement. DM and LB will oversee the selection of articles. DM will provide input on the eye-tracking (eg, evaluation of methodology), and LB will provide input on the clinical aspects of the studies. In the event of discrepancies at abstract or full text review, DM will make the decision for discrepancies involving eye-tracking methodology, and LB for discrepancies involving participants and all other issues.

Data collection process

Once included studies have been determined, RevMan will be used for data extraction and analysis. Data will be extracted by two independent reviewers (BN and AB) in duplicate. Data tables will be independently piloted with 10 papers to ensure all necessary information is being extracted.

Data items

The following data items will be extracted for saccade and smooth pursuit

N (by diagnosis)

Mean age and SD

Sex/gender

Ethnicity

Comorbidities

Diagnostic method

Interventions/treatment

Eye movement tasks

Eye movement outcomes

Results

The same data items will be extracted for free-viewing, attentional disengagement and visual search tasks, with the addition of stimuli (type and emotional valence).

Outcomes and prioritisation

For a list of eye movement outcomes, see the Eligibility criteria section. The following outcomes will be prioritised due to their frequency in the task accuracy, error rate, fixation duration/maintenance of gaze and reaction time. Additionally, we will prioritise pupil and blink outcomes as those have emerged as interesting metrics in our laboratory.

Risk of bias in individual studies

We will use the National Institutes of Health Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies to assess the risk of bias for each study. Bias will be assessed by two independent researchers (BN and AB) and discrepancies will be addressed by LB and DM. This information will not be used to exclude studies but will be reported and narratively discussed.

Data synthesis

The PRISMA flow chart will be presented to explain how and the number of articles that were retrieved/selected. Results from included individual studies will be categorised by eye movement task and expressed both narratively and in data tables. Means, SD, effect sizes and statistical significance will be reported. It is anticipated that effect sizes will be expressed as Cohen’s D. A meta-analysis will be performed if data from three or more studies exist for the same eye movement task. Pending the final study sample, we will also analyze data based on subgroups if three or more studies are available. The proposed subgroups include differential diagnoses (eg, MDD vs dysthymia), severity of depressive episode (mild, moderate, severe) and current versus remitted depression. It is anticipated that JASP will be used to do the meta-analyses, though other software will be considered at the time of analyses. To assess whether the results are homogenous, the Q statistic will be calculated.23 If quantitative synthesis is not appropriate for a subset of data, results will be described narratively.

Meta-biases

We will investigate in greater detail multiple publications coming out from the same research group and we will assess sample size and sample demographics to check for duplication. In case of overlapping samples/results, the first published reference will be included in the analyses, unless methodology is not sufficiently detailed to be included. Publication bias will be assessed using a funnel plot method.24

Confidence in cumulative evidence

BN and AB will independently assess the strength and quality of evidence using Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach criteria, a tool for meta-analyses that assesses study limitations, inconsistency of results, indirectness of evidence, imprecision and reporting bias.25 Strength of evidence will be rated as very low, low, moderate or high. In the event of discrepancies on GRADE rating, LB and DM will make the final decision. A quality assessment table will be presented to display which studies were included and which were excluded.

Ethics and dissemination

Ethics approval is not required due to the nature of this work. Results will be analysed and disseminated according to PRISMA guidelines for systematic review and meta-analyses. We aim to write a scientific publication about the findings. We may also present findings in the form of a poster or oral presentation at one or more scientific conferences.

Patient and public involvement

Due to the nature of this study, there will be no patient or public involvement.

Discussion

There is a growing need to identify reliable biomarkers of mental disorders, specifically as they relate to depressive disorders, to improve early diagnosis and treatment.26 Research into potential eye movement differences using eye-tracking has grown steadily in recent years, as this method offers the ability to use a variety of different task paradigms and is relatively cost-effective. Given the surge in eye-tracking research in depression, it is paramount to summarise these findings to effectively identify potential behavioural biomarkers.

This meta-analysis aims to update the knowledge that has been reported previously by past meta-analyses on attentional tasks in depression,14 16 provide the first meta-analyses of saccade, fixation and smooth pursuit tasks in depression, as well as to extend the body of knowledge by including task paradigms and outcomes not previously covered. In doing so, we hope to provide a comprehensive overview of current study findings which may help inform future clinical practice in the diagnosis and treatment of depression (and other related disorders). However, while this meta-analysis will attempt to extend the body of knowledge by considering studies using tasks not previously covered by other reviews (eg, attentional blink task) and those including various depressive disorder populations (ie, not simply individuals diagnosed with MDD), this may be limited by the number of published studies and their quality.

Supplementary Material