Authors: Michael J. Kofler (1Department of Psychology, Florida State University, Tallahassee, FL, USA), Elia F. Soto (2Department of Psychology, Louisiana State University, Baton Rouge, LA, USA), Leah J. Singh (1Department of Psychology, Florida State University, Tallahassee, FL, USA), Sherelle L. Harmon (1Department of Psychology, Florida State University, Tallahassee, FL, USA), Emma Jaisle (3Department of Psychology, Florida International University, Miami, FL, USA), Jessica N. Smith (3Department of Psychology, Florida International University, Miami, FL, USA), Kathleen E. Feeney (3Department of Psychology, Florida International University, Miami, FL, USA), Erica D. Musser (4Department of Psychology, Barnard College, Columbia University, New York, NY, USA)
Categories: Article
Source: Nature reviews psychology
Authors: Michael J. Kofler, Elia F. Soto, Leah J. Singh, Sherelle L. Harmon, Emma Jaisle, Jessica N. Smith, Kathleen E. Feeney, Erica D. Musser
Executive function deficits have been reported in both autism spectrum disorder (ASD) and attention-deficit/hyperactivity disorder (ADHD). However, little is known regarding which, if any, of these impairments are unique vs. shared in children with ADHD versus ASD. In this Review, we provide an overview of the current literature with a critical eye toward diagnostic, measurement, and third-variable considerations that should be leveraged to provide more definitive answers. We conclude that the field’s understanding of ASD and ADHD executive function profiles is highly limited because most research on one disorder has failed to account for their co-occurrence and the presence of symptoms of the other disorder; a vast majority of studies have relied on traditional neuropsychological tests and/or informant-rated executive function scales that have poor specificity and construct validity; and most studies have been unable to account for the well-documented between-person heterogeneity within and across disorders. Currently, the most parsimonious conclusion is that children with ADHD and/or ASD tend to perform moderately worse than neurotypical children on a broad range of neuropsychological tests. However, the extent to which these difficulties are unique vs. shared, or attributable to impairments in specific executive functions subcomponents, remains largely unknown. We end with focused recommendations for future research that we believe will advance this important line of inquiry.
Attention-deficit/hyperactivity disorder (ADHD) and autism spectrum disorder (ASD) are among the most common neurodevelopmental disorders observed in children^1^, as shown by prevalence rates of roughly 5% and 1%, respectively.^2–5^ ADHD is characterized by symptoms of inattention and/or hyperactivity or impulsivity that must be present prior to age 12 years and are associated with impairment. ASD is characterized by persistent deficits in social communication and social interaction and by restrictive, repetitive patterns of behavior, interests, or activities. The rates of co-diagnosis of ASD and ADHD are as high as 70%^6^, and both diagnoses share common clinical characteristics such as onset during childhood, developmental deficits or delays in brain development, and behavioral difficulties and impairments across social and academic domains^1^. However, the nature of these difficulties differs across disorders. For instance, the social difficulties observed in ADHD seem to reflect a performance deficit (such as intrusiveness, inattention to social cues, and impulsive social behavior that result in peer rejection), rather than a lack of social knowledge or skills.^7–9^ By contrast, the social difficulties in ASD seem to reflect deficits in social knowledge^10^ that result in social disengagement, isolation, and indifference to social cues^11,12^ (although the importance of social performance difficulties in ASD is being increasingly recognized^13^).
Deficits in executive function (a set of higher-order neurocognitive processes that enable goal-oriented behavior) have been hypothesized to play key roles in the development and/or maintenance of core behavioral symptoms^14–17^ and assessment of executive function plays a significant role in clinical practice, including early detection and intervention planning.^40^ Clinical researchers typically use cognitive and behavioral models^18–20^ to study shared and unique executive function components within and across diagnostic groups and symptom clusters. An expansive literature in ADHD spanning nearly three decades^29,31–34^ posits that executive function deficits are either causal mechanisms that give rise to ADHD behavioral symptoms; non-causal factors that nonetheless aid in developmental recovery from ADHD; or epiphenomenal (neither causes of ADHD nor involved in symptom expression).^38^ Similarly, the executive dysfunction hypothesis of ASD (one of several etiological theories of ASD) describes how executive function deficits contribute to core ASD diagnostic symptom domains, including disruptions in social communication and increases in restricted interests and repetitive behaviors.^5,35–38^
One of the most empirically supported and influential models of executive function^21,22^— the unity and diversity model^18^— proposes that there are three interrelated but uniquely specific and separable executive function working memory, inhibitory control, and set shifting.^9^ ‘Unity’ in this model refers to correlations between the three components, which are presumed to reflect a common underlying ability, whereas ‘diversity’ acknowledges that the components are also unique and separable.^22^ Developmental studies suggest that executive function abilities are present before three years of age, but specific executive function components are not yet discernible at this age.^23^ Indeed, executive function abilities continue to develop exponentially in early childhood,^24^ with working memory and inhibitory control becoming separable abilities in preschool and early school-aged children,^21,25,26^ and set shifting emerging as a unique ability in late adolescence or early adulthood.^21^ All three executive function components continue to develop and peak in young adulthood (approximately age 25) before plateauing and/or naturally diminishing with age.^24^
Furthermore, the three components in the unity and diversity model support a host of secondary higher-level cognitive processes. For example, deficits in one or more of these executive function components have been implicated theoretically and/or experimentally in functional and behavioral outcomes relevant to ASD and/or ADHD, including difficulties with organizational skills,^27^ planning,^18,27,28^ interference control (the suppression of interference due to resource or stimulus competition),^29^ goal-maintenance,^30^ vigilance,^31^ response consistency,^15,29^ delay tolerance,^32^ academic achievement and success,^33^ learning behaviors such as task engagement and persistence,^34^ social skills,^35^ emotion regulation,^36^ on-task behavior and visual attention,^37^ and self-control and regulation of motor activity.^38^ This evidence provides a clear and compelling rationale for clarifying the unity and diversity of executive dysfunction in ASD and ADHD.^39^
In this Review, we provide the first critical review of studies examining executive function profiles in ADHD vs. ASD based on rigorous methodological criteria informed by the ‘unity and diversity’ model and current best practice recommendations from the cognitive literature. We begin with a non-critical overview of the current evidence supporting and/or refuting executive function deficits in ADHD and ASD. Whereas prior reviews^41,42^ have generally accepted the ‘executive function’ construct labels used by the cited authors, our narrative review builds on prior work by introducing critical conceptual and measurement limitations, as well as construct validity concerns with clinical and neuropsychological executive function tests and behavioral ratings^41,42^. Then, we unpack executive functions to introduce the idea that the overlap in deficit profiles between ADHD and ASD might be due to deficits in different subcomponents that produce similar performances on executive function tests but for different reasons. Despite advancements in the methods and techniques to measure executive function in children, accurate assessment of executive function components remains challenging. Based on insights from what we believe are current best practices for executive function measurement and differential diagnostics, we then critically revisit the literature using a set of rigorous methodological benchmarks. Finally, we conclude with a series of evidence-based recommendations that we hope researchers will use to develop and conduct new studies that provide more definitive answers regarding the unity and diversity of executive function profiles in ASD and ADHD.
In this section, we summarize the evidence supporting or refuting the presence of deficits in the three executive function components (working memory, inhibitory control, and set shifting) and non-specific executive functioning (studies that combine scores from tests intended to measure multiple executive function domains) in children and youth with ADHD (Table 1), ASD (Table 2) and co-occurring diagnoses (Table 3). We prioritized available meta-analytic and systematic reviews in our summary of findings. In this initial overview, we have generally accepted the diagnostic and test construct labels used by the cited authors. We describe the results in terms of effect size, which in this context refers to the estimated magnitude of the impairments for each disorder. A Cohen’s d effect size of 0.20 is considered small (noticeably smaller than medium but not so small as to be trivial), d = 0.50 is considered medium (deficits that are visible to the naked eye of a careful observer) and d = 0.80 is considered large.^43^
Working memory refers to the active, top-down manipulation of information held in short-term memory, including the mental ability to hold, manipulate, and update multiple pieces of information.^16,20^ Working memory is arguably the most common executive function deficit in youth with ADHD.^53^ Deficits on tests on tests intended to measure working memory are consistently among the largest deficits of any executive function component^54^ in studies of youth with ADHD, with some meta-analytic estimates as high as d=0.69–0.74.^55^ Meta-regression estimates reach higher effect sizes (d=2.01–2.05)^55^ when analyses focus specifically on tests that place sufficient demands on the ‘working’ components of working processes that require active monitoring of incoming information and replacing outdated information with relevant information (continuous updating), maintaining information in mind while performing a secondary task (dual-processing), and/or maintaining and rearranging information in mind (or serial and temporal reordering).^55–58^ Meta-analytic estimates of working memory deficits are smaller in preschoolers with ADHD compared to children and adolescents with ADHD (d=0.32).^59^ This result might be due to substantial differences in the tasks administered to preschool-aged youth, which sometimes are simplified versions of the tasks given to children and adolescents, or are research-assistant administered game-like tasks.^60^
Meta-analyses have also consistently identified deficits on tests of working memory among individuals with ASD^61–63^. Studies suggest greater deficits on visuospatial (d>0.72) compared to verbal (d=0.44–0.67) tests.^61–63^ Further, effect sizes are larger among school-aged children (d=0.62) compared to adolescents (d=0.20),^61^ although some meta-analyses report no age effects.^62,63^ Additionally, some limited evidence suggests substantial working memory deficits in preschoolers with ASD compared to neurotypically developing peers.^64^
When ADHD symptoms are controlled for among ASD samples, effect sizes remain medium for verbal (*d=*0.53) and spatial tests (d=0.50).^42,65^ Impairments on tests intended to measure working memory remain notable among youth with ASD when controlling for symptoms of ADHD^65^ and among individuals without co-occurring ADHD.^66 67^ Similarly, these impairments remain notable among youth with ADHD when controlling for symptoms of ASD^68^ and among individuals without co-occurring ASD.^69,70^ Studies that include both ADHD and ASD groups consistently report substantial impairments on tests intended to measure working memory relative to neurotypically developing peers across both groups.^42,68,71^ Evidence suggests greater working memory impairment among individuals with ADHD compared to ASD.^71^ Studies also demonstrate greater impairment in ADHD and ASD co-occurring groups relative to neurotypical peers (d=0.65),^67,69,70^ but similar working memory performance relative to ASD-only groups.^69,70,72,73^
Inhibitory control refers to the ability to withhold or stop an on-going response, particularly within the context of goal-directed behavior.^74^ Youth with ADHD show deficits on tests intended to measure inhibitory control, with medium effect sizes (d=0.52)^54^ compared to neurotypical youth, and preschoolers with elevated ADHD symptoms show small-to-medium deficits overall (d=0.49).^60^ Two of the most commonly used inhibitory control tasks in the ADHD literature are the stop-signal task and go/no-go task, which test response inhibition.^59,74–76^ Studies based on these tasks show medium meta-analytic effect sizes in school-aged youth through adulthood (d=0.49–0.63)^75–77^ and medium effect sizes in preschoolers (d=0.37–0.87).^59^ By contrast, findings related to inhibitory control are generally mixed and meta-analytic effect sizes are null or small-to-medium when interference control tests are used (such as Stroop, flanker and Simon tasks).^59,60,78,79^
Early meta-analyses of inhibitory control deficits among youth with ASD revealed small to medium effect sizes on tasks similar to those used in the study of youth with ADHD (response d=0.55; interference d=0.31)^80^. A meta-analysis identified a small-to-medium effect (d=0.48), with younger children showing more pronounced deficits than adolescents (preschool d=0.72; vs. school-aged d=0.56; vs. adolescence d=0.42).^81^
Youth with either ADHD or ASD demonstrate impaired performance on tests intended to measure inhibitory control compared to neurotypically developing peers.^42,67,68,70^ Children with ADHD continue to show poor inhibitory control test performance after controlling for ASD symptoms.^68^ Similarly, children with ASD continue to show poor inhibitory control test performance after controlling for ADHD symptoms.^65^ When comparing ADHD to ASD groups, some evidence suggested greater inhibitory control impairment among individuals with ADHD compared to ASD,^67,71^ but other findings indicated the two groups exhibit equal levels of impairment.^72^ Co-occurring ADHD and ASD groups demonstrated impaired inhibitory control relative to neurotypically developing peers^67,69–71^ and the ASD-only group.^67,69,71^ By contrast, one review demonstrated comparable inhibitory control test performance among co-occurring ADHD and ASD and ASD-only groups.^72^ There are also mixed findings regarding inhibitory control skills among co-occurring groups relative to ADHD-only groups. Co-occurring ADHD and ASD groups exhibited inhibitory control skills comparable to the ADHD-only groups,^69,70,72^ but one empirical study demonstrated that the co-occurring ADHD and ASD group exhibited better inhibitory control relative to the ADHD-only group.^71^ Further, one review suggested there were no differences in inhibitory control among co-occurring and ADHD-only groups.^72^
Set shifting (also called cognitive flexibility) is defined as the ability to switch flexibly between mental sets.^22^ Set shifting has been understudied compared to working memory and inhibitory control among youth with ADHD, and has been associated with relatively smaller effect sizes (youth d=0.35, preschool d=0.26)^54,60^. However, interpreting these findings is challenging given the evidence that set shifting only develops as a separate, unique ability in late adolescence or early adulthood.^21^
Consistent with this developmental evidence, the impairments on set shifting tests in youth with ADHD might not be due to the tests’ set shifting demands per se, but to the inhibitory control and/or working memory demands required to perform these tests.^83–85^ Set shifting has also been studied less than the other executive function components among youth with ASD. However, empirical work shows that youth with ASD exhibit very large deficits compared to neurotypical children on tests assessing perseverative errors (continuation of same response strategy following a rule change) (d=2.17–2.55)^86^ and small-to-medium-sized difficulties maintaining a new ruleset following a successful initial shift (d=0.46).^66,87^ Additionally, set shifting performance seems to be more impaired among children than adolescents with ASD.^88^
When controlling for ASD symptoms, the evidence is mixed as to whether or not youth with ADHD continue to display deficits on set shifting tests.^68,89^ By contrast, children with ASD continue to show impaired performance on tests intended to measure set shifting compared to neurotypical individuals after controlling for co-occurring ADHD symptoms, with a medium effect size (d=0.61) across tasks.^65^ Youth with ASD also show worse set shifting compared to youth with ADHD^69^ and co-occurring ADHD and ASD;^72^ however, some studies show no group differences^42^ and one study observed better performance among youth with ASD compared to neurotypical and co-occurring ADHD and ASD peers.^71^ Co-occurring ASD and ADHD groups show medium-sized deficits relative to neurotypical peers (d=0.60).^71^
Under ‘non-specific’ executive functioning we review findings that are collapsed across tests of the three executive function components, as well as tests of additional neurocognitive and behavioral processes that are, at least in part, considered to be outcomes of the three core executive functions in the ‘unity and diversity’ framework (such as planning, organizing, and attentional focus) .^22,27,30^ Several large meta-analyses and mega-analyses offer conclusions about the magnitude of non-specific executive function deficits in youth with ADHD and ASD. According to a review of 34 meta-analyses, youth with ADHD exhibit medium-magnitude deficits compared to neurotypical youth on non-specific executive functioning (d=0.45), with larger deficits among children (d≈0.50) than adolescents (d≤0.30). Comparable meta-analytic effect sizes have been identified among preschoolers with ADHD (d=0.32–0.64).^42^ Similarly, according to a meta-analysis of 235 studies, the average non-specific executive functioning deficit in youth with ASD compared to neurotypical youth is in the medium range (d=0.48).^61^
Consistent with patterns reported among neurotypical populations,^90^ estimates of group-level impairments do not necessarily inform between-person heterogeneity in executive function across individual youth with ADHD and ASD. Specifically, although a majority of youth with ADHD (89%) have a deficit in one or more executive function components, individuals differ in which component is impaired (approximately 75–85% of youth with ADHD have impairments in working memory, 21–46% in inhibitory control, and 10–38% in set shifting).^53^ Only 4% of children with ADHD show impairments in all three components, highlighting the limited clinical utility of non-specific measures of executive functioning.^53^ Concerning ASD, approximately half of youth with ASD (47%) show executive function deficits in one or more executive function components.^91^ Thus, not all children with ADHD or ASD have executive function deficits. However, emerging evidence suggests that this executive function heterogeneity may prove fruitful for understanding heterogeneity in functional impairments for children with ADHD and/or ASD.^8,53,91–93^
Taken together, the available literature on school-aged children suggests that ADHD and ASD might be associated with moderate deficits on non-specific indices of executive functioning, and moderate-to-very-large deficits in working memory specifically. Further, ADHD might be associated with greater working memory and inhibitory control deficits compared to ASD (medium-to-very large deficits in ADHD vs. small to medium deficits in ASD), whereas ASD might be associated with greater set-shifting deficits compared to ADHD (medium-to-very-large deficits in ASD vs. small-to-null deficits in ADHD). However, methodological and diagnostic issues call these preliminary conclusions into question and highlight the need for more rigorous research to clarify the unity and diversity of executive function profiles in ASD vs. ADHD.
In this section, we discuss the challenges in drawing conclusions about similarities and differences in executive function profiles in ASD and ADHD from the current literature base. Specifically, we discuss the diagnostic challenges in differentiating between ASD and ADHD and consider the accuracy, precision and comprehensiveness of traditional test batteries.
The extent to which ADHD and ASD are associated with similar executive function impairments is complicated by several diagnostic challenges.^115,116.^ Specifically, there is concern regarding the validity of current gold-standard diagnostic methods to differentiate between ASD and ADHD symptoms. For example, none of the items on the gold-standard Autism Diagnostic Inventory – Revised (ADI-R) adequately differentiate ASD from ADHD.^11,94^ By contrast, select items from the Behavior Assessment System for Children-3 (BASC-3) and the Autism Diagnostic Observation System (ADOS-2) might accurately differentiate ASD from ADHD, although both measures fail to adequately differentiate ADHD from ASD.^11,95^ Differential diagnosis is also challenged by diagnostic overshadowing (the attribution of co-occurring symptoms to a disorder that has already been diagnosed when it is actually indicative of a co-occurring condition).^96^ For example, symptoms of ASD might be misattributed as ADHD symptoms in a child diagnosed with ADHD and vice versa.^96,97^ Such problems with differential diagnoses can reduce the validity of the research base and negatively impact children and their families due to inaccurate diagnoses and delays in implementing appropriate treatments.^97^
Co-occurring conditions and symptom variability pose further challenges. Unfortunately, current methodological and classification approaches are likely not sensitive enough to capture the full range of within-group and between-group symptom variability. For example, intellectual developmental disorder (defined as a standardized intelligence score at least 2 standard deviations below the mean with associated impairment) is more common among individuals with ASD (19–35%) than among the general population (2–3%).^98^ Further, the prevalence of co-occurring ASD and intellectual developmental disorder is 30–40%^98^. Similarly, 8%-39% of children with mild and borderline intellectual developmental disorder have ADHD.^14,41,99^ However, ASD and ADHD studies typically exclude children with low intellectual quotient (IQ) and/or intellectual disability^100^. This is an important limitation because working memory is a (likely causal) predictor of global IQ^101–103^ and age-related improvements in working memory lead directly to improvements in IQ.^103^ Thus, the methodological decision to exclude individuals based on IQ has the unintended consequence of excluding children based on their working memory abilities. This decision inadvertently yields an incomplete picture of the heterogeneity and nature of executive functioning profiles in ASD and ADHD.
Similarly, most ASD samples are limited to children with milder or more subtle symptom presentations who require minimal support (for example, participants meeting criteria for the lowest severity category, ‘Level 1, Requiring Support’), rather than individuals who present with more severe symptoms and require substantial support for daily living. Consequently, executive function results of ASD samples might not generalize across the broader autism spectrum.
An additional limitation stems from the fact that ADHD and ASD are usually diagnosed using nosological frameworks that conceptualize psychological disorders as fundamentally distinct and orthogonal (such as the Diagnostic and Statistical Manual of Mental Disorders)^104^. This categorical approach often fails to adequately capture clinically relevant symptoms that fall outside the diagnostic criteria in complex, heterogeneous, and highly co-occurring diagnoses and thus likely conflates ADHD and ASD between-group differences^104,105^. By contrast, dimensional approaches that conceptualize clinical presentations based on the frequency and severity of broad symptom dimensions (such as the “RDoC” Research Domain Criteria Initiative^95,104^) capture variability within ADHD and ASD and are used in research to differentiate disorder-specific deficits within broad areas of impairment^106–108^. However, dimensional diagnoses have not been linked to reimbursable mental healthcare services and their use in clinical practice is limited. Reconciling categorical and dimensional approaches within the realities of managed care healthcare systems will be critical to avoid further widening the research-to-practice gap.
Interest in executive function and its measurement has grown significantly across diverse fields including clinical science, cognitive science, neuropsychology, and developmental psychology. This piqued interest likely resulted from theory, research and empirical evidence linking executive function deficits to various psychopathologies (including neurodevelopmental disorders) and to adverse functional outcomes in clinical and non-clinical pediatric populations.^19,69,109^ As research and clinical interest grew, there was a corresponding commercial interest that featured a proliferation of executive function tests and measures marketed to clinicians. In some cases, conceptually-derived (rather than empirically-derived) executive function subscales based on pre-existing questionnaire items were added to broadband rating scales based on pre-existing item content. In other cases, performance-based ‘executive function’ tests for children were published that were fully or partially comprised of pre-existing tests originally designed to detect gross neuropsychological and frontal lobe deficits in adults.^110^ For example, the popular digit span test transitioned from a measure of verbal IQ to a measure of freedom from distractibility and is now reified as a test of working memory. Similarly, the trail making test was repurposed from a test of brain injury or gross neuropsychological functioning to a specific test of set shifting.^111^ Finally, new tests and measures were developed psychometrically, but frequently had smaller normative samples and were in most cases not adopted in widespread clinical and clinical-research practice.^112^ Thus, it might be unsurprising that the now-traditional executive function tests most frequently used in clinical practice lack the sensitivity and specificity necessary to capture the global and specific executive function deficits that are characteristic of children with neurodevelopmental disorders.^109^
Specifically, there is a preponderance of evidence questioning the construct validity and test specificity of most of the traditional norm-referenced, performance-based neuropsychological tests of executive function widely used in clinic settings,^29,109^ including the Delis–Kaplan Executive Function System,^113^ the Woodcock-Johnson III Tests of Cognitive Abilities,^114^ the Developmental Neuropsychological Assessment–II,^115^ and executive function-relevant factors from the several editions of the Wechsler intelligence test batteries^116^. Much of the criticism of these tests points to the fact that these measures are too broad in scope, lack specificity to assess executive function components, and were developed to assess gross frontal lobe dysfunction (for example, secondary to traumatic brain injury or in people with dementia) rather than the more subtle executive function deficits associated with psychopathology.^109,117^ Independent evaluations of these test batteries indicate that their subtests contribute meaningfully to a composite measure of global IQ or global neuropsychological functioning (psychometric g)^118^ , but do not provide a valid assessment of executive function components and distinct constructs when compared to well-validated performance-based executive function tasks from the cognitive literature.
For example, a sizeable proportion of the variance in the Delis–Kaplan Executive Function System executive function subtests was attributable to a general factor g rather than to the specific executive function components described in the test’s interpretation manual.^119^ Similarly, a re-analysis of the Developmental Neuropsychological Assessment–II standardization sample indicated that the 23 evaluated subtests do not meaningfully contribute to the assessment of psychometric g, or to the tests’ intended neuropsychological domains (general factor loadings for most subtests were less than .50, and domain-specific effects for all subtests were even lower). All subtests demonstrated strong subtest-specific effects, but it is not clear what constructs these subtest-specific effects represent.^120^
A similar pattern has been found for traditional clinically-used tests of working memory. For example, factor-analytic studies of the Wechsler Intelligence Scale for Children-V^116^ show that up to half (ranging from 24–50% across subtests) of the variance on working memory subtests is attributable to a general psychometric g factor, whereas a minimal proportion of1 variance (less than 3%) is attributable to a working memory factor after accounting for the general factor.^118^ The same issue has been identified with the working memory factor across the rest of the Wechsler intelligence scales using the tests’ original norming samples as well as independent samples.^118,121–127^. These findings indicate that the Wechsler working memory factor “possesses too little true score variance to support clinical interpretation”^123^ and is “not sufficiently reliable for clinical decisions.”^128^ Thus, the working memory factor in these scales cannot be used for the identification of working memory deficits in research or clinical practice.
For research purposes, modifications to the administration and scoring rules might help overcome some of the limitations of these scales. Modifications might include ignoring the rule to discontinue the test administration after several consecutive fails, administering all trials regardless of patient performance, and scoring patient responses using partial credit unit scoring (counting each stimulus recalled in the correct serial position) rather than the traditional all-or-nothing scoring (awarding a point only if the patient’s response was perfect for the complete trial).
A study in a sample of children with ADHD^129^ tested these recommendations for working memory assessment from the cognitive literature.^130^ Consistent with the factor analytic evidence above, the results revealed that traditional scoring of the Wechsler Intelligence Scale for Children-IV digit span backward subtest (a commonly used task assumed to test working memory) failed to predict working memory or achievement and instead showed moderate correspondence with fluid reasoning (general factor g)^131^. However, modifications to the subtest administration and scoring decreased its association with fluid reasoning (from a statistically significant r=.49 to a non-significant r=.15) and substantially increased the magnitude of its associations with latent estimates of working memory, specifically reordering and dual-processing (r=.53-.58) and academic achievement (r=.49).^129^ The results indicated that “digit span backward becomes a valid measure of working memory at exactly the point that testing is traditionally discontinued”.^129^
We and others have also argued that the working memory tests commonly used in clinical practice place relatively minimal demands on the executive components of working memory and thus might be better conceptualized as measures of short term memory .^53,101,130,132,133^ In either case, the construct valid measurement of working memory specifically, and executive functioning more broadly, is currently significantly limited in clinical practice, which has led some cognitive scientists to describe the clinical and neuropsychological literature as engaging in “parallel play” when it comes to executive function measurement.^109^ Other construct validity concerns include administration features and analysis and research considerations that should be taken into account when evaluating the utility of the available neuropsychological tests of working memory (Table 4; Supplementary information)
Informant-rated rating scales are a convenient and cost-effective method for assessing many of the constructs, syndromes, and symptoms encountered in clinical practice and research. The combination of informant-rated executive function scales and performance-based tests has often been considered the gold standard for clinical and neuropsychological assessment of executive function in children and adolescents.^134,135^ However, these two measurement modalities have shown non-significant to weak associations.^131,136–138^ Informant-rated scales only correlate r≈.20 or lower with construct valid, performance-based executive function tests.^131,135,138^ Stated differently, 96% of a person’s executive function abilities are not captured by informant-rated scales (.2^2^ = 4% shared variance between informant-rated and performance-based methods for assessing the same construct)^131^.
Further, methodological and conceptual issues limit the interpretation of informant-rated executive function rating scale scores and the conclusions that can be drawn from them. Several authors have questioned the content validity of informant-rated executive function rating scales and the evidence supporting their construct and predictive validity. For example, some popular informant-rated executive function rating scales have been criticized for being, essentially, recycled ADHD rating scales, with a majority of items on at least some subscales appearing identical, or nearly identical, to DSM-5 ADHD symptom criteria^131,135,138^. Based on the current literature, our conclusion is that informant-rated executive function rating scales cannot be used to assess neurocognitive abilities, and that more work is needed to clarify what these scales are actually measuring (Box 1).
Taken together, our goal of understanding the unity and diversity in executive function profiles across ADHD and ASD is limited by substantial clinical (differential diagnosis), cognitive (accuracy, precision and comprehensiveness of available test batteries), and research (exclusion criteria, dimensional vs. categorical approaches) challenges. Although traditional neuropsychological tests provided a promising starting point by highlighting the importance of executive functions for understanding both ASD and ADHD, converging evidence indicates that they generally do not provide the necessary level of specificity and construct coverage for reliably measuring the more subtle deficits associated with psychopathology.^109^ In the next section, we describe how we believe that leveraging advances in cognitive science can improve understanding of executive functioning profiles in ADHD vs. ASD.
Cognitive and clinical scientists have developed modern performance tests of executive function that are based on models of executive function from the cognitive literature (‘cognitively-informed’).^22,143^ In this section, we discuss these measures and how the results of studies that have used them have begun to improve the field’s understanding of executive function deficits in ADHD and ASD.
In contrast to traditional performance-based neuropsychological tests, modern performance-based executive function tests are supported by cognitive models of executive function.^22,143^ These ‘cognitively-informed’ performance-based tests provide reliable and valid estimates of the executive function components defined in the unity and diversity model^56,109,135^. These tests have also demonstrated ecological validity via robust prediction of important functional outcomes such as academic achievement, social functioning, attentive behavior, and organizational skills.^18,26,27,38,58,131^
The utility of using cognitively-informed measures of executive function for evaluating children with ADHD and ASD is particularly evident when impairment estimates are compared to the estimates yielded by traditional neuropsychological tests. For example, across meta-analyses using traditional neuropsychological executive function tests for children with ADHD, 33%-50% of cases exhibited executive function deficits (30%-37% impaired working memory, 21%-46% impaired inhibitory control).^90,144–149^ By contrast, studies using cognitively-informed measures report that 89% of ADHD cases exhibited impairments in at least one executive function (75–85% impaired working memory, 21–46% impaired inhibitory control, 10–38% impaired set shifting).^53,56^ Indeed, a study using meta-regression techniques^55^ concluded that 98% of children with ADHD score below average or worse on cognitively-informed working memory tests with high demands on the ‘working’ components of working memory. The large increases in impairment rates yielded by cognitively-informed tests are consistent with critiques suggesting that traditional neuropsychological tests often lack sensitivity and specificity for detecting the subtle executive function deficits associated with these disorders.^109^
Similarly, an empirical study using traditional neuropsychological tests suggested that about 47% of children with ASD demonstrate deficits in one or more executive function components .^91^ However, a meta-analysis revealed that these measures generally do not differentiate children with ASD from children without ASD, and concluded that the evidence did not support using these tests to fractionate children’s performance into specific executive function components.^61^ By contrast, meta-analytic work revealed that using cognitively-informed tests of executive functioning greatly improved discrimination in executive function deficits such as working memory^62^ and reaction time parameters^68^ in individuals with ASD compared to a typically developing group and/or a ADHD group. Cognitively-informed executive function tests also demonstrated superior predictive and ecological validity compared to informant-rated scales.^131^ This result was confirmed using informant-rated scales and performance-based scales of functional outcomes such as academic achievement, and was reported using latent variable analysis that would be expected to maximize test-rating correlations by removing error.
We echo recent recommendations^109^ to employ multiple, cognitively-informed measures of each executive function component – ideally assessed across multiple sessions on separate testing days – to maximize construct validity and yield more accurate impairment estimates relative to traditional neuropsychological batteries^109^. Together with latent estimation that models both unique and shared variance across the executive function components,^22,26,58,150^ these methodological refinements are expected to substantially increase the specificity and sensitivity of the scores produced by these modern cognitively-informed tests. However, the clinical utility of these tests remains limited because, with few exceptions,^151^ they lack the large, nationally representative normative samples needed to draw conclusions about individual patients. In addition, careful attention to these tests’ outcome metrics will be critical. For example, the stop-signal test is often considered the gold standard for inhibition measurement but produces fictitious inhibitory deficits if scored using the traditional method (Box 2).
Similar to how executive function can be fractionated into three primary components, the three primary components seem to be separable into subprocesses. For example, working memory can be divided into its ‘working’ component (the mental processes that operate on mentally held information) and ‘memory’ component (short-term memory, the passive storage and rehearsal mechanisms that temporarily hold information in mind). Further, the ‘working’ component can be fractionated into interrelated but distinguishable subprocesses involving continuous updating, dual-processing and serial-temporal reordering, and the ‘memory’ component can be fractionated into distinct verbal, visual and spatial short-term storage systems (Figure 1).^20,56,152^ Similar subdivisions are also apparent for inhibitory control and set shifting. Using specifically designed test batteries that enable performance to be fractionated into the three primary executive function components and their specific subcomponents will be imperative for better understanding the nuances of executive function strengths and difficulties in ADHD and ASD.
To date, research investigating executive function subcomponents in ADHD and/or ASD has been scarce. Thus, even if research confirmed that ASD and/or ADHD are associated with deficits in a specific executive function component, it would remain unclear whether these deficits are due to the same subprocesses. For example, deficits on inhibition tests might be related to perseverative processes in children with ASD that, in turn, predict engagement in restrictive or repetitive behaviors. By contrast, the same overall test scores in children with ADHD might be related to action-cancellation processes (stopping an in-progress behavior) that, in turn, predict impulsive or verbally intrusive behaviors.^153^ Similarly, a conclusion that ADHD or ASD is not associated with deficits in a specific executive function component might be premature if overall null findings are due to strengths in some subprocesses that mask deficits in other subprocesses.
A notable exception to this critique is a study that used a specifically designed test battery to evaluate the three subprocesses of ‘working’ component of working memory in children with ADHD^56^. Compared to children without ADHD, children with ADHD exhibited large impairments in serial/temporal reordering (d=1.34) and medium-sized impairments in continuous updating (d=0.64), but generally intact dual-processing working memory. This initial study highlights the importance of construct specificity. However, additional analyses also showed that what is shared between these three subprocesses—rather than their unique features—is critical for predicting ADHD symptoms. Thus, careful attention to both the unity and diversity within and across executive function components is needed to advance research in ADHD and ASD.
In this section, we revisit the available evidence base to critically review the studies examining executive function profiles in ADHD and ASD. To that end, we developed rigorous methodological criteria derived from our examination of the limitations of available reports and what we believe to be current best practices from the cognitive (executive function measurement) and clinical (differential diagnostics) literatures. We used three primary criteria to re-review the available literature. First, studies should include both ADHD and ASD samples. Second, studies should describe differential diagnostic methods suggesting reasonable certainty regarding the labeling of comparison groups as ADHD, ASD, co-occurring ADHD and ASD, or neurotypical. In the case of studies using a dimensional approach, construct-valid symptom assessments should be used. Third, studies should include valid cognitively-informed measurements of one or more executive function components (Table 5). Additional criteria including issues of representativeness and generalizability were also considered and impacted the level of certainty/strength of our conclusions and will be further discussed in the following section.
We were able to locate several executive function studies that included both ADHD and ASD samples and provided sufficient diagnostic details to suggest reasonable certainty regarding their clinical groups. However, almost all of the current literature relied on traditional neuropsychological tests that have been criticized for poor construct validity.^109^ Specifically, no study to date fully met our benchmarks regarding construct-valid working memory or set shifting measurement. Our conclusion that most extant ADHD and ASD executive function studies failed to meet methodological quality benchmarks to allow firm conclusions is consistent with a recent review^72^ that judged every extant co-occurring ADHD and ASD study as methodologically poor or fair (none received a rating of strong).
A partial exception to this conclusion is a study that explored a computationally derived ex-Gaussian index of response inconsistency (called tau)^154^, which has been shown to be a causally linked outcome of working memory but not inhibitory control^15,29,155^. In this study, children with ADHD and co-occurring ADHD and ASD demonstrated elevated tau relative to both children with ASD (without ADHD) and neurotypical children, and ADHD but not ASD uniquely predicted tau. Although concluding that working memory is implicated in ADHD but not ASD is arguably a stretch because tau is not solely a reflection of working memory^15,156^, this result speaks to the lack of robust evidence regarding unique vs. overlapping working memory profiles in ASD vs. ADHD.
Similarly, a series of studies by some of the authors of the current Review included children with ADHD and ASD, controlled for ASD when evaluating executive functions in ADHD, and used a battery of construct-valid working memory tests. Results indicated that children with ADHD have large working memory deficits and medium-to-null inhibitory control and set shifting deficits relative to children without ADHD.^51,53,56,58,75^ Interestingly, however, poor performance on inhibitory control tests was attributable, in large part, to the tests’ working memory demands rather reflecting actual inhibition deficits in children with ADHD.^31,152,153^ These studies also showed that working memory but not inhibition deficits predict ADHD-related difficulties with emotion regulation, academic achievement and productivity, organizational skills, activities of daily living, inattentive and hyperactive/impulsive symptom severity, information processing speed, and peer relationships.^8,27,31,36,37,58,156,158–161^ However, children with ASD comprised only about 10% of our ADHD and non-ADHD samples, and methodological control for ASD was limited to including an equal number of ASD cases in both the ADHD and non-ADHD comparison groups and conducting sensitivity analyses (comparing results when including versus excluding children with ASD). Thus, although these studies provide preliminary evidence for working memory deficits as a key, likely causal, factor in ADHD, they do not provide data on executive functioning for children with ASD.
By contrast, there is some, albeit still limited and mixed, evidence suggesting that ADHD might be more strongly associated with difficulties on tests of the action restraint (preventing a behavior before it starts) component of inhibitory control than ASD.^42,69,71,72,82^ However, these findings are preliminary because they are based on relatively small samples and because none of the available studies also included construct-valid tests of working memory and/or set shifting. This latter point is important because the three executive function components are moderately interrelated, and there is experimental evidence demonstrating that working memory impacts performance on inhibitory control tests (but not vice versa), which could potentially explain why children with ADHD show deficits on inhibition tests.^152,157–159^ Indeed, studying any executive function in isolation limits certainty because it is unclear whether the observed deficits are specific to the tests’ inhibitory control demands or to the myriad of other executive, neurocognitive, motor, or perceptual processes required for successful performance.^53,83,117^
The current literature indicates a substantial problem with task impurity and emphasizes the need for viable solutions.^109,165^ We recommend a set of study methods we hope researchers will adopt as guidelines for developing and conducting new studies to provide more definitive answers regarding the unity and diversity of executive function profiles in ADHD and ASD.
Our first recommendation is to use construct-valid, performance-based executive function batteries. Using multi-test approaches for each construct of interest will be critical for identifying the unique and overlapping executive functioning weaknesses – and potentially strengths – associated with these neurodevelopmental conditions. Stated bluntly, it is tenuous — if not scientifically indefensible — to reify any single test as measuring any single executive function.^53,150^ Test combinations should be selected to isolate executive function components and subcomponents, and performance should be assessed across multiple sessions on separate testing days – ideally at different times of day – to increase the specificity and reliability of executive function scores, to reduce participant fatigue, and to account for random and time-of-day effects.^130,166,167^
Our second recommendation is to carefully attend to differential diagnostics and consider dimensional assessment. ASD is often viewed as qualitatively different from other neurodevelopmental and clinical disorders. However, replicated evidence supports the view that conditions such as ASD^168,169^ and ADHD^170^ are extreme ends along natural continuums of characteristics that are normally distributed across the general population. Relatedly, the majority of existing studies rely on diagnostic status to define and compare ADHD and ASD groups, which limits understanding of symptom overlap and heterogeneity within and across these conditions. Using dimensional approaches to conceptualize these two conditions will more fully capture the underlying neurocognitive profiles and etiologies of these symptom dimensions and potentially link specific symptom clusters with specific executive function vulnerabilities.^171^ Of course, studies using categorical approaches will also be valuable. For these types of studies, clear consideration and communication of differential diagnostic challenges and clinical decision-making will be important for readers to assess the integrity of the grouping variable and the generalizability of the findings.
Our third recommendation is to assess and model unique and overlapping aspects of both syndromes. Most ADHD and ASD studies examined executive functioning in one disorder without consideration of the other, and even fewer studies included a co-occurring ADHD and ASD group. More research is needed to examine similarities and differences in executive functioning abilities across these two disorders by including ADHD, ASD, and co-occurring ADHD and ASD samples (as well as neurotypical children) in studies to capture the full range of symptom quantity, frequency, and severity for each ASD and ADHD symptom cluster and functional outcome. Capturing the full range of symptom severity and impairment is particularly important given that most studies to date have recruited only relatively high functioning children with ASD.
Our fourth recommendation is to reconsider IQ exclusionary criteria, and to not covary IQ in data analyses. Most reviewed studies set exclusion criteria based on IQ, which typically excludes children with borderline intellectual disability and intellectual developmental disorder. This methodological decision results in an incomplete picture of the heterogeneity of executive function profiles in ASD and ADHD. Further, it is important not to include IQ scores as a covariate in statistical analyses of neurodevelopmental disorders (see ref^172^ for a compelling statistical, methodological, and conceptual rationale). Researchers should reconsider study criteria and exclusionary cut points for intellectual functioning and instead build an executive function test battery based on the expected mental age of study participants.
Our fifth recommendation is to consider third-variable explanations. If overlapping executive function deficits are identified, additional research will benefit from determining whether this overlap is due to shared deficits between ASD and ADHD, to demographic similarities, and/or the increased risk that each condition carries for additional co-occurring syndromes (Box 3).
Our sixth recommendation is to increase access to research studies and improve generalizability of research findings. Understanding differential outcomes based on race, ethnicity, gender, sex, age, and socioeconomic factors^173,174^ is highly limited. Although efforts have been made to incorporate more diverse samples, most existing research has studied children, specifically boys. This bias is likely driven by the higher prevalence of ADHD and ASD diagnoses in school-aged boys versus girls.^175,176^ Boys also present with earlier onset and more serve symptoms, making them more likely to be identified as having ADHD and/or ASD at a younger age than girls.^177,178^ Given this bias, it is difficult to parse apart the extent to which sex differences in executive functions might be present in school-aged children with ADHD and/or ASD. More broadly, like most areas of psychological inquiry, ADHD and ASD research has historically been conducted primarily with White Non-Hispanic children from western, educated, industrialized, rich, and democratic (‘WEIRD’) societies.^179,180^ In light of continued inequalities regarding access to research studies, we recommend active strategies to optimize participation by diverse populations,^181^ with a focus on traditionally underserved populations to assess and reflect a broader range of experiences.^182,183^
Given the substantial conceptual and construct validity issues discussed in our Review, firm conclusions regarding the unity and diversity of executive function profiles in children with ADHD vs ASD are not warranted at this time. Instead, the most parsimonious conclusion is that children with ADHD and/or ASD tend to perform moderately worse than neurotypical children on a broad range of performance-based neuropsychological tests that likely place at least some demands on executive functions.^109,117^ However, the extent to which these deficits are attributable to impairments in executive function components and subcomponents remains largely unknown. The unfortunate consequence is that there is currently very little knowledge about specific strengths and weaknesses in executive functioning within and across ASD and ADHD.
Future work guided by the methodological considerations described herein holds considerable promise for improving our understanding of the neurocognitive causes, outcomes, and sequelae of these neurodevelopmental disorders. Specifically, future studies would benefit from using cognitively-informed, construct-valid executive function tests and symptom measures; adopting dimensional approaches to capture the full range of symptom frequency, quantity and severity for each symptom cluster and functional outcome of interest; improving sampling strategies and access to clinical research services for populations that have been traditionally excluded from these types of studies; and considering third-variable explanations for any detected overlap in executive function profiles. Work guided by these methodological considerations holds considerable promise for improving the field’s understanding of the neurocognitive causes, outcomes, and sequelae of ADHD and ASD. Clinicians might also be interested in maximizing the limited utility of current, commercially available executive function tests (Box 4).
At the same time, the nature and structure of executive function remains actively debated in the cognitive literature.^184^ Despite emerging experimental and longitudinal studies providing functional (and probably causal) evidence linking cognitive/behavioral models of executive function with important behavioral and functional outcomes in ADHD and ASD,^33,37,92,158–161^ alternate executive function conceptualizations and measurement approaches (such as neurobiological and sociocultural insights) are clearly warranted. We have taken the position that the improved sensitivity and specificity of cognitively-informed performance tests for differentiating ADHD and ASD from control groups will help future work to more definitively disentangle shared vs. unique neurocognitive deficits in these disorders. However, we acknowledge that bigger effect sizes are not inherently better. Ultimately, the utility of adopting cognitive or behavioral (or any other) models of executive function lies in their usefulness and the extent to which they help clinicians understand – and communicate to parents, teachers, and other stakeholders – why these children exhibit challenging behaviors.