Discriminative Power of Seoul Cognitive Status Test in Differentiating Subjective Cognitive Decline, Amnestic Mild Cognitive Impairment, and Dementia Based on CERAD-K Standards

Authors: Hasom Moon, Eek-Sung Lee, Seunghee Na, Dayeong An, Joon Soo Shin, Duk L. Na, Hyemin Jang

Abstract

Background and Purpose

We developed a new digital cognitive assessment called Seoul Cognitive Status Test (SCST), formerly called Inbrain Cognitive Screening Test. The purpose of this study was to validate the clinical utility of the SCST by comparing its scores of those with subjective cognitive decline (SCD), amnestic mild cognitive impairment (aMCI), and dementia diagnosed by the Korean version of the Consortium to Establish a Registry for Alzheimer’s Disease Assessment Packet (CERAD-K).

Methods

All participants (n=296) who completed the CERAD-K, SCST, and Instrumental Activities of Daily Living tests were included in this study. Total score, cognitive domain scores, and subtest scores of the SCST were compared among the 3 groups (SCD, aMCI, and dementia). Additionally, correlations between SCST and CERAD-K subtests were examined.

Results

Cognitive domain scores and total score of the SCST showed significant differences among the three groups, with scores being the highest in the order of SCD, aMCI, and dementia (p<0.001). Most subtests of the SCST also showed higher scores in the order of SCD, aMCI, and dementia (p<0.001). However, SCD and aMCI groups showed no significant differences in scores of the Phonemic Word Fluency Test (p=0.083) or Korean Trail Making Test-Elderly version Part A (p=0.434). Additionally, there was no significant difference in the score of Place Recognition (p=0.274) of the Word-Place Association Test between aMCI and dementia groups.

Conclusions

In conclusion, differences in total score, cognitive domain scores, and subtest scores of the SCST among the 3 groups of participants diagnosed using CERAD-K confirm the clinical utility of the SCST for cognitive assessment.

INTRODUCTION

According to international epidemiological reports,1 the number of dementia patients was estimated to be approximately 55 million in 2020. It is expected to reach approximately 140 million by 2050. This indicates that the prevalence of dementia is increased by 3% annually. In South Korea, domestic epidemiological surveys have reported an even higher increase rate of 3.8%.2 To address this issue, early diagnosis and timely intervention are crucial. Indeed, the recent advent of anti-amyloid antibody therapy has further highlighted the importance of early detection of the elderly at risk of dementia.

Cognitive tests play a crucial role in the early detection of dementia. These tests can be broadly categorized into screening tests and comprehensive tests. Screening tests such as the Mini-Mental State Examination (MMSE) and Montreal Cognitive Assessment (MoCA) are easy and quick to administer. However, their sensitivity is low for detecting mild cognitive impairment. On the other hand, comprehensive neuropsychological tests are sensitive in detecting cognitive decline in early stages of dementia. In South Korea, commonly used comprehensive neuropsychological tests include the Korean version of the Consortium to Establish a Registry for Alzheimer’s Disease Assessment Packet (CERAD-K),3 the Seoul Neuropsychological Screening Battery 2nd edition (SNSB-II),4 and Literacy Independent Cognitive Assessment (LICA). Although these paper-pencil-based tests provide detailed information including scores of such cognitive domain as attention, language, visuospatial function, memory, and frontal/executive function, they require a high level of proficiency in administration and interpretation, necessitating the examiner to be a trained professional. Since the test can take more than an hour, both participants and examiners can become fatigued, which may lead to less objective evaluation results. Additionally, since they are paper-pencil-based, they require significant time for scoring and interpreting results.

To address these limitations of conventional paper-pencil-based tests, a digital cognitive test called the Seoul Cognitive Status Test (SCST) has been developed. The SCST is a standardized test with age, education, and sex-specific norms established based on 480 cognitively normal adults.5 It has demonstrated convergent validity through correlation analysis of its subtests with those of the SNSB-II and CERAD-K.56 Additionally, studies have compared the diagnostic performance of the SCST with that of the CERAD-K in distinguishing between normal and mild cognitive impairment groups.7

However, no studies have yet compared the performance (subtest scores, domain scores, total score) of the SCST among 3 groups (normal cognition, mild cognitive impairment, dementia) classified using the CERAD-K. Such comparison might further validate the diagnostic performance of the SCST. Furthermore, previous studies have primarily used data from university hospitals and Regional Centers for dementia, with no research conducted on patients from private neuropsychiatric clinics. Therefore, the objective of this study was to use data of a large sample of patients from university hospitals, regional centers for dementia, and a private neuropsychiatric clinic to validate the clinical utility of the SCST by comparing its scores of those with subjective cognitive decline (SCD), amnestic mild cognitive impairment (aMCI), and dementia diagnosed by the Korean version of the CERAD-K.

METHODS

Participants

Participants were 296 patients (SCD, n=134; aMCI, n=68; dementia, n=94). Specifically, this study collected data from 2 1) participants of the PREMiER study (PREcision medicine platform for mild cognitive impairment on multi-omics, imaging, and evidence-based R&BD), a prospective community-based cohort study, and 2) patients visiting a neuropsychiatry clinic in Seoul. PREMiER study data included 108 participants who visited 9 regional centers for dementia (Chuncheon, Hongcheon, Hwacheon, Sosa, Wonmi, Yangsan, Gijang, Haeundae, Changwon) and university hospitals (Samsung Medical Center, Soonchunhyang University Hospital in Bucheon). From this PREMiER study, two participants were excluded due to missing Instrumental Activities of Daily Living (IADL) scores. A total of 190 participants from a neuropsychiatry clinic in Seoul were also included in this study.

The study protocol was approved by the Public Institutional Review Board (IRB approval P01-202306-01-033) and the Soonchunhyang University Hospital IRB (IRB approval 2017-04-058). Written informed consent was obtained from all participants.

Diagnosis criteria

Participants were classified into 3 groups (SCD, aMCI, dementia) according to the following diagnostic criteria. SCD was defined as individuals who reported cognitive decline but performed normally on neuropsychological tests without having any other psychiatric or neurological disorders. Those with aMCI were selected based on Petersen’s criteria8. They showed subjective complaints of cognitive decline, objective memory impairment, normal activities of daily living (ADL), and absence of dementia. Although dementia is a heterogeneous condition with various causes, combined incidence rate of Alzheimer’s disease and vascular dementia has reached up to 90%.9 Therefore, the dementia group included only those with Alzheimer’s dementia (probable AD) and subcortical vascular dementia. Alzheimer’s dementia followed the diagnostic criteria of the National Institute on Aging and Alzheimer’s Association for probable AD.10 Subcortical vascular dementia followed the Clinical Research Center for Dementia of South Korea (CREDOS) diagnostic criteria.11 Objective memory decline was defined as scoring below −1.5 z-score on at least one of the following CERAD-K word list memory test, word list recall test, word list recognition test, and constructional recall test.3 ADL was assessed using the K-IADL1213 or S-IADL14 scale. Participants with scores of 0.4 or higher on the K-IADL or 8 or higher on the S-IADL were defined as having abnormal ADL.

Exclusion criteria

Participants were excluded when they had degenerative brain diseases other than Alzheimer’s disease dementia (e.g., frontotemporal dementia, dementia with parkinsonism, dementia with Lewy bodies, progressive supranuclear palsy, corticobasal degeneration) or secondary dementia due to conditions such as vitamin deficiency, neurosyphilis, and thyroid disease. Participants with structural brain abnormalities such as tumors, traumatic brain injury, hydrocephalus, and territory infarction were also excluded based on brain MRI.

SCST

The SCST, formerly called Inbrain Cognitive Screening Test, is a digital cognitive function test based on a 12-inch Android tablet (Galaxy Tab S7Fe, SM-T733; Samsung, Suwon, Korea). It consists of 7 subtests that evaluate 5 cognitive attention, language, visuospatial function, memory, and frontal/executive function. Test structure of the SCST is shown in Table 1. The seven subtests take approximately 30 minutes to complete in total. Except for some memory subtests that include verbal components and the language test, all subtest scores were automatically scored. Additionally, in order to double-check the accuracy of participants’ verbal responses, their responses in verbal tasks were automatically recorded on the exam device for voice playback. In most participants (n=263, 89%), the SCST was administered after CERAD-K (within an average of 28 days), while others completed the SCST first and then completed the CERAD-K (within an average of 53 days).

CERAD-K

CERAD was a paper-based comprehensive neuropsychological test developed in the United States in 1989. The Korean version of CERAD was standardized.2 The second edition was released in 2015.15 CERAD-K is a battery test consisting of 11 subtests that assess four cognitive domains (excluding attention) and the MMSE in the Korean version of CERAD assessment packet, which evaluates overall cognitive function. The test structure of the CERAD-K is shown in Table 2.

Statistical analysis

Using one-way analysis of variance (ANOVA), demographic and clinical characteristics were compared among three cognitive groups (SCD, aMCI, dementia) specifically classified by CERAD-K. In particular, for the following analyses, we examined whether age and years of education significantly differed among the 3 groups, as cognitive performance might depend on these variables. The three groups had no statistical differences in age (p=0.247) or years of education (p=0.606). Therefore, performances of the SCST, including subtest scores, domain scores, and total score, were compared among the three groups using one-way ANOVA without adjusting for age or years of education. Post-hoc analysis was conducted using the Bonferroni test and paired comparisons. Additionally, convergent validity of SCST subtests was verified using Pearson’s correlations between each corresponding pair of subtests from the CERAD-K and the SCST. Receiver operating characteristic (ROC) curve analysis was performed to determine the sensitivity and specificity of the SCST total score, confirming its utility in distinguishing among the SCD, MCI, and dementia groups. All statistical analyses were conducted using Statistical Package for the Social Sciences (SPSS ver. 24; IBM Corp., Armonk, NY, USA).

RESULTS

Comparison of demographic and clinical characteristics of 3 groups

Demographic and clinical characteristics of study participants (SCD, n=134; aMCI, n=68; and dementia, n=94) are presented in Table 3. There was no significant difference in age (p=0.247) or years of education (p=0.606). As expected, there were differences in Korean version of the MMSE scores among the 3 groups (p<0.001). Specifically, as participants were at more advanced stages from SCD to aMCI and further to dementia, they had lower scores. While Clinical Dementia Rate (CDR) and CDR-Sum of Boxes (CDR-SoB) showed differences between aMCI and dementia groups (p<0.001), their differences between SCD and aMCI groups were not significant (CDR: p=0.354; CDR-SoB: p=0.177).

Comparison of SCST domain & total scores in 3 groups

As shown in Table 4, SCST domain scores and total scores demonstrated significant differences between groups (p<0.001). SCST z-scores showed significant differences among the 3 groups (p<0.001). Specifically, participants at more advanced stages of dementia had lower SCST z-scores.

Comparison of SCST subtest scores in 3 groups

Results of comparing CST subtest scores are presented in Table 5. Raw scores of SCST subtests showed significant differences among the three groups (p<0.001). Specifically, participants at more advanced stages had poorer performances. However, post-hoc analysis revealed that there was no significant difference in the Phonemic Word Fluency Test (PWFT) or Korean Trail Making Test-Elderly version (K-TMT-E) Part A between SCD and aMCI. There was no significant difference in Word-Place Association Test (WPAT): Place Recognition (PR) between aMCI and dementia groups either.

All subtests showed differences in SCST z-scores among the 3 groups (p<0.001). Specifically, participants at more advanced stages exhibited lower SCST z-scores. However, post-hoc analysis revealed no significant difference in Visual Span Test (VST) forward, PWFT, or K-TMT-E Part A between SCD and aMCI groups. WPAT: PR showed no significant difference between aMCI and dementia groups either.

Correlation between domain specific subtests from SCST and CERAD-K

There were 8 SCST subtests that might correspond to those of CERAD-K in measuring each cognitive domain. Convergent validity between each pair of tests was evaluated using correlation analysis. As presented in Table 6, verbal fluency of CERAD-K might correspond to the Semantic Word Fluency Test of SCST as both tests could assess word generation capabilities. Both the Boston Naming Test from CERAD-K and the Difficult Naming Test from SCST could assess confrontational naming abilities in the language domain. Constructional praxis from CERAD-K and Block Design Test (BDT) from SCST could be equivalent in that both tests could assess visuospatial abilities. The word list memory tasks from CERAD-K might correspond to the WPAT from SCST because both tests could assess immediate recall, delayed recall, and recognition parts of memory task. Lastly, CERAD-K’s trail making test and SCST’s K-TMT-E were similar in that both tasks could assess the psychomotor speed and cognitive set-shifting abilities by employing Part A and Part B of trail making. Correlations between each pair of corresponding subtests were all significant (r=0.271–0.803; p<0.001).

ROC curve analysis of the SCST total score

To confirm the power of the SCST total score to distinguish between cognitively impaired individuals (aMCI and Dementia groups) and cognitively normal individuals (SCD group), we conducted an ROC curve analysis and calculated the area under the curve (AUC) (Fig. 1). First, in comparing the SCD and aMCI groups, the SCST total score had a sensitivity of 0.662 and a specificity of 0.769 with a cut-off score of 50.51. The AUC of the SCST was 0.773, demonstrating good discriminatory power. Second, when distinguishing between the cognitively impaired (aMCI + Dementia) and the cognitively normal (SCD) groups, the SCST total score had an AUC of 0.873 with a cut-off score of 46.06, the sensitivity and specificity were 0.765 and 0.851, respectively. Finally, in the comparison between the Dementia and non-dementia (aMCI + SCD) groups, the AUC for the SCST total score was 0.898. Applying a cut-off score of 42.06, the SCST total score showed a sensitivity of 0.862 and a specificity of 0.782.

DISCUSSION

The main objective of this study was to assess the ability of SCST to distinguish between SCD, aMCI, and dementia groups. Previous studies have already demonstrated differences in SCST scores among the 3 groups diagnosed using SNSB, a precise neuropsychological test in Korea.5 In this study, we reaffirmed the diagnostic validity of SCST by showing differences in total scores, cognitive domain scores, and most subtest scores among the 3 groups classified by CERAD-K.

SCST total scores and domain scores showed differences among the 3 groups, with the dementia group having lower scores than the aMCI group and the aMCI group having lower scores than the SCD group. However, in post-hoc analysis, some subtests showed no significant differences. Firstly, when using SCST z-scores, VST forward, PWFT (/diguet/), and K-TMT-E Part A showed no significant differences between SCD and aMCI groups. This finding might be due to a ceiling effect as these tests might have a relatively lower level of difficulty. Furthermore, our SCD and aMCI participants did not differ in CDR or CDR-SoB scores, suggesting that aMCI participants in this study might have an early stage of aMCI. Therefore, the combination of the ease of tests and the earlier stage of aMCI in our participants might have resulted in no significant differences in the three tests. Secondly, when using SCST z-scores, there were no significant differences in the Place Recognition task of WPAT between aMCI and dementia. This could be attributed to a floor effect due to a higher difficulty level of this test compared to others.

To assess convergent validity of SCST, we examined correlations of corresponding subtests. Although these corresponding tests were not identical, they evaluated the same cognitive domains respectively. All corresponding subtests showed moderate to high correlations (r=0.477–0.803) except for a weak correlation between CERAD-K’s trail making test Part A and SCST’s K-TMT-E Part A (r=0.271). There are some intriguing points to be discussed.

First, CERAD-K’s trail making test Part A and SCST’s K-TMT-E Part A had a relatively weak correlation despite having only slight differences in number span (CERAD-K: 25 vs. SCST: 15). This might be attributed to individual differences in familiarity with a touchscreen device. Specifically, some participants unskilled with touchscreen devices might have more difficulty and need more time to draw the line, which might have influenced the time to complete K-TMT-E Part A. As they became more familiar with the device, this confounding factor might no longer affect the following K-TMT-E Part B. Thus, additional practice sessions might be required for participants to become familiar with the touchscreen before administering the K-TMT-E Part A.

Second, CERAD-K’s trail making test Part B and SCST’s K-TMT-E Part B showed a high correlation. While CERAD-K required participants to alternate between 13 numbers (from ‘1’ to ‘13’) and 12 letters of the Korean alphabet (‘ga’ to ‘ta’), SCST required participants to alternate between 8 numbers (from ‘1’ to ‘8’) and 7 days of the week (‘Monday’ to ‘Sunday’). CERAD-K required participants to navigate Korean alphabetical order, which could be challenging for some individuals and time-consuming. It contrasted with SCST’s relatively shorter duration and the ease of memorizing days of the week compared to Korean alphabets. Despite these differences between the two tests, the high correlation observed was an intriguing finding.

Lastly, the constructional praxis from CERAD-K and the BDT from SCST exhibited a moderate correlation despite some differences. Specifically, while other pairs of corresponding tests had highly similar formats, the Constructional praxis and SCST’s BDT required different behavioral responses. The Constructional praxis involved copying four images, but the BDT required reproducing the same pattern by dragging specific patterned squares (a digitalized version of BDT). This suggests that the concept of visuospatial ability might be sufficient to produce a moderate correlation.

A limitation of this study should be noted. There might be a potential practice effect between CERAD-K and SCST, as most participants completed SCST after CERAD-K. Thus, future studies might benefit from more rigorously mitigating potential biases by randomizing the sequence of administering these 2 tests.

In conclusion, previous studies have validated the clinical utility of SCST through its strong correlation with SNSB. Through comparison of CERAD-K and SCST in this study, we reaffirmed its clinical validity.