Authors: J. Rubel, T. Väth, S. Hanraths, L. Pruessner, C. Timm, S. Hartmann, S. Barnow, C. Lalk
Categories: Full Length Article, Generalized anxiety disorder, Internet-based interventions, Randomized controlled trial
Source: Internet Interventions
This study aimed to evaluate the effects of an online self-help intervention for generalized anxiety disorder (GAD). Our primary outcomes were generalized anxiety symptoms, measured using the Generalized Anxiety Disorder – 7 (GAD-7; Spitzer et al., 2006), and wellbeing based on the World Health Organization Wellbeing Index - 5 (WHO-5; Topp et al., 2015).
A total of 156 German-speaking patients aged 18 to 65 with a diagnosis of GAD and internet access were included in this randomized controlled trial. The intervention group (N = 78) received access to a 12-week online self-help program, while the waitlist control group (N = 78) received access after the 12-week waiting period.
The intervention group showed a significant improvement in generalized anxiety symptoms compared to the control group (t(df = 123.73) = 4.52, p < .001) with a large effect size (d = 0.88, 95 %-CI: 0.50; 1.26). Additionally, the intervention group demonstrated a significant increase in wellbeing compared to the control group (t(df = 87,86) = 3.48, p < .001), with a moderate effect size (d = 0.62, 95 % CI: 0.27; 0.98). However, no significant effects were observed for secondary outcomes of functional impairments, work productivity, mental health literacy, and healthcare demands. For exploratory outcomes, improvement was found for anxiety and worry symptoms.
These findings suggest that an online-based self-help intervention effectively reduces GAD symptoms and improves overall wellbeing. Future research should explore the long-term effects of this intervention and investigate potential mechanisms underlying its efficacy.
Online-based self-help programs provide a promising treatment option for individuals with GAD who face barriers to traditional face-to-face therapy.
Keywords: Randomized controlled trial, Internet-based interventions, Generalized anxiety disorder
Generalized anxiety disorder (GAD) is a common mental health condition marked by excessive and uncontrollable worry about past, present, or future (DSM-5; American Psychiatric Association, 2013). GAD causes distress and impairment in daily live (Judd et al., 1998), often co-occurring with depressive disorders (Tyrer and Baldwin, 2006). GAD's economic costs are a challenge to healthcare systems (Bereza et al., 2009; Grant et al., 2004), with an estimated lifetime prevalence of 1.6 %–10.5 % (Preti et al., 2021). Hoffman et al. (2008) reviewed three studies assessing economic costs of GAD based on healthcare use and absenteeism. The largest cost factors were outpatient treatment (42 % of costs), sick leave (33 %), and hospitalization (21 %). GAD patients with a comorbidity, most often depression (26 %), substance use disorders (24 %), and gastroenterological symptoms (14.6 %), had almost twice of the costs than patients without a comorbidity. Altogether, the costs of GAD were estimated to be significant cost drivers due to referrals to family doctors and specialists on the grounds of unclear somatic illnesses. This may be a result of misdiagnosing and mistreating GAD as a somatic illness (Hoffman et al., 2008).
GAD is treated with pharmacotherapy or psychotherapy (Carl et al., 2020; Portman et al., 2012) with lower response rates than other anxiety disorders, likely due to delayed treatment (Becker and Hoyer, 2005), personality disorder comorbidity (Friborg et al., 2013), and misdiagnosis and mistreatment (Mowbray et al., 2006). Recently, technology-based solutions provide new treatment approaches (Aguilera, 2015). Cognitive behavioral therapy (CBT) is suitable for adaptation into an online intervention due to its structured nature and its emphasis on psychoeducation and homework assignments (Berger et al., 2011). Internet-based CBT (iCBT) effectively reduces GAD symptoms (g = 0.70–0.79) compared to control (Andrews et al., 2018; Eilert et al., 2021). Andrews et al. (2018) conducted at meta-analysis with 9 trials on GAD and over 1100 subjects comparing iCBT with support to a waitlist-control condition with a large effect size for iCBT. More recently, Eilert et al. (2021) conducted another meta-analysis of iCBT with a waitlist-control condition. Again, with the exception of two trials, iCBT was conducted with additional support via email, telephone, or other means. They found large effect sizes for GAD severity and worry, and moderate to large effects for depression, functional impairments, and quality of life.
Mewton et al. (2014) conducted a review of 37 RCTs to assess the effects of iCBT with minimal guidance (e.g., email support) versus face-to-face CBT and found no differential effect. However, therapists spent 13 times as much time for each patient in the face-to-face CBT condition. Also, they found no indication that level of therapist guidance (e.g., face-to-face meetings, phone or email contact, or no support) was predictive of iCBT effects. This is particularly relevant considering the rising healthcare costs associated with GAD treatment (Hoffman et al., 2008) and the prolonged waiting times for outpatient therapy in Germany, which have been further exacerbated by the COVID-19 pandemic (Bundespsychotherapeutenkammer, 2018, Bundespsychotherapeutenkammer, 2021).
Notably, no differences in symptom reduction, completion rates, and user satisfaction emerged in a large RCT (n = 338) between guided and unguided iCBT for GAD (Dear et al., 2015). Furthermore, specific aspects of guidance, such as the frequency of supportive monitoring or the experience level of support personnel, may not significantly impact GAD symptom reduction (Hadjistavropoulos et al., 2017; Robinson et al., 2010). Taken together, minimally guided online interventions offer low-threshold access to mental healthcare. However, most studies on iCBT for GAD still involve high levels of guidance (e.g., support by therapist or researcher via email, telephone, or text messaging) (Eilert et al., 2021). Therefore, there is a need to strengthen the evidence-base for interventions without therapist support.
In addition to their effects on symptom reduction, iCBT programs have demonstrated positive outcomes in terms of quality of life and functioning (Eilert et al., 2021). However, effects on wellbeing, mental health literacy and healthcare usage are not yet thoroughly In contrast to quality of life, wellbeing is a more general and more subjective assessment of life satisfaction (Skevington and Böhnke, 2018). It encompasses cognitive and emotional reactions to one's life, while quality of life is measured as life satisfaction across several areas of life. Yet, in a combined model of quality of life and wellbeing, positive emotions are central (Skevington and Böhnke, 2018), underscoring the importance of wellbeing. It has utility across different study domains (e.g., suicidology and pain; Topp et al., 2015) and is positively associated with longevity (Martín-María et al., 2017). Further, wellbeing allows individuals to explore and develop new resources (Fredrickson, 2004), leading to decreased long-term mental health risk (Burns et al., 2022). Mental health literacy refers to knowledge about mental health and treatment, as well as attitudes that promote recognition and help-seeking (O'Connor and Casey, 2015). Levels are higher among people with mental illness and people with prior treatment (O'Connor and Casey, 2015). Also, it has been shown that lower levels can impede help-seeking behavior (Furnham and Swami, 2018), which is particularly relevant for GAD because of notoriously late treatment-seeking (Becker and Hoyer, 2005). Healthcare usage refers to the scope of specific healthcare services utilized by the patient, allowing for an analysis of the associated costs (Chisholm et al., 2000). Due to restricted financial resources in health care systems around the globe, healthcare usage provides a complementary perspective on treatment effectiveness by focusing on a health economic perspective. It is worth noting that the impact of iCBT on mental health literacy and healthcare burdens has yet to be thoroughly investigated (Eilert et al., 2021).
We aim to assess effects of an iCBT intervention for GAD (Selfapy) in conditions comparable to routine care. The outcomes symptom reduction, well-being, functioning, mental health literacy, and healthcare burdens will be analyzed. For this purpose, we conducted a randomized controlled trial comparing the self-help intervention group (IG) with a waitlist control group (CG).
The primary hypotheses focused on disorder-specific symptoms and well-being. The secondary hypotheses focused on functioning in daily life and work ability, mental health literacy, and healthcare burden. Mental health literacy has been selected since lack thereof could impede treatment-seeking behavior.
The primary hypotheses were compared against the control group (CG) after 12 weeks:
A two-arm randomized controlled trial was conducted as preregistered (https://drks.de/search/en/trial/DRKS00023799) to test the efficacy of the online self-help intervention for GAD (Rubel et al., 2023). Before randomization and to check the inclusion and exclusion criteria, a structured diagnostic interview with the Diagnostic Interview for Mental Disorders-Open Access (DIPS-OA, Margraf et al., 2017a; Margraf et al., 2017b) was conducted via video call with every subject. The DIPS-OA allows reliable diagnoses according to the DSM-5 and ICD 10. Regarding DSM-IV-TR criteria, the DIPS-OA was found to have acceptable interrater (0.78) and retest (0.76) reliability for anxiety disorders (Suppiger et al., 2008). Patients who met the criteria were then randomized into either the CG or the IG in a nonstratified 1 fashion. Patients in the IG had immediate access to the intervention, while the CG were only permitted access after a waiting period of 12 weeks. Interim and final evaluations occurred 6 (T2) and 12 (T3) weeks after randomization. This study and its report adhere to the CONSORT statement (Schulz et al., 2010) by reporting details (e.g., randomization procedure) about the study to ensure methodological rigor and replicability.
Study recruitment in Germany utilized social media ads, flyers to medical professionals, therapists, and self-help groups, as well as a university newsletter. Interested individuals took a screening questionnaire to assess the inclusion criteria age, internet access, and language skills, and to screen for the diagnostic inclusion and exclusion criteria. For this purpose, we used items from the DIPS-OA and the whole Generalized Anxiety Disorder-7 and Patient-Health-Questionnaire-9 (see outcomes section). Subjects scoring above threshold in the screening questionnaire underwent diagnostic video interviews with the DIPS-OA afterwards. In total, 4361 started screening, with 156 (3.58 %) enrolled in the trial after meeting the criteria.
To clarify the inclusion and exclusion criteria, video call structured interviews were conducted to assess study eligibility via the DIPS-OA. All interviews were conducted by trained interviewers with a master's degree in psychology. Furthermore, they were closely supervised by a certified psychotherapist (CBT).
Eligible subjects 1) were between 18 and 65 years of age, 2) had sufficient knowledge of the German language, 3) had uninterrupted Internet access, 4) provided electronic informed consent to participate in the study, and 5) currently met criteria for a diagnosis of GAD (DSM-5300.02; American Psychiatric Association, 2013).
Subjects were excluded if they met any of the following 1) past or current diagnosis of bipolar disorder, 2) past or current diagnosis of psychotic disorder, 3) current diagnosis of substance dependence, 4) current diagnosis of severe major depressive episode, 5) acute suicidality. A primary diagnosis of one or more disorders other than GAD, in general, was not an exclusion criterion, as we wanted to represent routine care. Substance dependence, bipolar disorder, or psychotic disorders, however, were selected as exclusion criteria as they are deemed to interfere with the successful implementation of the intervention according to German regulations. Subjects who did not meet our inclusion criteria but presented with a diagnosis were encouraged to seek therapeutic help. For all included patients, an individual emergency plan was developed in case of symptom deterioration or suicidality. Adequate language skills were determined during the initial interview.
Selfapy is an online self-help program for the treatment of GAD. The program is based on evidence-based methods, CBT exercises, and elements from Mindfulness-Based Therapy (e.g., Hoyer et al., 2016; Volz and Stieglitz, 2010). The online course consists of a core course, which includes mandatory and optional exercise content and a subsequent set of in-depth modular areas that are individually selectable. It can be accessed via the web and on mobile devices. The course is divided into 12 modules that can be worked on for a week each. Each module covers a specific topic, such as exposure, mindfulness, or problem-solving training. The modules contain informative texts, videos, audio, and interactive exercises. The first seven modules are considered the main course, while subsequent modules include additional content. An overview of all modules of the intervention can be found in the Online supplementary material (OSM) 51.
The online course was completed independently by the patient. However, as part of the patient safety concept, a psychologist monitored the participant's progress to respond to adverse events such as suicidality. Via an integrated messaging function, the patients could contact a psychologist in the event of safety concerns. Active communication by the psychologist occurred only in case of safety concerns and to maintain patient safety.
Subjects in the intervention group had immediate access to the 12-week internet-based self-help treatment. They were advised to spend at least 15 to 20 min daily on the program. Patients in the CG did not receive any treatment from the researchers during the first 12 weeks after the initial survey. However, they could seek other assistance, including pharmacological and psychological treatments. All concurrent treatments were measured repeatedly using self-reports.
A waitlist design was chosen for the CG to control for changes due to spontaneous improvement, e.g., regression to the mean. Besides, 12 weeks is a typical waiting time for psychotherapy in Germany (Bundespsychotherapeutenkammer, 2018), meaning that this comparison will allow us to assess the additional benefit of the intervention in the current situation.
The sociodemographic characteristics of all patients are displayed in Table 1. Eighty-three (53.5 %) patients fulfilled the criteria for a diagnosis of social phobia, and 29 (18.7 %) fulfilled the criteria for panic disorder in addition to a diagnosis of GAD. Also, 25 (16.0 %) patients were diagnosed with major depressive disorder, and 20 (12.9 %) patients had dysthymia. Depression levels were comparably low because patients with severe depression were excluded from participation. An overview of the frequency of all past or present diagnoses is shown in OSM 50.
All patients gave informed consent. The study was approved by the ethics committee at the study center at Heidelberg University (Ethics Committee-No. AZ Prüß 2021 1/1). Patients were asked about suicidality at T1, T2, and T3. If they reported 1 (“on individual days”) or higher on a 4-point Likert scale (0: “not at all” - “almost every day”) to the extent that they “had thoughts [that] you would rather be dead or would like to harm yourself” in the last two weeks, the patients were contacted by phone, and an emergency plan was drawn up with them. If contact by phone was not possible, contact was made via email. If patients were contacted due to suicidality, it was ethically necessary to implement a protocol that prevented further participation in completing questionnaires for these individuals to prioritize their immediate needs and ensure appropriate support. However, all data collected up to this point was still used. Exclusion due to suicidality had to be conducted once during the trial in the CG.
Primary and secondary measures were conducted at the beginning of the intervention (T1, baseline), after 6 weeks (T2, mid-treatment), and after 12 weeks (T3, post-treatment). The patients had to complete questionnaires regarding the primary, secondary, and exploratory hypotheses at all measurement points, except for the negative effects questionnaire, which was only given at T2 and T3. Additionally, at T1, a demographic questionnaire had to be filled out.
The change in generalized anxiety disorder symptoms was evaluated using the Generalized Anxiety Disorder Scale-7 (GAD-7; Spitzer et al., 2006). As a reliability measure for our data, we calculated McDonald's omega (Hayes and Coutts, 2020) with ω = 0.75 for the GAD-7 at T1. Wellbeing was assessed by the five-item World Health Organization Well-being Index (WHO-5; Topp et al., 2015). The reliability of the WHO-5 was high, with ω = 0.81 at the T1.
Four secondary outcomes were collected. Self-reported difficulties in daily functioning were measured by the Work and Social Adjustment Scale (WSAS; Mundt et al., 2002). The WSAS is a five-item scale with a test-retest correlation of r = 0.73 (Mundt et al., 2002) and had good reliability with ω = 0.73 at T1.
Working ability was measured with the iMTA Productivity Cost Questionnaire (iPCQ; Bouwmans et al., 2013) to assess the amount of lost working hours due to the mental disorder in the last four weeks. The iPCQ consists of 18 questions aimed at measuring these hours. The iPCQ has been shown to be feasible and easily understandable with a retest-reliability of icc = 0.83 (Bouwmans et al., 2013, Bouwmans et al., 2015).
Mental health literacy was measured with the Mental Health Literacy Scale (MHLS; O'Connor and Casey, 2015). The MHLS contains 35 items about knowledge, attitudes, and competencies regarding mental health with high reliability (ω = 0.85 at T1) and good retest-reliability of r = 0.79 (O'Connor and Casey, 2015).
The extent of therapy-related healthcare usage was collected using three subscales of the Client Sociodemographic and Service Receipt Inventory (CSSRI; Chisholm et al., 2000): CSSRI partly inpatient to assess the amount of time of partly inpatient treatment (e.g., psychiatric day care), CSSRI complementary to assess attendance of complementary services (e.g., self-help groups), and CSSRI outpatient to assess attendance of outpatient services (e.g., psychotherapy treatment, medical treatment). Sousa et al. (2013) report excellent interrater-reliability of the CSSRI with coefficients between 0.8 and 1.0.
The following outcomes were deemed as exploratory, since they seem less central and there is already some data on these outcomes (e.g., Eilert et al., 2021). Adverse treatment effects were collected with the Negative Effects Questionnaire (NEQ; Rozental et al., 2016) as an exploratory measure. The NEQ contains 32 items and shows satisfactory reliability with ω = 0.71 at T3. Also, the Beck Anxiety Inventory (BAI; Beck et al., 1988) assessed general symptoms of anxiety. The BAI is a 21-item scale with a high internal consistency with ω = 0.88 at baseline measurement and retest-reliability of r = 0.78 (Geissner and Huetteroth, 2018). Depressive symptoms were collected with the Patient Health Questionnaire-9 (PHQ-9; Kroenke et al., 2001). The PHQ-9 is a short scale of nine items with good reliability (ω = 0.75 at T1) and good retest-reliability with r = 0.84 (Kroenke et al., 2001). Levels of trait worry were collected with the Penn State Worry Questionnaire (PSWQ; Meyer et al., 1990). The PSWQ is a 16-item scale for assessing the worry facet as an accompanying syndrome of Generalized Anxiety Disorder. The PSWQ showed high reliability with ω = 0.82 at T1.
Presupposing recent meta-analytic data for unaccompanied online psychological interventions for anxiety disorders (d = 0.45; e.g., McCall et al., 2021), a planned mixed model with two measurement time points with a general correlation structure (Lu et al., 2008), a directed hypothesis, 1 group allocation, 80 % power, and an alpha level of 0.025 after Bonferroni-Holm correction, a total of 156 patients (78 per group) were needed (longpower; Donohue, 2021). For the secondary outcomes, we estimated a minimal effect size of d = 0.48 to achieve 80 % power with an alpha level of 0.0125 (Bonferroni-Holm adjustment) based on a post-hoc power analysis of the WSAS (simr; Green and MacLeod, 2016).
Eligible subjects were randomly allocated to either immediate online self-help for GAD (IG) or intervention after 12 weeks (CG). Randomization happened after the clinical interview and was conducted by a non-project member using computer assistance. Therefore, allocation remained concealed until after the video interview so that interviewer blindness was ensured. Subjects were assigned in a 1 ratio. Patients were informed of their group via email, and were told that the waiting time was randomly varied. After data collection, statistical analysis occurred blindly, managed by an independent team member. The analysis R script was prepared before data collection.
The statistical analyses were conducted following the study protocol (Rubel et al., 2023), which was prepared and agreed upon before the analysis began. The analyses were performed with R, version 4.2.0 (R Core Team, 2022).
For the analyses of the psychometric outcome measures, different analyses were calculated. As documented in the study protocol, all hypotheses were evaluated based on an ITT analysis with missing values replacement by multiple imputations (“Multivariate Imputation by Chained Equations”; MICE; with n = 5 imputations; Azur et al., 2011) based on the control arm, using the variables “age” and “gender” as predictors in addition to the measurement-repeated variable. Four additional sensitivity analyses were A completer analysis using only patient data with completed T1 and T3 measures, “last-observation-carried-forward” (LOCF), “baseline-observation-carried-forward” (BOCF), and a “reference-based-multiple imputation” (J2R; Carpenter et al., 2013).
The primary outcome parameters were examined regarding the fixed interaction effect between group allocation and time for T3 (last survey). In order to adjust for multiple testing, a Bonferroni-Holm adjustment was performed for the primary and secondary outcomes. The results reported in the text refer to the MICE imputation analyses unless specified otherwise, as these were the decisive criteria for significance.
The confirmatory analysis of the primary endpoints consisted of calculating a mixed model with two measurement time points with a general correlation structure (Lu et al., 2008). A random effect for the subject was calculated (random intercept), and three fixed effects (group assignment, time, and the interaction of the two effects). The two measurement time points were nested within subjects. Secondary confirmatory outcomes were calculated only after success in the primary analysis similarly.
Independent t-tests and χ2-tests were used to estimate differences between groups in pretreatment sample characteristics. Also, t-tests were used to identify differences in negative effects in the NEQ at T2 and T3. In addition to the ITT sample, a “per-protocol” sample sensitivity analysis was defined for exploratory analyses, including all IG patients who completed at least 4 of the 12 modules.
To assess the magnitude of the treatment effects, the fixed interaction effect of time and group assignment was divided by the root of the summed variances of the random effects (Westfall et al., 2014). Effect sizes can be roughly interpreted according to Cohen's d: Effect sizes of 0.20 are considered small, 0.50 moderate, and 0.80 large (Cohen, 1988). Differences in response rates and rates of use of additional care system services were examined with t-tests and χ2-tests. Additionally, reliable change was assessed using the Reliable Change Index (RCI; Jacobson and Truax, 1992) to calculate reliable improvement or deterioration. The RCI3 was calculated for the MICE imputation by calculating change scores and taking into account the variability of these scores to assess if change is systematic or just random. Patients were either classified as reliably deteriorated, unchanged, or reliably improved.
The CSSRI questionnaire was split into three subscales (CSSRI partially inpatient, CSSRI outpatient, and CSSRI complementary) subject to Bonferroni-Holm adjustment by dividing by 3, 2, and 1, respectively.
Due to highly skewed data, the iPCQ and the CSSRI partially inpatient scales were log10-transformed. The CSSRI outpatient and CSSRI complementary scales wer0e dichotomized because of rare extreme outliers, in which case a transformation is not useful (Lei et al., 2017). The analysis was kept as close as possible to the preregistration for these dichotomized measures by calculating a mixed logistic regression model with random intercept and three fixed effects (group, time, and group*time). Odds ratios were calculated as the effect size.
The exploratory outcome measures BAI, PHQ-9, and PSWQ were analyzed using the same model as the primary and secondary outcomes but without alpha adjustment and as a two-sided test because of the exploratory nature of these outcomes.
All data and the respective analysis code have been made publicly available at the OSF repository and can be accessed at https://osf.io/tj4re/. Materials about the content of the intervention are reported in the study protocol (Rubel et al., 2023).
Recruitment spanned from 17th February 2021 to 1st April 2022 (see Fig. 1). In total, 4361 participants filled out the screening questionnaire, and 764 met participation criteria, while 160 withdrew. Of the remaining 604, 292 (48.3 %) lacked appropriate diagnosis, and 156 (26.0 %) were excluded due to panic disorder eligibility and were treated in another trial targeting panic disorder (Lalk et al., 2023). Due to the parallel enrolment for iCBT for GAD and for panic disorder, the number of exclusions is larger than it would be expected for a single trial of this size. Unfortunately, it is impossible to assign excluded subjects to a specific trial, leading to these high numbers. Chi-square testing did reveal significant differences between the groups (see Table 2) regarding their relationship status (χ^2^ = 9.53, p = .013) with fewer married and more singles in the IG. Further, the IG was younger on average (t = −2.13, p = .035). Regarding diagnoses, significant differences were found for obsessive-compulsive disorder (5.2 % in IG versus 18.0 % in CG; p = .013).
Fig. 1 CONSORT flow diagram.Note. Deviations from the sample size did occur for some of the secondary and exploratory outcomes.
No differences were found for current psychopharmacology (χ^2^ = 0.029, p = .866), psychotherapy (χ^2^ = 0.521, p = .470), or baseline scores of any primary or secondary outcome. Altogether, 156 patients were randomized, with 78 patients in the IG and 78 in the CG. All randomized patients were included in the analysis.
One patient from the CG reported suicidality and had to be excluded from the study. However, their previous data were still included and missing data was analyzed adhering to ITT principles.
Non-completion rates for GAD-7 were 16.0 % at post-treatment and 19.9 % for WHO-5 resulting in 76.9 % (n = 60) completion of GAD-7 and 73.1 % (n = 57) completion of WHO-5 in the IG, and 91.0 % (n = 71) completion of GAD-7 and 87.2 % (n = 68) completion of WHO-5 in the CG. Logistic regression indicated no associations between non-completion and baseline variables like group, sex, age, fitness to work, medication, psychotherapy, or baseline values of outcomes. Dropouts occurred in 18 (23.1 %) patients from the IG and 6 (7.7 %) from the CG. Group differences were non-significant at T2 and T3 but significant overall (χ^2^(1) = 7.09, p = .008). No significant gender differences in dropout emerged.
Due to a technical problem, usage data was missing for three persons in the IG. Based on data from 75 patient, the IG completed an average of 7.1 (SD = 4.0) modules out of the total 12 modules. 21 (28.0 %) patients finished the whole course, 40 (53.3 %) patients finished seven modules (main course), and 53 (70.7 %) underwent the first four modules, which was chosen as a sensitivity analysis to assess a basic amount of engagement.
Descriptive statistics for the primary outcome measures (GAD-7 and WHO-5) for each assessment point are shown in Table 2 and OSM 2–3 for ITT raw scores and MICE imputation.
Regarding the main interaction effect at T3 (see Table 3), the GAD-7 measure was highly significant (t = −4.52, p < .001) across all imputations (see OSM 4). Further, the WHO-5 measure (t = 3.48, p < .001) was also highly significant with Bonferroni adjustment and across all imputations (see OSM 5). The imputations and effects can be seen in OSM 2–11. For GAD-7, the effect size was large (−0.88, 95 %-CI: −1.26; −0.50) and for WHO-5, it was moderate (0.62 95 %-CI: 0.27; 0.98).
Within-group effect sizes were large in the intervention group (1.02, 95 %-CI: 0.74; 1.29) and marginal to small in the control group for GAD-7 (0.14, 95 %-CI: −0.11; 0.39). The within-group effects for the WHO-5 were moderate to large in the intervention group (0.65, 95 %-CI: 0.36; 0.94) and marginal in the control group (0.03, 95 %-CI: −0.15; 0.21).
Reliable improvement was achieved in the GAD-7 by 43.8 % of the patients in the IG and 13.8 % in the CG. In the IG, 4.6 % deteriorated, while 9.7 % deteriorated in the CG. Therefore, a significant group difference in improvement and deterioration was identified (p < .001). Reliable improvement was achieved in the WHO-5 by 32.1 % of IG patients and 7.2 % of CG patients, while 5.6 % in the IG and 8.2 % in the CG deteriorated, constituting a significant difference (p < .001).
Additionally, per-protocol sensitivity analyses were calculated for patients that underwent the first four modules. Because no hypotheses were specified, the effects are presented as two-tailed tests without alpha adjustment. A significant group*time interaction (t = 6.11, p < .001) was found for the GAD-7 with a large effect size (d = 1.24, 95 %-CI: 0.84; 1.63) and the WHO-5 (t = 4.56, p < .001; d = 0.93, 95 %-CI: 0.53; 1.33).
Descriptive statistics can be found in Table 4, Table 5. Additional descriptive statistics, effect sizes and significance levels across the different imputations can be found in the OSM (OSM 12–35). None of the interaction effects was significant after the Bonferroni-Holm adjustment (see Table 6) and no effects in the expected directions were found in the imputations.
No interaction effect was found for all outcomes, even before the Bonferroni-Holm adjustment. Only in the WSAS a within-group effect was found but only for the IG (0.32 95 %-CI: 0.10; 0.53).
Descriptive values, significance levels, and effect sizes across all imputations can be obtained from the OSM for the exploratory outcomes (OSM 36–47). The effects of the exploratory outcomes are displayed in Table 7. A significant interaction was found for the BAI (t = 2.87, p = .004) with a small to moderate effect size (0.38, 95 %-CI: 0.12; 0.64). Significance was maintained across all imputations (see OSM 43). Effect sizes for the within-group changes were moderate to large for the IG (0.64, 95 %-CI: 0.45; 0.84) and small for the CG (0.27, 95 %-CI: 0.10; 0.43).
For the PHQ-9, no significant interaction could be found (t = 1.83, p = .067), though all imputations were significant (OSM 39). Within-group effects were small to moderate (0.37, 95 %-CI: 0.14; 0.59) in the IG and were minimal (0.10, 95 %-CI -0.08; 0.28) in the CG.
For the PSWQ, a significant interaction was found (t = 2.58, p = .009) with a moderate effect size (0.47, 95 %-CI: 0.11; 0.82). Significance was only maintained across the completer cases and BOCF imputations (OSM 47). Only a moderate within-group effect (0.46, 95 %-CI: 0.21; 0.71) was found in the IG with no within-group effect in the CG (0.01, 95 %-CI: −0.22; 0.23).
Regarding side effects, significant differences were found in the Negative Effects Questionnaire (NEQ, Rozental et al., 2016) with fewer adverse effects in the IG (t = 2.85, p = .005; see Table 8 and OSM 48–49). In a sensitivity analysis without the NEQ items that specifically addressed treatment (without items 20–32; e.g., “I did not always understand my treatment”), still fewer negative effects were reported in the IG (t = 3.98, p < .001).
The present study aimed to investigate the effects of an online self-help intervention for patients with GAD in conditions similar to routine care. Patients using the online self-help intervention revealed a more pronounced reduction in generalized anxiety symptoms compared to the waitlist control group. Additionally, well-being improved significantly compared to the CG. No effects were found for the secondary outcomes functioning in daily life, mental health literacy, working ability, and healthcare use. Regarding the exploratory outcomes, more substantial reductions were found for anxiety and rumination symptoms but not for depression.
The investigated iCBT intervention resulted in substantial within-group reductions in GAD symptom severity over the course of the treatment (d = 1.02). Symptom reduction was significantly more pronounced in patients who received the iCBT compared to those who did not receive treatment (d = 0.88). This effect is comparable to those reported in meta-analytic reviews on iCBT for GAD (g = 0.79, Eilert et al., 2021) and even traditional CBT (g = 0.84; Cuijpers et al., 2014). Also, similar to other iCBT trials (e.g., Ritola et al., 2022), effect size seemed to be affected by Patients with at least 4 of altogether 12 completed modules had an even bigger effect size (d = 1.24). The robustness of the effect is supported by the effects on worry (d = 0.47) and anxiety (d = 0.38). Therefore, it seems like the intervention is targeting both cognitive processes associated with GAD (i.e. worrying), as well as physical symptoms of anxiety. Yet, in their meta-analysis on the effects of iCBT on anxiety and depression across 19 trials, Păsărelu et al. (2017) found large effects on both anxiety and depression and moderate effects on anxiety comorbidities. Therefore, the effects on worry, anxiety and depression are smaller than would be expected, which may be in part due to the exclusion of severe depression as discussed later.
Regarding wellbeing, the effects were moderate to large (d = 0.65) within-group and moderate in comparison to the waitlist (d = 0.62). These effects are stronger than meta-analytic findings on quality of life (d = 0.33, Eilert et al., 2021). However, this might also be due to the different constructs used since the WHO-5 assesses wellbeing (Topp et al., 2015), while Eilert et al. (2021) looked at broader quality of life outcomes. Since quality of life is measured across different areas (e.g., mental health, physical health, relationships, purpose, …) it may be less sensitive towards change in mental health intervention studies. In contrast, wellbeing in general and the WHO-5 specifically are focused on current overall life satisfaction (Skevington and Böhnke, 2018). Also, the WHO-5 is known to be sensitive to change and is also used as a depression measure (Topp et al., 2015). To our knowledge, no iCBT study on GAD investigated the effects of the intervention on psychological wellbeing.
Regarding the secondary hypotheses, no significant interaction effects were found. However, there was a small to moderate within-group effect on functioning (d = 0.32) for the intervention with a marginal effect (d = 0.11) in the CG. No within-group effects were found for the MHLS or any CSSRI subgroup. In the iPQC, a small within-group effect (d = 0.19) was detected in the IG and no effect (d = 0.01) in the CG. There are several reasons for the absence of significant findings in the secondary outcomes. Firstly, the trial was not specifically powered to detect effects in the secondary outcomes, which was further impeded by the additional Bonferroni-Holm adjustment. A post-hoc power analysis showed that only moderate effects (d = 0.48) could be detected with 80 % power after alpha adjustment. Secondly, both the iPCQ and CSSRI subscales exhibited a strong floor effect (Ho and Yu, 2015) due to the rare occurrence of lost working hours (46.1 % reported none) and additional treatments (33.8 % no outpatient treatments, 90.8 % no inpatient appointments, 87.7 % no complementary treatments) at the beginning of the study. The MHLS scale also presented a ceiling effect (mean of 4.3 and a range from 2.9 to 5.0 on a scale from 1 to 5). Thirdly, the CSSRI subscales measured treatment use over the past three months, including the initial phase of the intervention, which might not have captured its full impact. Similarly, the iPCQ assessed the past six weeks, potentially leading to similar issues. In summary, the lack of significant results in the secondary outcomes can be attributed to the inadequate power analysis, substantial floor and ceiling effects, and extended measurement timeframes. Regarding the WSAS, meta-analytic evidence by Eilert et al. (2021) reported a substantial effect on functioning (d = 0.66) across ten studies with 343 patients, contradicting our findings. Nevertheless, this study has the most power to assess functioning and should not be discounted easily.
The exploratory outcomes revealed a significant interaction effect for anxiety and significantly lower levels of trait worry in favor of the IG. The lack of effects on depressive symptoms via the PHQ-9 is a surprise since Eilert et al. (2021) found a large effect in their meta-analysis. One reason might be the exclusion of patients with severe depression from the trial, which lead to high levels of patients with only mild or no depression in both the IG (82.0 %) and the CG (76.9 %).
Regarding negative treatment effects, patients in the IG reported fewer negative treatment effects than the negative effects of waiting reported in the CG. This was confirmed in a sensitivity analysis without items that directly focused on treatment effects. The reported negative effects are comparable in quality and quantity to the negative effects of face-to-face psychotherapy (e.g., increase in unpleasant memories and unpleasant feelings; Strauss et al., 2021). As such, we can conclude that the iCBT for GAD tested in the present study can be safely used for the included study population.
The effects of iCBT on functioning at home and work need to be investigated more thoroughly. Even though current meta-analytical evidence suggests an effect (Eilert et al., 2021), most studies, including this one, are too small and not adequately powered with a minimal detectable effect size of d = 0.48 for the secondary outcomes. Regarding health care usage and lost working hours, assessing these outcomes in a sample with higher impairment and a follow-up measurement to account for the floor effects and long measurement timeframes might be worthwhile.
The study has several limitations. First, patient blinding wasn't feasible due to the study design. Second, choosing a waitlist CG might overestimate treatment effect by delaying remission and help-seeking in the CG (e.g., Cuijpers et al., 2016). Still, in the present trial we decided that this is the most appropriate condition given our primary goal to evaluate the added value of iCBT for GAD in comparison to the current situation where treatment delays are the rule. As such, the true differential effects may in fact be slightly smaller with the extent of the overestimation remaining uncertain. To counteract this, CG patients were encouraged to seek additional help, minimizing perceived passivity. Third, the study sample was predominantly female (82.1 %), highly educated (42.6 % with university degrees), and relatively young (M = 35.2, SD = 11.6). This demographic might result from higher female anxiety disorder prevalence (Bandelow and Michaelis, 2015), women's increased help-seeking (Kessler et al., 1981), and recruitment methods (e.g., university and social media). Therefore, caution is warranted in applying findings to older or male populations. Still, existing studies show no barriers for males (Carl et al., 2020) or older individuals (Silfvernagel et al., 2018; Cremers et al., 2019) in online GAD interventions. Fourth, although the online course structure is standardized, we had no insight into how much time each patient spent on the course content. Though effect sizes were more pronounced for patients who finished at least four modules, still substantial effects were identified on the primary outcomes irrespective of the time spent on the course. Fifth, due to limited resources, we did not include a structured diagnostic interview post-treatment, so that we did not assess how many patients recovered, which would be an important additional assessment.
Despite these limitations, the present trial was comparably well-powered and provides additional support for the beneficial effects of iCBT without therapist support in GAD patients by striking a balance between external (e.g., conditions comparable to routine patients may have comorbid diagnoses, additional treatments) and internal validity (randomization and blinding of investigators). Further, there is little research on the effects of iCBT on mental health literacy, working ability, and health care use.
Approved by the ethics committee of the behavioral and empirical cultural science faculty at the study center at Heidelberg University (Ethics Committee-No. AZ Prüß 2021 1/1).
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Not applicable.
The study was funded by a European Regional Development Fund awarded to Selfapy. The Professorship for Adult Clinical Psychology and Psychotherapy at Osnabrück University was asked by Selfapy to independently evaluate its product to be assessed to the best of our knowledge and following the standards of good scientific practice. Selfapy made suggestions for the selection of survey instruments in the study's design but had no influence on the data collection, analysis, and interpretation or production of the resulting publications.
J. Rubel: Supervision, Methodology, Project administration, Writing – original draft. T. Väth: Writing – original draft, Formal analysis. S. Hanraths: Writing – original draft, Formal analysis. L. Pruessner: Writing – review & editing. C. Timm: Writing – review & editing. S. Hartmann: Writing – review & editing. S. Barnow: Writing – review & editing. C. Lalk: Formal analysis, Investigation, Software, Writing – original draft.
None.
J. Rubel, Email: julian.rubel@uni-osnabrueck.de.
L. Pruessner, Email: luise.pruessner@psychologie.uni-heidelberg.de.
C. Timm, Email: christina.timm@psychologie.uni-heidelberg.de.
S. Hartmann, Email: steffen.hartmann@psychologie.uni-heidelberg.de.
S. Barnow, Email: sven.barnow@psychologie.uni-heidelberg.de.
C. Lalk, Email: Christopher.lalk@uni-osnabrueck.de.