Validity and accuracy of the Whooley questions to identify symptoms of depression in Norwegian postpartum women

Authors: Kamilla Rognmo, Silje Haga, Susan Garthus‐Niegel, Catharina Elisabeth Arfwedson Wang, Malin Eberhard‐Gran

Abstract

Introduction

Screening for postnatal depression is widely acknowledged as an important public health initiative. The Whooley case‐finding questions are well suited for screening purposes in primary health care settings, as the instrument is quick and easy to administer. However, the validity and diagnostic accuracy among postpartum women remain unclear. The purpose of the present study was to evaluate the validity and diagnostic accuracy of the Whooley questions compared to the Edinburgh postnatal depression scale (EPDS) in a community sample of postpartum women in Norway. The diagnostic accuracy of the Whooley questions was examined across different EPDS thresholds and compared to the measures of related constructs, including symptoms of childbirth‐related post‐traumatic stress disorder (PTSD) and anxiety.

Material and Methods

Cross‐sectional data were collected through an online questionnaire by postpartum women (0–52 weeks postpartum), recruited via social media, well‐baby clinics, and other locations frequently visited by postpartum women. In total, 1154 women participated. The diagnostic accuracy of the Whooley questions was compared to three commonly used EPDS cutoffs (≥10, ≥12, and ≥13).

Results

The sensitivity of the Whooley questions relative to the EPDS was high and increased with higher thresholds for defining depression, correctly identifying 89% (EPDS ≥10), 96% (EPDS ≥12), and 97% (EPDS ≥13) of cases. Specificity was somewhat lower, at 0.82 (EPDS ≥10), 0.77 (EPDS ≥12), and 0.75 (EPDS ≥13). Positive predictive values were low, whereas negative predictive values were excellent, ranging from 0.97 (EPDS ≥10), through 0.99 (EPDS ≥12) to 1.00 (EPDS ≥13), increasing with higher thresholds. Convergent and divergent validity were supported by strong correlations with EPDS scores and moderate correlations with symptoms of childbirth‐related PTSD and anxiety.

Conclusions

The Norwegian version of the Whooley questions demonstrates strong psychometric properties, supporting their usefulness as a case‐finding tool for depression among postnatal women.

Key messageIdentifying time‐efficient case‐finding instruments for postpartum depression is imperative for early detection. The results of the present study show strong psychometric properties of the two‐item Whooley instrument, making it a suitable case‐finding instrument for symptoms of depression in primary health care.

INTRODUCTION

Depressive symptoms are widely recognized as a significant concern during the perinatal period. The estimated prevalence of antenatal depression ranges from 7.4% to 20%, ^1^ , ^2^ while it can rise to as much as 20% in the first 3 months postpartum. ^3^ Antenatal depression is associated with several adverse factors, including malnutrition, substance abuse, poor self‐reported health, low utilization of antenatal care services and negative neonatal outcomes. ^4^ Similarly, postnatal depression not only affects the mother but also has repercussions for her partner and family. ^5^ , ^6^ , ^7^ It can disrupt mother–baby interactions and has long‐lasting implications for the child's emotional and cognitive development. ^8^ , ^9^

To date, the Edinburgh Postnatal Depression Scale (EPDS) remains the most widely utilized screening tool for postnatal depression in numerous countries. The EPDS is a self‐report questionnaire comprising 10 items. Its application in prevention and psychological intervention is widely recognized as an effective method for identifying mothers experiencing postnatal depression. ^10^ In Norway, antenatal and postnatal health checks are typically conducted by primary healthcare professionals, including general practitioners (GPs), midwives, and public health nurses. Despite the relative simplicity of the EPDS, many healthcare professionals encounter time constraints and a lack of resources to adequately address perinatal mental health. ^11^ A recent study indicated that one‐third of new mothers in Norway were not asked about their mental health. ^12^ Consequently, there is a clear need for an alternative tool that is even more accessible and time efficient.

In this context, the Whooley case‐finding questions offer a significant advantage. ^13^ , ^14^ Comprising only two questions, the Whooley tool is brief, making it ideal for primary care settings where time is often limited. This streamlined approach not only facilitates the quick identification of potential mental health issues but also increases the likelihood of screening or case‐finding being conducted more consistently. Compared to the longer EPDS, the Whooley questions provide an efficient and effective option for healthcare professionals, ^13^ enabling them to swiftly identify mothers at risk and deliver timely interventions.

The National Institute for Health and Care Excellence (NICE) endorses the use of the Whooley questions for identifying symptoms of depression among women in the postpartum period. ^15^ These questions originate from the Primary Care Evaluation of Medical Disorders Procedure (PRIME‐MD) questionnaire, which has been validated for detecting depression in primary care settings. ^16^ A systematic review encompassing various populations, including men and non‐pregnant women, reported that the Whooley questions exhibited a pooled sensitivity of 0.95 and a pooled specificity of 0.65. ^17^ More recently, these questions have been validated for use in pregnant populations, demonstrating their effectiveness as a case‐finding tool for antenatal maternal depression. ^18^

To our knowledge, only a limited number of studies have examined the diagnostic accuracy of the Whooley questions in a postnatal sample, ^19^ , ^20^ , ^21^ with just one study conducted in a country comparable to Norway. ^22^ The results of these studies are somewhat mixed. One study compared the diagnostic accuracy of the Whooley questions to the EPDS and found that the Whooley questions exhibited acceptable accuracy for ruling out perinatal depression but demonstrated low sensitivity. ^21^ In contrast, the three studies that compared the case‐finding abilities of the Whooley questions to a diagnostic interview in postpartum samples reported sensitivity ranging from fair ^19^ to excellent, ^20^ , ^22^ though with poor specificity.

The objective of the present study was to evaluate the validity of the Whooley questions compared with the Edinburgh Postnatal Depression Scale (EPDS) in a community sample of postpartum women in Norway. As previous studies have compared the Whooley questions across different thresholds, a secondary objective was to examine the diagnostic accuracy of the Whooley items using multiple EPDS cut‐off scores. In addition, the study aimed to assess the divergent validity of the Whooley questions in relation to related constructs, including childbirth‐related post‐traumatic stress symptoms by the Impact of Events Scale, and anxiety symptoms by the Hopkins Symptom Checklist‐Anxiety subscale. Given the inconsistent findings from the limited number of previous studies conducted among postpartum women, this study seeks to provide more robust evidence regarding the utility of the Whooley questions in this population.

MATERIAL AND METHODS

Study population

Postpartum women residing in Norway were recruited to participate in an online cross‐sectional study conducted between August 2022 and May 2023. Recruitment took place through multiple channels, including social media platforms (Facebook, Instagram, and online parenting forums), the distributing of information at locations frequently visited by postpartum women (e.g., coffee shops, shopping malls, parks), and Maternal and Child Healthcare Centers (MCHC) during routine consultations. Participation was voluntary, and interested women accessed the online questionnaire via a link provided in the recruitment materials.

Eligible participants were Norwegian‐speaking women aged 18 years or older who had given birth within the past year. Apart from having Norwegian language proficiency, no additional exclusion criteria were applied. Informed consent was obtained electronically prior to participation. The questionnaire was administered through a secure online portal. Only participants with complete data on the relevant measures were included in the analyses. In total, 1154 women who met the inclusion criteria provided informed consent and were included in the study.

The study was reported in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines. Data were stored and managed in a secure environment provided by the Services for Sensitive Data (TSD). Data were de‐identified prior to analysis, and the re‐identification code was stored separately and securely.

Measures

Depression

The Whooley questions consist of two items that focus on the core symptoms of depression, designed to provide an easy and practical method for case‐finding in primary healthcare settings. ^13^ The questions are as (1) “During the past month, have you often been bothered by feeling down, depressed, or hopeless?” (yes/no), and (2) “During the past month, have you often been bothered by having little interest or pleasure in doing things?” (yes/no). An affirmative response to at least one question is considered a positive screen. The translated Whooley questions are available in Figure S1.

The EPDS is a widely used screening tool designed to identify postpartum depression. The EPDS measures depressive symptoms experienced over the past week and comprises 10 items, each rated on a 4‐point scale from 0 to 3, yielding a maximum possible score of 30. ^23^ In the present study, three different cutoffs for the EPDS were a score of ≥10 indicating symptoms of minor depression (referred to as EPDS10), a score ≥12 signifying moderate depression (EPDS12), ^23^ , ^24^ and a score ≥13 indicating a strong probability of major depression (EPDS13). ^25^

Anxiety

Symptoms of anxiety were measured using the Norwegian translation of the 10‐item anxiety subscale of the Hopkins Symptom Check List (referred to as SCL‐A hereafter). ^26^ , ^27^ Each item was rated on a four‐point scale, ranging from ^1^ (“not bothered”) to ^4^ (“extremely bothered”). The total score for the 10 items ranges from 10 to 40, with higher scores indicating more severe anxiety symptoms. In the present study, a score of 18 was defined as a positive screen, consistent with previous findings. ^28^

Childbirth‐related post‐traumatic stress

The Impact of Event Scale–Revised (IES‐R) was utilized to measure childbirth‐related post‐traumatic stress symptoms. ^29^ The IES‐R is a self‐rating scale that measures symptoms of intrusion (seven items) and avoidance (eight items) connected to a specific situation, in this case childbirth. It features four response categories (0 = not at all, 1 = rarely, 3 = sometimes, and 5 = often). Values above 35 on the summative scale are likely indicative of a post‐traumatic stress disorder (PTSD) diagnosis. ^30^ Hereafter, the scale is referred to as IES35, indicating the threshold for defining a likely childbirth‐related PTSD diagnosis.

Statistical analyses

All analyses were conducted using IBM SPSS Statistics (Version 29). The criterion validity of the Whooley questions was assessed in comparison to EPDS10, EPDS12, and EPDS13 using 2 × 2 contingency tables. Key metrics were calculated, including sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and the likelihood ratio for a positive test.

Receiver Operating Characteristic (ROC) and Area Under the Curve (AUC) were estimated to evaluate the discriminatory ability of the Whooley questions. The AUC plot provides a graphical representation of the false positive rate (1‐specificity) and the true positive rate. The graph was generated with Python using Maplotlib.

Convergent and divergent validity of the Whooley questions were evaluated by comparing to the EPDS, IES35, and SCL‐A (both binary and continuous variables) using correlation analyses. Associations between binary and continuous variables were measured using point‐biserial correlations, while phi coefficients were employed for associations between binary variables. Pearson correlations were calculated for continuous variables. Reliability was measured using Cronbach's alpha.

RESULTS

Descriptive statistics

Sample characteristics are presented in Table 1. The majority of the women were married or cohabiting with a partner. Most participants had a high level of education and incomes at or above the national average. The mean age was 31.7 years, which is comparable to the mean age of women giving birth in Norway. ^31^ Nearly 99% of participants had singleton pregnancies, consistent with national statistics. ^31^ The mean gestational age was 39 weeks (standard deviation (SD) = 1.93 weeks), indicating that most births occurred at term. The average time since birth at participation was approximately 20 weeks.

Overall, 32.1% of participants responded affirmatively to at least one of the Whooley questions. Nearly 20% of participants responded ≥10 on the EPDS, with 12.4% scoring ≥12 and 9.9% scoring ≥13 on the EPDS. Childbirth‐related post‐traumatic stress symptoms above the cut‐off on the IES‐35 were reported by 8.5% of participants, while anxiety symptoms above the SCL‐A cut‐off were reported by 15.4%. Frequencies, means, and standard deviations for all symptom scales are displayed in Table 2.

Validity of the Whooley questions compared with EPDS, IES‐35, and SCL‐A

The sensitivity of the Whooley questions relative to EPDS was high and increased with higher EPDS thresholds, correctly identifying between 89% (EPDS ≥10), 96% (EPDS ≥12), and 97% (EPDS ≥13) of the cases. In contrast, specificity was lower than sensitivity, with the highest specificity observed at EPDS ≥10 (0.82) and the lowest for EPDS ≥13 (0.75). Positive predictive values (PPVs) were low, indicating that individuals scoring positive on the Whooley questions had a 30 to 55% probability of also screening positive on the EPDS. Negative predictive values (NPVs) were excellent, ranging from 0.97 to 1.00, and increased with higher EPDS thresholds. These results are displayed in Table 3. The area under curve (AUC) ranged from 0.85 to 0.86, indicating good discriminatory ability (see Figure 1).

When compared with the IES‐35, the sensitivity of the Whooley questions was 0.86 and specificity was 0.73. The PPV was low, with only 23% of participants scoring positive on the Whooley questions also reporting clinically significant childbirth‐related post‐traumatic stress symptoms. In contrast, the NPV was high, indicating that 98% of participants scoring negative on the Whooley questions did not report such symptoms. The AUC was 0.79, reflecting fair discriminatory abilities. In comparison with the SCL‐A, the sensitivity of the Whooley questions was 0.80 and the specificity was 0.77. The PPV remained relatively low, with 39% of participants scoring positive on the Whooley questions also screening positive for anxiety. The NPV was high at 0.96. The AUC of 0.78 also indicates fair discriminatory ability (see Figure 2). Results are presented in Table 4.

Reliability, convergent and divergent validity of the Whooley questions

Cronbach's alpha for the Whooley questions was 0.78, indicating acceptable internal consistency, particularly given that the scale consists of only two items. Strong correlations between the Whooley questions and the EPDS support convergent validity. Divergent validity was assessed examining correlations between the Whooley questions and scores on the IES‐35 and SCL‐A. As expected, these correlations were moderate, indicating that while the constructs overlap clinically, the Whooley questions measure a distinct construct. Correlation coefficients are presented in Table 5.

Sensitivity analyses

Supplementary sensitivity analyses were conducted to assess whether case‐finding accuracy varied according to time since childbirth. Participants were categorized using both quartiles and clinically relevant postpartum time intervals (0–6, 6–12, 12–24, and 24–52 weeks). The results are provided in the Tables [Link], [Link], [Link], [Link], [Link], [Link]. The case‐finding accuracy of the Whooley questions remained high across all groups, with only minor, non‐systematic variations. A slightly lower sensitivity was observed in the earliest postpartum group; however, this subgroup was small, and the estimate was highly sensitive to small changes in classification. Given the absence of a consistent trend and the instability of subgroup estimates, it is unlikely that the performance of the Whooley questions differs meaningfully by postpartum timing. These findings support the use of the Whooley questions throughout the first year postpartum.

DISCUSSION

The validity estimates of the Whooley questions indicate that the Norwegian version is an effective case‐finding tool for depression symptoms among postnatal women, with performance comparable to that of the EPDS. The Whooley questions identified the majority of individuals who screened positive on the EPDS, while a negative result strongly suggested the absence of depressive symptoms. Furthermore, the overall discriminatory ability of the Whooley questions, as reflected by the area under the curve (AUC), was good, indicating that a positive case‐finding result effectively differentiated between women with and without depressive symptoms as defined by the EPDS.

For case‐finding purposes, it is essential that an instrument correctly identifies individuals with a high likelihood of depression. Accordingly, sensitivity and negative predictive value (NPV) are important performance indicators, as high sensitivity reduces the risk of missed cases, while a high NPV increases confidence that individuals screening negative are unlikely to have screened positive on the EPDS. The high sensitivity observed in the present study is consistent with previous research on the validity of the Whooley questions in postpartum samples. Mann et al. ^32^ and Navarrete et al. ^20^ reported sensitivity values of 1.00, which are comparable to the sensitivity of 0.97 observed at the EPDS ≥13 threshold in the present study. Notably, specificity values in the current study were higher than those reported in several earlier studies.

The NPVs observed in this study were excellent, indicating that women who screened negative on the Whooley questions were highly unlikely to screen positive for depression on the EPDS. Only two previous studies have reported NPVs in postpartum samples, ^19^ , ^21^ with values comparable to, though slightly lower than, those found here. Taken together, the present findings and existing literature suggest that the Whooley questions is a well‐suited case‐finding tool for depression symptoms during the postpartum period. Moreover, the results are comparable to the pooled sensitivity (0.95) and specificity (0.65) reported in a meta‐analysis examining the diagnostic accuracy of the Whooley questions in the general population. ^17^

In contrast, the low positive predictive values (PPVs) indicate that a substantial proportion of the women screening positive on the Whooley questions may not screen positive on the EPDS. In the present study, only 30 to 55% of women who screened positive on the Whooley questions also screened positive for depression on the EPDS. These findings are consistent with previous studies ^19^ , ^20^ , ^21^ and highlight the importance of follow‐up assessment. Consequently, positive screening results should be followed by more comprehensive assessment tools, such as the EPDS or structured clinical interviews, which offer greater specificity and are better suited for assessing symptom severity and confirming a diagnosis.

The convergent validity of the Whooley questions was established through the strong correlation with the EPDS, while divergent validity was indicated by the moderate correlations with childbirth‐related post‐traumatic stress and anxiety. There is a clinical overlap between postnatal depression and childbirth‐related post‐traumatic stress, ^33^ as well as between postnatal depression and anxiety, ^34^ making moderate correlations between the Whooley questions and IES35 and SCL‐A expected. Healthcare professionals should be aware that women who screen positive on the Whooley questions during the postpartum period may also exhibit symptoms of childbirth‐related post‐traumatic stress and anxiety, in addition to core symptoms of depression.

Different cutoffs on the EPDS correspond to variability in severity of depressive symptoms. Examining how the Whooley questions perform at these different EPDS cutoffs allows for a more robust and comprehensive understanding of their validity, as well as facilitating comparisons with previous and future research. Additionally, the clinical utility of the Whooley questions may be enhanced, as the variations in the sensitivity‐to‐specificity ratio reflect the strictness of the EPDS criteria.

The Whooley questions are effective in broad case‐finding settings, such as primary health care or well‐baby clinics, but are also suitable for detecting more severe cases of depression symptoms, as demonstrated by their high sensitivity when compared to the EPDS with a cutoff at ≥13. Regardless of the EPDS cutoff, the diagnostic accuracy of the Whooley questions remains high, strongly indicating that they are an appropriate tool for assessing depressive symptoms during the postnatal period.

A major strength of the present study is the large, community‐based sample of over 1000 postpartum women. Additionally, comparing the Whooley questions to a well‐established, valid, and reliable measure of perinatal depression enhances the study's robustness. The assessment of the diagnostic accuracy of the Whooley questions across different thresholds for defining depression aligns with best practices in test validation. To the best of our knowledge, this study is the first of its kind conducted in Norway and only the second in a European context.

However, there are some limitations of the present study. Although the EPDS is a well‐established measure of depression, it is not intended to be used as a diagnostic tool. As such, using the EPDS as the gold standard of depression is not ideal. However, in the context of this study, which aims to explore the use of the Whooley questions as a case‐finding tool for postnatal depression, using the EPDS as the reference is appropriate. Parts of the sample were recruited through online channels, which may limit the generalizability of the sample. However, the majority of women were recruited in locations where postpartum women tend to visit, which in turn may counteract some of the uncertainties regarding the representativeness of the sample. Only 3% of the women in our sample were not in cohabiting relationships. Although the sample was similar to postpartum women on most parameters, the percentage with university education was higher than the general population of women of similar age groups ^35^ and income levels were average or above average. Thus, it is possible that the sample on average may have been better equipped to correctly interpret and respond to the Whooley questions. This may limit the generalizability of the findings to populations with lower educational or socioeconomic backgrounds. The performance of the Whooley questions was not shown in relation to time since giving birth. However, the sensitivity analyses presented in the supplementary material suggest that the performance of the Whooley questions is not dependent upon time since birth.

CONCLUSION

The Norwegian version of the Whooley questions demonstrates strong psychometric properties, making them a valuable tool for healthcare professionals in identifying depression among postnatal women. They can effectively serve as an initial case‐finding tool in primary care settings, focusing on identifying potential cases rather than confirming diagnoses, which can be carried out using other established methods to assess the severity of depression in primary or specialized healthcare services. A screen positive test result should prompt further assessment using comprehensive instruments enabling the differentiation between postpartum depression, anxiety symptoms, and childbirth‐related traumatic stress. This stepped approach may help the allocation of healthcare resources more efficiently and ensure that women with mental health challenges receive appropriate and timely support.

AUTHOR CONTRIBUTIONS

Silje Haga, Susan Garthus‐Niegel, and Malin Eberhard‐Gran contributed to the study conception, design, and data collection. Kamilla Rognmo performed the statistical analyses. Catharina Elisabeth Arfwedson Wang contributed to study conception and drafting of the manuscript. The first draft of the manuscript was written by Kamilla Rognmo and Malin Eberhard‐Gran, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

CONFLICT OF INTEREST STATEMENT

The authors declare that they have no conflicts of interest.

ETHICS STATEMENT

Ethical approval was obtained by the Norwegian Regional Committees for Medical and Health Research Ethics (project 275370) on September 17, 2021, and this study was preregistered at https://osf.io/jrqvs. The study was performed in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki. All participants consented to participate in the study and provide data to be published in peer‐reviewed journals.

Supporting information