Authors: Bonnie Mowinski Jennings, Katherine A. Yeager
Categories: Article, Checklists, Mentorship, Methodology, Nursing research, Qualitative research, Saturation
Source: International Journal of Nursing Studies Advances
Authors: Bonnie Mowinski Jennings, Katherine A. Yeager
The term saturation is used ubiquitously in reports of qualitative research. The term is often used, however, as a throw-away line, sans definition, rationale, or explanation of the process. Consequently, there remains a question about what is meant when saturation is used.
The concept of saturation originated in grounded theory. Achieving theoretical saturation is one of the tenets of grounded theory. Over time, this specific and singular term has migrated to a general term that reflects various types of saturation such as code, data, theme, and meaning saturation. There is a lack of clarity among saturation types and a lack of transparency in reporting what is meant by saturation and how it was achieved. There is also a question about the relevance of saturation to qualitative methodologies outside grounded theory.
In seeking to raise awareness of the issues surrounding the term saturation, in this discussion paper we offer a synopsis of the history and evolution of the term saturation, address the current conundrum about saturation in qualitative research, and examine the use of saturation in qualitative articles published in the six nursing journals with the highest journal impact factors. This re-view reflects the need to critically assess the status of saturation as it is used in nursing journals. Better definitions of terms are needed to assure consistency in language that will enhance understanding for the end user of qualitative research reports. Those conducting and assessing qualitative inquiries need clear understanding of what is appropriate for each qualitative methodology—saturation does not apply to some qualitative approaches.
Mentorship and teaching are at the heart of the saturation issue. Finding a mentor who is a solid methodologist may require nurses to reach out to individuals in other disciplines such as sociology, psychology, or anthropology. In the absence of a mentor, investigators conducting qualitative inquiry are urged to become well-versed in the methodology they believe is best suited to their question by turning to original sources written by well-trained methodologists. It is imperative that students who may become educators or researchers who then become reviewers and perhaps journal editors have a solid understanding of key qualitative concepts such as saturation.
What is already known •In all forms of qualitative inquiry, the term saturation is broadly discussed.•There are conceptual and methodological problems surrounding the use of saturation. Alt-text: Unlabelled box What this paper adds •Saturation was examined in 152 qualitative articles published in 6 nursing journals. Saturation was mentioned in 68 of the articles, used as sample size justification in 40 reports, and defined by authors of only 19 articles.•Editorial policies regarding checklists may support reporting qualitative studies in a manner that is inauthentic to the methodology used. Authors may lean on checklists to justify the credibility of their work rather than conform to methodological tenets.•Nursing students, educators, researchers, reviewers, and journal editors are called upon to have a firmer grasp of the meaning of saturation and use it appropriately. Alt-text: Unlabelled box
Using a tongue-in-check style, Thorne and Darbyshire (2005) took aim at an assortment of “land mines” (p. 1105) they regarded as “the most problematic and troublesome” (p. 1107) issues in qualitative inquiry. Their goal was to offer guidance for enhancing the rigor and quality of empirical science in qualitative research. The advice offered by Thorne and Darbyshire is a gold mine. They illuminated the missteps that have occurred, and continue to occur, in studies conducted in the qualitative tradition. Theoretical saturation was among the issues they identified dubbing it “the wet diaper” “Surprisingly, not in the exclusive domain of grounded theorists, this phenomenon represents the claim of ‘theoretical saturation’ (that is no new information will arise from further sampling)…the saturation claim is often invoked as a convenient stopping point” (p. 1108). Almost 20 years later, the saturation land mine remains.
It is our impression that authors, at times, use a throw-away line about achieving saturation (generally sans definition, rationale, or explanation of the process). Saturation is “expected” (O'Reilly & Parker, 2012, p. 196) in qualitative reporting. Phrases such as “gold standard” (Guest, 2006, p. 60), “mantle of rigor,” and “flagship of validity” (Constantinou et al., 2017, p. 2, p. 15) contribute to researchers asserting they achieved this goal. Saunders et al. (2018), in their treatise on how the meaning of saturation has shifted, noted that saturation has reached the “status of orthodoxy” (p. 1894) despite the conceptual and methodological problems surrounding it.
The emphasis on seeking saturation is further illustrated by authors who propose using mathematical models to determine the sample size needed for saturation a priori (e.g., Fofana et al., 2020; Guest 2020; van Rijnsoever et al., 2017; Weller et al., 2018). These models cannot account for “the complexities inherent in any human…experience” (Thorne & Darbyshire, 2005, p. 1108). They also ignore various assumptions inherent to qualitative inquiry. As expressed by Hennink and Kaiser (2022), “many of the formulas developed [in the mathematical models] are based on implicit assumptions that do not align with the conduct of qualitative research” thereby potentially yielding “a misleading veil of scientific authenticity” (p. 7).
Thorne (2020) noted that a methodological debate about saturation has surfaced in qualitative literature. Specific to the nursing discipline, Thorne questioned “when we invoke the sacred language of ‘saturation,’ what is it that we really mean?” (p. 3). It is our goal, therefore, to join the conversation about saturation in qualitative research as it applies to studies published in nursing journals. This conversation unfolds in three parts. First, we synopsize the history and evolution of the term saturation. Then we address the current conundrum about saturation in qualitative research. Finally, we examine the use of saturation in qualitative articles published in the six nursing journals with the highest journal impact factors.
Many authors who discuss saturation in qualitative inquiry appropriately note the term is rooted in grounded theory (e.g., Bowen, 2008; Braun & Clarke, 2021; Low, 2019; Saunders et al., 2018). What is missing from some articles, however, is information about what constitutes grounded theory. By naming only the methodology without also mentioning its purpose, key ideas may be lost. We therefore begin by shifting the gaze from the mere mention of grounded theory to a few reminders about what grounded theory involves, including the place of saturation.
Grounded theory methodology was developed by two sociologists, Glaser and Strauss (1967). An interpretive form of knowledge generation with philosophical roots in symbolic interactionism and pragmatism (Corbin, 2009; Wuest, 2007), grounded theory had its inception in the 1960′s at the University of California, San Francisco (Baker et al., 1992; Benoliel, 1996; Stern, 2009). Jeanne Quint Benoliel, a nurse, worked with Glaser and Strauss in their investigation about the process of dying among patients in California hospitals (Stern & Covan, 2001; Stern, 2009). Their research yielded several classic books, the first of which was Awareness of Dying (Glaser & Strauss, 1965). After writing Awareness of Dying, Glaser and Strauss realized they had used a different analytic approach by generating theory inductively from the data (Stern, 2009). This realization was the catalyst for their seminal book, The Discovery of Grounded Theory: Strategies for Qualitative Research (Glaser & Strauss, 1967).
The purpose of grounded theory is, as the name suggests, to inductively generate a theory from the data. The theory is typically a middle range theory that explains human behaviour within a social context (Charmaz, 2008; Glaser, 1978; Morse, 2001; Wuest, 2007). Rather than confirming or disconfirming existing theories, grounded theory is designed to propose new theories. It is also important to note that grounded theory is “a way of thinking about and conceptualizing data” (Strauss & Corbin, 1995, p. 275). Understanding grounded theory methodology, however, is complicated by the sociological language used by Glaser and Strauss, a language unfamiliar and potentially confusing to many individuals outside the discipline of sociology, including nurses (Stern, 1980, 1985; Wuest, 2007).
What is not complicated by sociological terminology, but what is often overlooked, is that data for theory development come from multiple sources. Along with interviews and field work, data may be generated by observations, documents, video recordings, correspondence, news articles, and books (Glaser & Strauss, 1967). Data include “anything that might shed light on the area of questions under study” (Corbin & Strauss, 1990, p. 5). Diverse data sources are relevant to many forms of qualitative inquiry. In phenomenology, however, only individuals who have experienced the topic under investigation are appropriate data sources (Baker et al., 1992; van Manen, 1984). These individuals might be interviewed, observed, or their diaries read but all data are from only the individual whose experience is being studied (van Manen, 1984).
There are several principles comprising grounded theory. These tenets include collecting, coding, and analysing data concurrently; using constant comparison during analysis; applying various coding strategies; writing memos; sampling theoretically; achieving theoretical saturation; and writing the final product at a theoretical level (Glaser & Strauss, 1967; Schreiber, 2001; Stern & Covan, 2001; Wuest, 2007). Adherence to these tenets is essential to produce a product that is, in fact, a grounded theory.
Researchers, however, may claim to have generated grounded theory when they have followed only some of the procedures or used the procedures incorrectly (Corbin & Strauss, 1990; Wuest, 2007). For instance, some individuals focus only on coding as the defining feature of grounded theory. This stance is illustrated by Kuziemsky et al. (2007) who claimed the “hallmark of grounded theory is three systematic coding open, axial and selective coding” (p. 143). Not only does such a perspective ignore other tenets and the goal of developing theory, but it fails to include an essential form of coding in grounded theory, theoretical coding (Charmaz, 2006; Glaser, 1978; Stern, 1985; Strauss & Corbin, 1995).
Despite the importance of all the essential features of grounded theory, our focus here is on theoretical saturation, sometimes called theoretical redundancy (Schreiber, 2001) or theoretical sufficiency (Dey, 1999). Theoretical saturation focuses on category development because categories are used to develop the theory. “…categories are ‘saturated’ when gathering fresh data no longer sparks new theoretical insights, nor reveals new properties of your core theoretical categories” (Charmaz, 2006, p. 113). In other words, theoretical saturation is manifest when the categories and theory are fully developed and no new information about the core processes is discovered in further data collection (Schreiber, 2001). Thus, achieving theoretical saturation depends more on the quality of the data and less on sample size (Bowen, 2008). Charmaz (2008) noted that theoretical saturation is widely claimed yet rarely realized.
Theoretical sampling, a precursor to theoretical saturation (Benoliel, 1996), concerns how data are collected as a grounded theory study progresses (Glaser & Strauss, 1967; Morse & Clark, 2019). Because collecting, coding, and analysing data occur concurrently in grounded theory, the researcher uses the evolving findings/theory to determine what data to collect next. Only initial data collection can be planned in advance. The “next steps” in grounded theory data collection are determined by the “emerging theory. …The basic question in theoretical sampling … what groups or subgroups does one turn to next in data collection? And for what theoretical purpose?” (Glaser & Strauss, 1967, p. 47). In other words, early in the study, the investigator seeks informants who have broad knowledge of the topic. Later in the study, the investigator seeks to sample those who have knowledge specific to concepts that are relevant to the developing theory (Conlon et al., 2020). Morse (2015) described the connection between theoretical sampling and theoretical saturation as “bring[ing] unanswered questions, ‘holes’ in the analysis back to the participants” (p. 588) to ultimately achieve theoretical saturation. The link between saturation and theoretical sampling is addressed in more detail later in this article.
The importance of theoretical sampling underscores how mathematical models are not suitable for determining sample size in grounded theory. By contrast, Conlon et al. (2020) depicted how theoretical sampling was used in a grounded theory study of hoping and coping among 16 people with malignant glioma conducted by O'Keefe. It was after 43 interviews among the same 16 participants that theoretical saturation was achieved. Thus, the number of interviews among knowledgeable participants based on theoretical sampling was the salient source of information that led to theoretical saturation—having sufficient data to develop the theory—not the number of participants per se.
Over time, grounded theory has become “a compendium of different methods” (Corbin, 2009, p. 41) yielding a “Genealogy of Grounded Theory” (Morse, 2009, p. 17). The differences were first manifested when Glasser and Strauss began to diverge in their thinking, giving rise to Glaserian and Straussian grounded theory (Stern, 1994). Corbin, a student of Strauss, contributed to developing Straussian grounded theory (Morse, 2009). Other types of grounded theory followed, each developed by students of Glaser and/or Strauss: dimensional analysis (Bowers & Schatzman, 2009), constructivist grounded theory (Charmaz, 2006, 2008, 2009), and situational analysis (Clarke, 2005, 2009). Wuest (2007) and Morse (2020) noted that such methodological shifts and changes are expected as science evolves. Although the different approaches to grounded theory share some procedures in common, each has unique methodological tenets (Corbin, 2009). Thus, Wuest (2007) underscored the importance of researchers declaring the type of grounded theory they are conducting. Regardless of type, however, theoretical saturation is a tenet across grounded theory methodologies (Charmaz, 2006).
Remaining true to the philosophical and procedural differences among the types of grounded theory is important to remain true to methodology and ensure rigor. Nonetheless, some reports illustrate a blend of features from the various types of grounded theory, yielding a product that is less rigorous. For example, in a review of 134 articles published in three nursing journals in which authors claimed using grounded theory, McCrae and Purssell (2016) found instances in which the various types of grounded theory were mixed in the same report.
In addition to mixing grounded theory approaches, Benoliel (1996) noted a shift away from the original tenets of grounded theory. In an analysis of 146 abstracts of grounded theory studies, Benoliel found that authors tended to focus on select grounded theory procedures rather than the full research methodology, giving rise to the phrase “grounded theory approach” (p. 412). Moreover, Benoliel's analysis showed that nurse investigators typically relied on interviews as the singular data collection technique.
Just over a decade later, Kearney (2009), a former student of Glaser and Strauss, showed further evidence of the shift from the tenets of grounded theory. Comparing dissertations from medical sociology and practice professions (e.g., nursing, social work) in which students claimed to conduct grounded theory, Kearney found the work from sociology was more congruent with the grounded theory tradition. Kearney speculated the shift might be attributable to the sociology mentors who had a strong history of accomplishments in qualitative work compared to the practice professions where faculty mentors had less experience with qualitative inquiry. In 2020, Thorne noted a continuation of this shift by underscoring the use of saturation to justify sample size, and thus stop data collection, rather than staying true to theoretical saturation for the purpose of discovering theory.
It is important to revisit the words of Glaser and Strauss (1967) who did indeed address sample size—but as it pertained to theoretical category “The criterion for judging when to stop sampling … is the category's theoretical saturation … [when] no additional data are being found whereby the [investigator] can develop properties of the category” (p. 61). It is noteworthy that when Glaser and Strauss wrote The Discovery of Grounded Theory, there was only one form of saturation—theoretical. At that time, the only meaning ascribable to saturation was theoretical. That has changed.
The “ubiquitous” use of saturation (Caelli et al., 2003; King, 2021; O'Reilly & Parker, 2012; Thorne, 2020) reflects a failure to remain true to the original meaning of saturation within the grounded theory methodology (Caelli et al., 2003; Thorne, 2020). As theoretical saturation has diffused into other qualitative approaches (e.g., Caelli et al., 2003; Low, 2019; O'Reilly & Parker, 2012; Vasileiou et al., 2018), there has been “… limited discussion of its meaning” (Caelli et al., 2003, p. 7). What once was a specific term has become a general term. What once was an essential feature for inductively developing a theory has become a holy grail for justifying sample size. Saturation has become a problematic term.
Morse, who is widely cited regarding saturation, seems to implicitly refer to grounded theory throughout her publications about saturation. For example, Morse wrote “The more complete the saturation, the easier it is to develop a comprehensive theoretical model” (1995, p. 149). Moreover, Morse (2000), refutes a simplistic view of saturation in "saturation depends on a number of factors, including the quality of data, the scope of the study, the nature of the topic, the amount of useful information obtained from each participant, the number of interviews per participant, the use of shadowed data [where participants talk of others’ experiences in addition to their own], and the qualitative method and study design used." (p. 3)
Low (2019), a sociologist, regards using saturation to determine sample size as “contentious” (p. 134). Lost in the focus on sample size, typically reflected in the number of participants, are important issues concerning data quality (Wuest, 2007). More than three decades ago, Morse (1989) proposed the concepts of appropriateness and adequacy, both of which apply to data quality, as guides to sample size. Appropriateness reflects participants’ knowledge of the topic under investigation. In grounded theory, appropriateness shifts as the study progresses and theoretical sampling comes into play. Adequacy refers to the “sufficiency and quality of the data obtained” (Morse, 1989, p. 123) thereby reflecting “saturation” (Morse, 1989, p. 123). Adequacy is about “relevance, completeness, and the amount of information [italics added] obtained…Data are adequate [when] there are no ‘thin’ areas” (p. 123). Thus, saturation is more about the data (Morse, 2006a, 2006b) than the number of participants. Moreover, data quality can be enhanced by completing multiple interviews with select participants “to expand the depth or address gaps in the emerging analysis” (Morse, 2002, p. 16). Such an approach does not expand the sample size as reflected by the number of participants.
As previously noted, in the early days of grounded theory saturation did not require an adjective because it always referred to theoretical saturation. Although Glaser and Strauss (1967) did use saturation as the sole criterion to signal when to stop sampling, what is sometimes lost is their point about the adequacy of data to develop category properties. Saturation is about category development because categories are used to develop the theory (Charmaz, 2006).
As saturation migrated into other qualitative methodologies, the term became more tightly aligned with sample size (Hennink et al., 2017). Concerns about data quality were lost in the transition. As noted previously, the mention of saturation is now “expected” (O'Reilly & Parker, 2012, p. 196) in qualitative reporting, having reached the “status of orthodoxy” (Saunders et al., 2018, p. 1894).
Over time, four prevailing types of saturation in qualitative methodologies other than grounded theory have taken code saturation, data saturation, theme saturation, and meaning saturation. Informational redundancy, when “no new information, codes or themes are yielded from the data” (Braun & Clarke, 2021, p. 202) is reflected in three types of saturation. Code saturation is redundancy in the codes, data saturation is redundancy in the data, and thematic saturation is redundancy in the themes. Coding involves labelling data (Stern, 1985). Hennink et al. (2017) questioned the sufficiency of code saturation because codes identify issues within segments of data, thereby lacking analytical depth. Nonetheless, code saturation has joined the saturation terminology. Of note, Saunders et al. (2018) posit that data saturation is more about the amount of data, often reflected in the number of interviews. As previously stated, the number of interviews may not reflect data adequacy. Authors have appropriately argued that using informational redundancy as an indicator of saturation is problematic because new insights will most likely arise as new data are collected from different individuals (Low, 2019; Morse, 1989).
By contrast, meaning saturation, introduced by Hennink et al. (2017), refers to when investigators “fully understand the issues” (Hennink et al., 2017, p. 605). Meaning saturation seems to reflect having data that are appropriate and adequate, thereby supporting analytical depth. Meaning saturation may therefore resemble theoretical saturation because investigators have generated data that allow developing a deep understanding of the topic under investigation. Meaning saturation, as it pertains to phenomena nurses study, is exemplified by ongoing interest in dying in institutions (e.g., hospitals, homes, long-term care). The crux of meaning saturation requires nurses to conduct intensive fieldwork using multiple data sources to uncover processes that are not yet known. This will yield new theories, just as Glaser and Strauss (1965) discovered a theory involving four types of awareness surrounding dying—open, closed, suspicion, and mutual pretense.
The lack of clarity regarding saturation types has prompted authors to propose alternate phrases to replace saturation in qualitative inquiry outside grounded theory. These alternate phrases differ in name but share the desire to guide qualitative researchers to focus on the quality of data and the depth of their analyses rather than sample size as reflected in the number of participants. These terms include information power (Malterud et al., 2016) and conceptual density or depth (Nelson, 2017). Just as there is confusion and conflation among the more traditional terms used in combination with saturation, these alternate terms may be misused by researchers who focus on sample size. Nonetheless, these alternate terms seem to better reflect the goal of having good quality data that support researchers conducting an in-depth analysis.
Regardless, numbers still dominate thinking, even among some qualitative researchers, and the number of participants is easier to address than the quality of the data and depth of the analysis. What also may dominate is a more descriptive rendering of the data (Wuest, 2007) rather than the creative thinking and conceptualizing that are inherent in qualitative analysis (Morse, 1999; Strauss & Corbin, 1995).
“Having the researcher simply state that data were saturated is not an indicator [of saturation]” (Morse, 1999, p. 164). Numerous authors have addressed the lack of transparency in reporting what saturation means and how it was achieved (e.g., Caelli et al., 2003; Hennink & Kaiser, 2022; Malterud et al., 2016; Morse, 1995, 2000, 2015). One way to achieve transparency is to write for the “critical reader” and “… articulate…the logical processes” used to develop findings (Thorne, 2000, p. 70).
The absence of transparency is exemplified in reviews completed by Francis et al. (2010) and Vasileiou et al. (2018). Francis and colleagues analysed articles describing qualitative studies published over 16 months in Social Science & Medicine. Data saturation was mentioned in 18 articles with investigators of 15 reports claiming that data saturation was achieved. It was unclear, however, how data saturation was justified. Similarly, Vasileiou and colleagues reviewed 214 articles covering a 15-year span from three well-respected health-related journals (i.e., psychology, sociology, medicine) to assess sample size sufficiency. Saturation was most used to justify sample size yet, here too, saturation was not substantiated. Vasileiou and colleagues called for researchers to offer more transparency in reporting and for reviewers to expect transparency. Although it may be challenging for researchers to show how categories were saturated in grounded theory or that information was redundant in other qualitative methodologies, it is possible to identify and define the type of saturation and show the process used to achieve so-called saturation.
The relevance of saturation to studies based on philosophical assumptions and tenets outside grounded theory is debatable (e.g., Caeilli et al., 2003; Charmaz, 2008; Malterud et al., 2016; O'Reilly & Parker, 2012). For instance, Braun and Clarke (2021) questioned whether data saturation is useful in reflexive thematic analysis. More unequivocally, van Manen stated “there is no saturation point with respect to phenomenological meaning” (van Manen, et al., 2016, p. 5).
Nonetheless, many reviewers who serve as referees for journals expect to see mention of saturation in manuscript submissions based on qualitative inquiry. O'Reilly and Parker (2012) illustrated this expectation by describing reviewer critiques of an article they wrote based on conversation analysis. Along with receiving constructive comments, they were criticized for failing to mention saturation. This critique was not viewed as appropriate by O'Reilly and Parker because saturation is irrelevant as a sign of quality (reflected by sample size) in conversation analysis.
Various authors have pointed to expectations of reviewers, editors, and criteria in journal guidelines as reasons for the proliferation of saturation outside the grounded theory tradition. Journal requirements are likely fuelled by the desire to implement criteria for evaluating the validity of qualitative studies (Popay et al., 1998). Thorne (2020) noted that saturation is commonly addressed in published guidelines for qualitative reporting, guidelines used by various nursing journals that also appear in methods texts as a signal for stopping data collection.
Checklists are designed as reporting guidelines yet they are often interpreted as quality assessments. This conflation further complicates issues surrounding the saturation conundrum. For instance, Hennink and Kaiser (2022) posited that checklists may contribute to vagueness in reporting when authors simply state that saturation was reached, a statement, as previously noted, that Morse (1999) has deemed inadequate. Morse, 2020, Morse, 2021 articulated rationale for not using checklists in the review process. Yet checklists abound as shown by Munthe-Kaas et al. (2019) who found 102 critical appraisal tools specifically designed to assess qualitative research.
Commonly used among nursing journals are the EQUATOR guidelines that rely on the Consolidated Criteria for Reporting Qualitative Research (COREQ) checklist developed by Tong et al. (2007) and the Standards for Reporting Qualitative Research (SRQR) checklist developed by O'Brien et al. (2014). Morse (2020) has noted that no one list is suitable for all qualitative methods. Moreover, the COREQ guidelines explicitly pertain to interviews and focus groups, both of which are data collection techniques, not methodologies. In addition, the focus of the COREQ guidelines is data saturation defined as recruiting participants until “no new relevant knowledge was being obtained” (p. 356). The SRQR guidelines ask authors to provide details about “important processes,” processes that include saturation (p. 1246) as well as the rationale for terminating sampling. These details seem congruent with the goal of transparency, yet the details may be glossed over by authors and reviewers who seek, instead, to verify that a box was checked. Checking a box on a checklist is far adrift from achieving any type of saturation.
To assess the use of the term saturation in research articles published in nursing journals, we collaborated with an information librarian who is a search strategy expert. We began by identifying the nursing journals with the highest impact factors. This approach was congruent with work reported by McCrae and Purssell (2016) and Vasileiou et al. (2018) who used journal metrics to guide article selection for their reviews.
We targeted both Web of Science and Scopus databases because they best reflect scholarly impact (Powell & Peterson, 2017). Moreover, the Scopus database indexes the most nursing journals (Powell & Peterson, 2017). Our search was limited to the past five years, 2019–2023, with the intent of including the five journals in the “nursing” category with the highest journal impact factor according to the Journal Citation Reports database. Instead, we included the top six journals because two shared a journal impact factor of 4.3. As evident in Table 1, the journal impact factors ranged from 8.1 to 4.3; qualitative reports were most prominent in two International Journal of Mental Health Nursing and Journal of Nursing Management.Table 1Six Nursing Journals with the Highest Journal Impact Factor (JIF).Table 1Journal20192020202120222023TotalJIF 8.1International Journal of Nursing Studies5330314JIF 5.6International Journal of Mental Health Nursing12101361152JIF 5.5Journal of Nursing Management129917249JIF 5.3Intensive and Critical Care Nursing5236824JIF 4.3Nursing Outlook3132110JIF 4.3Worldviews on Evidence-Based Nursing010203Total3726313325****152
In using journal metrics to guide journal selection, we did not examine publications from other reputable journals where qualitative studies are published. This approach might be viewed as a limitation. Our use of metrics, however, was consistent with other studies and the metrics offered clear criteria for choosing journals. Moreover, impact factors also reflect citation frequency, suggesting the articles could be used as models for reporting qualitative findings. We invite others to consider completing an analysis of articles published in journals not explored in this analysis.
In further collaboration with the information librarian, best search terms were identified for locating publications in the targeted journals. Search terms were determined by combining the word saturation with various methodologies in addition to grounded theory (e.g., phenomenology, qualitative description), analytic approaches (e.g., content analysis, hermeneutics), and data collection techniques (e.g., interviews, observations). The search yielded 1166 articles. Once 264 duplicates were deleted, we had an initial sample of 902 articles. We then randomly selected a twenty percent sample of the articles (n = 185), by journal and by year (see Table 1) and uploaded these articles into Covidence (Covidence systematic review software, Veritas Health Innovation, Melbourne, Australia).
Although our goal was not to conduct a traditional systematic review, these 185 citations were assessed using a template created in Covidence to guide data extraction. In the initial screening process, we excluded 33 articles that were either not based on qualitative methodologies or in which authors reported findings from mixed methods without stand-alone qualitative results. This yielded a final sample of 152 articles. After reviewing six articles together and standardizing our approach, both authors independently extracted data into the Covidence template. The authors met weekly to address questions or concerns as the data extraction proceeded. Decisions to exclude articles were made jointly.
Data were extracted using the following Was saturation mentioned? If yes, was saturation defined; was the process for achieving saturation explained; what types of saturation were noted? How was the sample size justified? What qualitative methodology was claimed? What data collection techniques were used? Were checklists mentioned and if yes which ones?
Saturation was mentioned in almost half the articles (68/152). A definition of saturation, however, was offered by authors of only 19 articles. The process for achieving saturation was explicitly explained in 27 articles. Failing to achieve saturation was mentioned as a limitation in 1 of the 152 articles.
Although saturation was mentioned in 68 articles, the type of saturation was noted by authors of 47 articles. Data saturation was the type most mentioned (25/47 of the articles that stated a type). Theoretical saturation was noted in 9 reports. Informational redundancy was reflected as the type of saturation in 8 articles. Information power, Malterud et al.’s (2016) alternative to saturation, was cited twice. Meaning saturation and thematic saturation were each used once.
Of note, theoretical saturation was sometimes implied by using the term theoretical sufficiency in lieu of theoretical saturation. In 4 of the 9 instances where investigators claimed theoretical saturation, methodologies other than grounded theory were used reflecting a mismatch between methodology and methods. The use of imprecise terminology, at times, made it difficult to know what type of saturation, if any, investigators were claiming. For instance, phrases such as “theoretical saturation of the data” and “theoretical data saturation” were unclear.
In the articles reviewed for this analysis, authors justified their sample size in 79 of 151 reports. Claims of saturation were used as sample size justification in 40 reports. Sample size was justified in the remaining reports through statements about the completeness of data or citing methods papers in which recommended ranges were proposed for an adequate number of interviews (e.g., Guest, 2006; Hennink et al., 2017) or focus groups (e.g., Guest, 2017; Hennink et al., 2019). An unexpected discovery involved a single case where authors justified their sample size based on reaching saturation and then proceeded to interview everyone who volunteered, claiming participant interest as the driving force for continuing data collection. This approach more than doubled the sample size.
A plethora of methodologies were mentioned by authors (e.g., grounded theory, phenomenology, action research, qualitative description). It is noteworthy that thematic analysis, an analytic technique, was regarded as a methodology. Although we were not intent on assessing whether authors followed all the methodological tenets of their approach, it was clear in many cases that there was a misfit between the claimed methodology and the approaches used.
We did, however, seek to determine how saturation was used in reports claiming grounded theory, given that theoretical saturation is a tenet of grounded theory. Of the 12 articles in which authors claimed grounded theory methodology, 8 mention saturation. Of these 8, authors of 3 reports did not mention a type of saturation, 3 specified theoretical saturation, and 2 specified theoretical sufficiency. The 4 reports in which saturation was not mentioned and the 3 reports where authors did not specify theoretical saturation reflect the discrepancy that sometimes was evident between expected methodological features and the way in which studies were reported.
Again, our focus was on saturation, not a full assessment of how methodological tenets were applied. Nonetheless, authors of only 1 of the 12 articles claiming grounded theory methodology used theoretical sampling to achieve saturation and discover a theory. Authors of another article claimed to have discovered a theory but they did not mention theoretical sampling and they commented they may not have achieved saturation across various groups used in their analysis. Other authors used words such as concepts and themes, not theory, in their reports.
We also sought to determine whether saturation was mentioned in reports claiming reflexive thematic analysis and phenomenology, given that the appropriateness of saturation for those methodologies is questionable. For the 5 reports in which authors asserted using reflexive thematic analysis, achieving saturation was noted in 2 articles. For the 18 reports in which authors claimed using phenomenology, 8 addressed data saturation, 1 mentioned informational redundancy, and 3 mentioned saturation without identifying the type.
We were interested in approaches to data collection because, despite the array of options, the default seems to be individual interviews. Data collection was expressly mentioned in 150 articles. Individual interviews dominated the approach to data collection (118/150). Although individual interviews were mostly used as the single approach to data collection (92/118), they were sometimes combined with other techniques such as focus groups, open-ended survey questions, observations, and photographs (26/118). Focus groups were the second most frequently used means of data collection (18/150). Like individual interviews, these were mostly conducted as the only data collection technique (17/18) although in one instance focus groups were accompanied by health record reviews. The remaining reports were based on other approaches to data collection such as open-ended surveys, document reviews, textual analysis, or audio recordings.
Because interviews dominated how data were collected, and because it is the number of interviews from knowledgeable informants more than the number of participants that is relevant to achieving high quality data, we examined the number of interviews conducted with each participant. Of the 117 articles in which authors addressed the number of interviews per participant, multiple interviews/participant (2–4) were conducted by authors of 10 reports. In one article where investigators claimed phenomenology as their methodology, a single interview was done with each participant rather than the multiple interviews with each participant that are consistent with that methodology. In grounded theory and other methodologies, multiple interviews are useful with knowledgeable individuals who can provide new insights into understanding the study topic.
Counter to expectations, there was no mention of checklists in 103 articles in this sample. When checklists were mentioned, COREQ was most frequently cited for a total of 45 citations across the 6 journals. COREQ was mentioned most often (17 citations) within the International Journal of Mental Health Nursing and least often in the International Journal of Nursing Studies (2 citations), Nursing Outlook (1 citation), and not at all in Worldviews on Evidence-Based Nursing. When checklists are mentioned, it seems as though authors are leaning on them to justify the credibility of their work.
Author guidelines were examined for the six journals used in this evaluation to determine whether any required using qualitative reporting checklists. The guidelines for Nursing Outlook and Worldviews on Evidence-Based Nursing made no mention of checklists. Guidelines for the other 4 journals promoted using checklists in the EQUATOR network. The International Journal of Mental Health Nursing endorsed the COREQ checklist (Tong et al., 2007) and the Journal of Nursing Management endorsed the SRQR checklist (O'Brien et al., 2014). Guidelines for Intensive & Critical Care Nursing were more general, indicating that authors should follow EQUATOR checklists. Guidelines for the International Journal of Nursing Studies also indicated that authors should follow the EQUATOR guidelines, adding that the checklist was for the reviewers and there was no need to mention the guidelines in the article. Editorial policies regarding checklists may support reporting qualitative studies in a manner that is inauthentic to the methodology used.
The need to ensure rigor and quality in qualitative inquiry, like all science, is not in dispute. There is evidence, however, that the meaning of saturation has become muddied. The meaning of saturation as it originated in grounded theory has been overshadowed by the notion that saturation applies to all qualitative inquiry and it equates to an adequate sample size and rigor. Moreover, adequate sample size is often addressed by the number of participants rather than the quality of the data.
The evidence in this re-view reflects that for at least six nursing journals, saturation is mentioned in almost half of the articles. The evidence is sparser, however, regarding authors including definitions of saturation or explaining the process for achieving it. There is evidence that data saturation is a dominant type of saturation claimed by authors, but other types are mentioned as well. There is evidence that the concept of saturation is used across a multitude of qualitative methodologies, even those such as reflexive thematic analysis and phenomenology where it is inconsistent with the methodology. There is strong evidence that investigators rely on interviews as the mode of data collection, ignoring other potentially rich data sources. Perhaps more important, there is also strong evidence that investigators focus more on the number of participants rather than the quality of the data and interviews with key participants. Lastly, there is evidence that some of the nursing journals used in this analysis require authors to follow checklist guidelines, yet few of the authors mentioned the use of checklists—and this is a promising sign.
This re-view, therefore, reflects the need to step back and critically assess the status of saturation as it is used in articles published in nursing journals. Better definitions of terms are needed to assure consistency in language that will enhance understanding for the end users of research reports, including students. The various qualitative methodologies must be considered so that proper terms (and therefore research techniques) are used appropriately for the various methodologies. For instance, defining saturation numerically, including rules-of-thumb to determine the number of participants, must be regarded as a general guide, one that is supplemented with an understanding that the appropriateness and adequacy of the data are paramount. A key ingredient in grounded theory studies is understanding that theoretical sampling is the only means to derive the final number of participants, a number determined by achieving theoretical saturation.
It seems too that those conducting and assessing qualitative inquiries must become well-versed in the numerous qualitative methodologies to know what is appropriate for each. Expertise in multiple methodologies is rare though and also rare even within methodologies with varied approaches as illustrated by the different types of grounded theory. In this regard, grounded theory experts recommend having an expert mentor (Kearney, 2009; Morse, 2001; Schreiber, 2001; Stern, 1994; Stern & Covan, 2001). Schreiber (2001) underscores that learning grounded theory involves conducting research using grounded theory because “doing” is a key feature of learning; the doing is more effective when the novice has an experienced mentor. Merleau-Ponty (1962; as cited in van Manen (1984) echoes the importance of “doing” as it pertains to phenomenology. We would expand the importance of mentorship to all qualitative methodologies, a belief supported by authors who note “a mentor [is needed] for each method” (Pursley-Crotteau et al., 2001, p. 207). Likewise, we would expand the importance of “doing” to learning any qualitative methodology.
The need for mentorship and solid teaching is at the heart of the saturation issue. When grounded theory was first discovered, Glaser and Strauss taught students who formed a cadre of skilled grounded theorists. The former students became mentors, having passed the test of learning by doing and becoming “powerful, experienced teachers [who] mentor less experienced proteges” (Stern & Covan, 2001, p. 30). The positive effects of mentorship are evident in other qualitative methodologies as well as exemplified by Benner's mentorship from Lazarus and Dreyfuss that led to her classic text, based on Heideggerian phenomenology, From Novice to Expert (Benner, 1984). Whether the mentorship comes from nurse experts in a particular methodology or methodological experts from other disciplines, the effect of mentorship is undeniable.
Concurrently, there is a cadre of well-intentioned qualitative researchers who have had an overview course and, with luck, some exposure to more rigorous “doing” in one of the qualitative methodologies. Some of these students become educators who pass along their knowledge to their students. If this knowledge is acquired without intense study and understanding, then methodological “slurring” (Baker et al., 1992, p. 1355) and misunderstanding can occur. Passing along misunderstanding then affects students who, along with becoming researchers, become educators who may become reviewers with a few becoming journal editors.
Although not a substitute for good mentorship, in the absence of a mentor, investigators conducting qualitative inquiry are urged to become well-versed in the methodology they believe is best suited to their question by turning to original sources written by well-trained methodologists. Rather than relying on “how to” methodology texts or read a few references on the surface and cherry pick ideas, investigators are urged to read widely, deeply, critically, and methodically from primary sources, methodological literature, and current methodological debates. In this way, the true intent is reflected rather than someone's interpretation of the methodology, an interpretation that may be somewhat inconsistent with important nuances of any particular methodology (e.g., Glaserian versus Straussian grounded theory, descriptive versus interpretive phenomenology).
In re-viewing saturation, we propose a call to action to strengthen the authentic reporting of qualitative inquiry published in nursing journals. The “… problematic and troublesome …” (p. 1105) saturation land mine exposed by Thorne and Darbyshire (2005) not only remains but it is expanding. We remain challenged to answer Thorne's (2020) question “when we invoke the sacred language of ‘saturation,’ what is it that we really mean?” (p. 3). Authors need to ensure their use of saturation is applicable to their study based on a sound understanding of what saturation means and whether it is consistent with the qualitative methodology used. If claims of saturation are made, authors need to provide a transparent and auditable explanation of how they used it and for what purpose. Such an approach will make proper use of research funds, investigator time, and improve the quality of qualitative research reporting. In turn, this will yield knowledge from rigorous publications that not only expand understanding of a particular content area but also remain true to methodological principles rather than merely referencing checklists and reporting guidelines.
No external funding.
Bonnie Mowinski Jennings: Writing – review & editing, Writing – original draft, Validation, Supervision, Resources, Methodology, Investigation, Formal analysis, Conceptualization. Katherine A. Yeager: Writing – review & editing, Visualization, Validation, Software, Resources, Methodology, Investigation, Formal analysis, Data curation.
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.