Authors: Sandro Colombo (1, Independent Consultant, El Escorial, Spain), Chiara Altare (2, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, United States; 3, Johns Hopkins Center for Humanitarian Health, Baltimore, MD, United States)
Categories: Public Health, humanitarian crises, data interpretation, decision-making, epidemiology, uncertainty
Source: Frontiers in Public Health
Authors: Sandro Colombo, Chiara Altare
Humanitarian decision-making occurs in volatile and politically charged environments where information is often incomplete, outdated, or conflicting. Effective humanitarian response often requires interpreting poor-quality data to guide interventions, allocate resources, and assess impact. Despite advances in evidence generation, knowledge gaps persist, and decisions are frequently influenced by political and organizational factors rather than by data. This paper argues that data interpretation is an area of weakness in humanitarian response. Data availability and quality vary across crises, with methodological challenges and political sensitivities further complicating interpretation. The three examples of Darfur (Sudan), Yemen and Ethiopia illustrate how conflicting information and ambiguous interpretation can negatively impact critical decisions with far-reaching consequences on the affected communities. This paper concludes with suggestions for making better interpretation and use of data in humanitarian crises.
Humanitarian actors operate in highly politicised environments often marked by chaos and volatility. They must manage incomplete, inaccurate, and quickly outdated information and coordinate with numerous agencies, each pursuing their agendas. Politicians, managers, and humanitarian workers make a variety of decisions, such as whether to initiate, scale up, or scale down a response; which interventions to prioritise; how to allocate financial, technical, and human resources; whether and how to target the most vulnerable population groups, and how to adjust the response based on monitoring the crisis’s evolution and the effectiveness of operations (2). Some of these decisions—or the lack thereof—can have significant negative and long-lasting consequences on affected populations. For example, misinterpreting or overlooking available information—even when incomplete or of poor quality—can lead to underestimating the severity of a crisis, neglecting priority needs, and ignoring human rights abuses. This, in turn, may result in reduced humanitarian aid, weakened advocacy, ineffective response strategies, and inadequate interventions—ultimately exacerbating the suffering of affected populations and increasing morbidity and mortality. Furthermore, interpreting crisis situations through universal standards and thresholds, with the implication that indicators and benchmarks are equally applicable across diverse contexts and cultures, has resulted in tensions among humanitarian organizations, exemplified by the criticisms levelled against the Sphere Project (3).
Evidence—defined as “any form of knowledge, including but not limited to research, that is of sufficient quality to inform decisions” (4)— is essential for guiding action. Despite the increase in research studies in humanitarian settings over the last decades and the generation of critical evidence in key sectors (5, 6), knowledge gaps remain. The existence of evidence does not ensure, however, that evidence-based decisions are made. Political considerations and organisational aspects influence decisions, sometimes regardless of, or even contrary to, the available information. In addition, past knowledge and experience, assumptions, established narratives, and prior decisions (e.g., on budget allocation or prioritisation of some population groups) restrict the available response options. Moreover, the quality of evidence can be compromised by security constraints, data collection challenges, biases, and partisan manipulation. Insecurity or political barriers may prevent humanitarian workers from gathering critical information. Some evidence can also be withheld when its disclosure might harm political agendas. As a result, conflicting information and gaps in understanding are common, making it difficult to make sense of data and inform effective decisions.
Interpreting crises is not solely reliant on hard data. Insights from key informants, shaped by their experience in similar contexts and underlying assumptions, also play a crucial role in understanding the crisis and guiding the response. Experience is particularly valuable when decisions must be made urgently, the information available is inadequate, and the situation is familiar to the decision maker (7). Indeed, the cost of delaying decisions to obtain more and better data can be high in terms of increased suffering and lives lost. Yet, previous experience is part of the evidence to use, as most crises’ effects can be anticipated, especially when knowing the context where the crisis strikes. However, just as poor-quality data can lead to misinterpretation, overconfidence in intuitive choices and experience, cognitive biases, and neglect of background information can result in misunderstanding the situation. In complex contexts, interpretation inevitably involves some degree of subjectivity in data selection, assessment of their quality, and analysis “There is no objective measure for the subjective components of interpretation” (8). Consequently, the risk of interpretive biases is a constant concern.
This article argues that data interpretation is a weakness in humanitarian response. It emphasizes the importance of not accepting the data at face value but scrutinising them and understanding their limitations, recognising the importance of contextual factors, and integrating local knowledge to interpret data effectively. We first highlight key issues related to data availability and quality in humanitarian settings and we then provide three examples of interpretation challenges in selected domains of public health mortality, malnutrition and famine, and health service delivery. We conclude by sharing suggestions on interpreting data more effectively, when understanding is complex due to conflicting facts, information gaps, or discrepancies between data sources.
“We are absolutely sure that we cannot be sure about the data” [Rosling, H., 2015, cited in (9)]. Data availability and quality vary extensively across crises and countries, although numerous initiatives have strengthened data collection methods and systems in humanitarian settings over the last decades. Such efforts include, but are not limited to, the SMART methodology for nutrition and mortality assessments (10), the District Health Information System (DHIS2) for routine health data management at district level (11), innovative approaches to estimate mortality that aim to overcome the complexity of collecting data in challenging settings (12) maternal death and perinatal surveillance and response (13), as well as emergency disease surveillance systems (14). Besides providing guidelines and standard operating procedures, these initiatives have trained thousands of health practitioners across crises, and several of these systems are now part of the backbone of the health information systems in many countries. Yet, the lack of financial and human resources limits such complex data systems’ functionality, coverage, and completeness.
Quality standards and related assessment methods exist, yet they are not routinely used. Assessing data quality, essential for accurate interpretation, involves expert knowledge of data collection and analysis methods and their limitations, and understanding the challenges faced in the specific humanitarian context. Irrelevant or flawed data—lacking sources, derived from poor methods, or missing a discussion of methodological limitations—should be excluded from analysis only after thorough scrutiny. The risk of over-interpreting poor-quality data must be balanced against ignoring them too quickly, which could lead to overlooking unexpected findings and important clues. Understanding the context in which data were collected is essential for determining whether their quality is sufficient for use. For example, insecurity may hinder data collection in inaccessible areas or restrict it to ‘quick and dirty’ methods. In such cases, the only available options may be qualitative information from relevant informants (maybe even conducted remotely) combined with realistic assumptions about the situation and the unknown humanitarian needs. Data deemed plausible and reliable should be retained for further analysis and triangulation with other sources and methods. Filtering out reliable data from existing information is often challenging, given the wide range of indicators from diverse sources and methods and in assorted formats typically available during a crisis.
Armed conflicts result in injuries, disabilities, and excess morbidity and mortality, especially when humanitarian assistance is inadequate, and health systems have collapsed. A rise in mortality is a crucial indicator of the severity of a crisis. Monitoring mortality, along with other selected indicators, can contribute to assessing the overall effectiveness of the response. It also serves to document human rights violations, hold warring parties accountable, advocate for protecting civilians in armed conflicts, and enhance the crisis’ visibility among international actors and donors. Yet, the death toll is one of the most sensitive and politically charged metrics to estimate during conflict (see Box 1). From a methodological perspective, estimating mortality is fraught with challenges (15), including multiple potential sources of biases and lack of capacity in the design and supervision of data collection and analysis.
Climate change and conflicts are the key drivers of current food crises. In recent years, food crises have reached catastrophic levels with extremely high mortality in countries affected by conflicts, such as South Sudan, Sudan, Yemen, Ethiopia, Somalia, and the Gaza Strip. In these contexts, intentional starvation of civilians has been used as a tool of war (19).
A classification system of food insecurity—The Integrated Phase Classification (IPC) (20)—provides a rigorous, evidence- and consensus-based analysis framework to assess the severity of the crisis for early warning and real-time analysis. The IPC is based on multiple food insecurity, malnutrition, and mortality indicators. For example, a famine (i.e., phase five, the worst category of the IPC scale) is characterised by a crude death rate greater than 2 per 10,000 people per day, a prevalence of global acute malnutrition greater than 30%, and more than 20% of households facing extreme food deficits. For a situation to be classified as a famine, all three indicators must reach the specified thresholds; otherwise, it is categorised as an emergency or catastrophe. The IPC relies on a data-driven approach, requiring rigorous data collection and analysis. As challenges remain in applying this approach in data-scarce settings (see Box 2 on Yemen), officially declaring a famine remains complex and politically charged, although special additional protocols have been adapted for areas with limited or no humanitarian access (21). Indeed, governments may oppose an official declaration of famine that would carry the stigma of their inability to feed their people or purposely starve them in a civil war, as in the Tigray region of Ethiopia in 2021 (22).
Interpreting food insecurity based solely on severity is, however, insufficient. The magnitude and duration of the crisis also contribute to the public health and social consequences of hunger. Excess mortality can be significantly high even in crises that do not reach the severity of the famine phase of the IPC, especially in protracted food insecurity crises (26) (Box 2).
Children in crisis-affected countries are at increased risk of excess morbidity and mortality from vaccine-preventable diseases due to the disruption of preventive and curative health services, food insecurity, and unsanitary living conditions, to name a few risk factors.
Immunisation is a key public health intervention to mitigate excess morbidity and mortality in crises. Therefore, assessing and monitoring its coverage is crucial for designing an effective humanitarian response strategy and targeting the most vulnerable children. Administrative data from the routine health information system, small-scale cluster surveys, and country-wide demographic surveys are the main approaches for estimating and monitoring immunization coverage over time (Table 1).
Comparing immunisation coverage estimates from different approaches should be conducted cautiously, as the indicators can refer to different population groups and reflect the limitations of each method (see Box 3). In humanitarian crises, weakened health information systems, limited access to health services, and population movements can exacerbate existing challenges to quality immunization data, such as imprecise denominators, target overreliance, and workforce capacity gaps (28). Triangulating immunization coverage data from different methods may reveal inconsistencies. In such cases, informants’ perspectives help explain the discrepancies and identify the most reliable source to use.
The examples discussed in this paper illustrate that data interpretation can be challenging, particularly in crises marked by complexity and uncertainty where relevant information is missing, or discrepancies exist between indicators from different sources. In such situations, there is no fixed template for interpretation, rather we offer the following recommendations.
For
For decision-makers:
Data do not always speak for themselves. The examples in this article underscore the need for greater attention to data interpretation. The controversy surrounding the occurrence of famine in Yemen highlights that a lack of evidence does not necessarily imply evidence of absence. Famine is not a sudden, binary event but a gradual process of worsening food insecurity, during which thousands can die from starvation and related causes without a famine being declared.
The conflicting findings of the two mortality surveys in Darfur can be understood only by examining differences in the populations studied, recall periods, and possibly the timing of violence peaks. However, politicians, journalists, and activists are not always equipped to interpret the findings in light of such methodological nuances. Better coordination among the agencies that designed and conducted the surveys could have enhanced the efficiency of the studies and improved the consistency of the results, thereby reducing confusion.
The discrepancy between routine and survey estimates of immunisation coverage in Ethiopia underscores the need for caution in the interpretation of findings from triangulation of different sources. Indeed, both administrative and survey data are susceptible to shortcomings and errors, which, in the Ethiopia case study, were understood through key informants’ interviews. As outlined in our recommendations, qualitative information can add depth to interpretation by shedding light on aspects that may not be immediately apparent.
Humanitarian crises—particularly conflicts—are inherently unstable and chaotic, with outcomes that are often unpredictable. They stand in stark contrast to the controlled environments typically required for rigorous data collection, analysis and interpretation (35). Crises therefore call for honest interpretation of poor-quality data and complex contexts—interpretation that acknowledges its limitations, while drawing on local knowledge and lessons learned from past crises. Oversimplified and flawed narratives, often presented by news media, can distort public understanding, misinform decision-makers, and ultimately contribute to ineffective or even harmful responses.
“When reporters do not know what is happening, remarked the journalist Hilsum, they call it anarchy. And when aid workers do not know what is happening, Duffield added, they call it a complex emergency” (36). Uncertainty is a constant attribute of humanitarian crises, with both “known and unknown unknowns” -in the Rumsfeld’s categorisation-confronting humanitarian actors. Efforts aimed at reducing it should continue by leveraging new technologies, sustained analytical capacity strengthening and advocating for humanitarian access. Given recent effective and announced funding cuts (37), all efforts to manage uncertainty are more necessary than ever to make the best use of available information for improved humanitarian action.