Authors: Ciara M. Greene, Maryanne Brassil, Eden Bryan, Skylee Blair Glass, Rhiannon O’Connor, Valerie O’Keeffe, Emilia Howarth, Leenisha Russel D’Silva, Ruiqi Tong, Gillian Murphy
Categories: Article, Fake news, Misinformation, Behaviour, Food, False memory, Psychology, Human behaviour
Source: Scientific Reports
Authors: Ciara M. Greene, Maryanne Brassil, Eden Bryan, Skylee Blair Glass, Rhiannon O’Connor, Valerie O’Keeffe, Emilia Howarth, Leenisha Russel D’Silva, Ruiqi Tong, Gillian Murphy
While misinformation is often assumed to directly affect behaviour, experimental tests of this hypothesis are rare. We examined effects of a single exposure to misinformation on political and non-political behaviours. Study 1 participants (N = 2,397) were exposed to a fabricated news story about food contamination, with a subset (n = 143) subsequently invited to taste the targeted foods in a laboratory setting. Exposure to the fabricated story did not significantly affect attitudes towards or consumption of the target food. Study 2 (n = 417) confirmed that results were not specific to the particular story presented. Finally, Study 3 (n = 413) tested effects of misinformation about climate change. Exposure to climate-skeptical misinformation reduced signatures on an online petition, but had no effect on two other targeted behaviours. We conclude with a call for further experimental research to delineate the conditions under which misinformation does and does not affect behaviour.
The rise of social media, and the increasing use of online sources as citizens’ main source of news, has led to increased awareness of the dangers of misinformation and so-called “fake news”^1^. The terms “misinformation”, “disinformation” and “fake news” are often used interchangeably, though they do have distinct definitions^2^. In this paper, we use the term “misinformation” to refer to any misleading content, whether intentionally or unintentionally disseminated. This includes information shared by word of mouth or on social media. We use the term “fake news” to specifically describe misinformation that purports to be real news content, often in the form of a fabricated or debunked news story or headline. Direct evidence regarding the effects of misinformation on real-world behaviour is lacking. In a recent scoping review, we reported that just 1% of published papers on the topic of misinformation evaluated actual behaviour as an outcome, with the vast majority focusing instead on belief in the misinformation^3^. The extent to which exposure to misinformation directly affects behaviour, and the factors which might moderate that relationship, is therefore chronically under-investigated.
There are several potential routes through which (mis)information might influence behaviour. The Theory of Planned Behaviour^4^ holds that behaviour is shaped by beliefs about a particular behaviour (including its possible outcomes), in combination with normative beliefs (which are in turn shaped by the individual’s social networks and media exposure) and the individual’s perceived control over the behaviour. To take the example of vaccination, Jane’s intention to get vaccinated against COVID-19 is influenced by her own beliefs about vaccine efficacy (“vaccines help prevent diseases”), her beliefs about what is normative (“everyone else is getting vaccinated”) and her beliefs about her own ability to obtain the vaccine (“I can just go the pharmacy and ask for it”)^5,6^. Exposure to misinformation might affect several points in this process; for example, reading a misleading story about the dangers of vaccination or talking to a coworker about its efficacy might shift Jane’s personal attitudes and beliefs (“the vaccine might do more harm than good”). Exposure to repeated misinformation, especially with feedback from members of her social network, might change her beliefs around vaccination norms (“no one in my community is getting vaccinated, and it isn’t necessary anyway”) or her sense of behavioural control (“it would be hard for me to access a safe vaccine”). Each of these factors might result in a reduction in intention to vaccinate, but the information to behaviour pipeline is a leaky one^7,8^, and there is no guarantee that exposure to (mis)information will necessarily affect an individual’s actions.
There are of course several real-world cases of misinformation affecting behaviour. Some particularly dramatic examples include misleading reports of a link between measles, mumps and rubella (MMR) vaccine and autism, which led to a sharp reduction in childhood vaccination rates^9^, and widespread misinformation regarding the 2020 US presidential election, leading to the Capitol riots on 6 January 2021^10,11^. In cases such as these however, population-level behavioural change is typically linked with the rise of significant and widespread movements, such as the anti-vax movement and QAnon conspiracy, in which individuals are exposed to vast amounts of misinformation on the same topic from multiple sources^12–14^. There is also some evidence of an association between regular exposure to misinformation and People living in U.S. counties with higher viewership of Fox News were less likely to engage in health-promoting behaviours during the COVID-19 pandemic, including social distancing and masking^15^, and more likely to vote Republican in state and national elections^16^. In a similar study, Italian municipalities with more exposure to “fake news” demonstrated increased support for populist parties in the 2018 general election^17^.
Such population-level research is informative, but as these studies are not experimental they are also plagued by questions of do television viewers in certain counties change their behaviour or become more right-leaning because they watch Fox News, or do they watch Fox News because they (and their social networks) are ideologically aligned with its content and positioning? People are more likely to fall for misinformation that is consistent with their existing views, or appears to come from ideologically-aligned sources^18–22^. Previous research has also demonstrated that multiple exposures to misinformation make the information seem more believable – a phenomenon termed “illusory truth”^23^. Thus, a social network of likeminded people seeking out and sharing politically congenial content might reasonably be expected to experience behavioural consequences. However, analyses of group-level exposure to misleading content do not allow inferences to be drawn regarding the specific effects of misinformation.
Researchers have used creative approaches to address this question, including quasi-experimental studies examining the consequences of particular news stories. One study measured the impact of an Italian court decision that formally linked the MMR vaccine with autism, and used a difference-in-differences approach to link coverage of the decision with a subsequent reduction in national child immunisation rates^24^. Studies such as this are very helpful in shedding light on causal relationships, but still operate at the group level and so do not allow individual-level predictions. Can reading a single news story about the dangers of vaccination have a direct one-to-one effect on the individual’s decision to vaccinate? Could watching a single TikTok video about immigration influence a voter’s electoral choices? How is the causal relationship affected by the individual’s pre-existing beliefs, or by characteristics of the message content or source? Addressing questions such as these requires experimental manipulations in which participants are systematically exposed to misinformation and its effects are measured.
Evaluating the effects of individual items of misinformation or fake news on individual behaviour is inherently challenging; as van der Linden and Kyrychenko^25^ note, “it would neither be ethical nor feasible to randomly assign half the population to misinformation about vaccines or elections to determine whether that changes vaccine uptake or election outcomes” (p. 959). As a result, many researchers have defaulted to examining behavioural intentions as a proxy for real-world behaviour. The findings of these studies have been mixed; for example, while some studies have shown that exposure to anti-vaccine misinformation reduced COVID-19 vaccination intentions^26,27^, others have found null effects of similar exposure^28,29^. The aforementioned studies were all conducted during the COVID-19 pandemic, during which individuals’ health behaviours were frequently politicised and influenced by partisan identity^30–32^. It is thus possible that participants in these studies were on high alert for the presence of misinformation; moreover, their responses may have been driven by partisan concerns rather than the specific information available to them.
For the reasons described above, there have been limited attempts to directly examine the effects of specific misinformation exposure on behaviour. In one study, priming participants with a “fake news” story that associated finger tapping speed with either positive or negative traits was associated with a small increase or decrease in maximum tapping speed^33^. There is of course a very significant difference between the physiological priming described in this paper and an individual’s decisions regarding how to vote or whether to take health precautions. Real-world impacts may be larger than those reported in the finger-tapping study – as the author of that paper suggests – or the influence of an individual item of misinformation may be drowned out by personal beliefs and the volume of social and political information available to participants. Supporting the latter argument, de Saint Laurent and colleagues reported that any effects of exposure to pro- or anti-vaccine misinformation on intentions to vaccinate were completely overshadowed by the influence of pre-existing views^34^. In another study, van der Linden^35^ reported that exposure to a two-minute video articulating conspiracy theories about climate change reduced signatures on an online petition to reduce global warming. Interestingly, while political ideology was a strong predictor of behaviour, it did not interact with the exposure to pro-climate or conspiratorial misinformation, suggesting potentially independent effects.
In the present paper, we directly evaluated the effects of exposure to a single fake news story on real-world behaviour. Our first studies aimed to isolate the effects of news exposure from ideological context; thus, in Studies 1 and 2 we focused on eating behaviours, and in particular on consumption of nuts, a behaviour selected because it has no obvious ideological or political implications. Our primary goal was to investigate whether one-off exposure to a fabricated news story alleging widespread contamination of the target nut would reduce participants’ liking for the nut, and their willingness to eat it in a subsequent laboratory experiment. To do this, we adapted a paradigm previously used to evaluate effects of false childhood memories on subsequent food consumption^36–39^. In these studies, participants are presented with a detailed and highly personalised computer-generated profile, which is based on their responses to an earlier questionnaire and designed to convince them that they became very ill as a child after eating a particular food (e.g., spoiled peach yogurt). Participants are then invited into the lab – typically under the guise of completing an unrelated marketing experiment – to taste various foods, including the targeted yogurt. Participants in these studies who were provided with the false autobiographical memory subsequently ate less of the targeted food; this effect transferred to a related food (strawberry yogurt) but not to an unrelated food (crackers).
Study 1 in the present paper consisted of two first, participants completed an online survey, in which those in the misinformation condition were exposed to a fake news story about a particular nut. We selected two nuts – almonds and cashews – and counterbalanced their assignment as the target nut. Participants rated their preference for a range of foods, including the target and non-target nut, before and after reading the fake news story. Next, a subset of participants were invited to take part in an in-person laboratory study about one week after the online study. As part of this study, they were asked to taste and rate various foods, including almond and cashew nuts, ostensibly as part of an effort to determine the role of sustainability in their selection of snack foods. To anticipate the results, Study 1 indicated that exposure to misinformation about nuts had no effect on attitudes towards the target nut or the quantity of nuts eaten in the laboratory. In Study 2, we replicated the first part of Study 1 to investigate whether attitudinal effects were specific to the fake news story used in this study, and found that the null results generalised across different news stories. In Study 3, we returned to the question of political ideology and investigated effects of misinformation exposure in the context of climate change. In this study, we tested the influence of one-off exposure to misinformation that was either supportive of or skeptical about climate science on intentions to engage in a series of behaviours as well as actual engagement in related online behaviours (signing a petition, signing up to a mailing list, or donating a portion of the study payment to a climate change organisation). Here, we were particularly interested in the role of pre-existing beliefs about climate change, and investigated whether participants’ belief that the earth’s climate is changing would moderate the impact of the fake news items. As described below, we report that while climate change beliefs significantly and substantially affected both behaviour and behavioural intention, the effects of misinformation exposure were generally weaker and inconsistent.
The first research question in Study 1 asked whether exposure to a fake news story about food contamination would change attitudes towards the target food. We preregistered three hypotheses related to this
H1a: Attitudes towards the target nut would become less favourable after viewing the fake news story.H1b: Attitudes towards the non-target nut would not change after viewing the fake news story.H1c: Attitudes towards the target and non-target nut would not change for participants in the control condition.
Our second research question asked whether exposure to the fake news story would change actual consumption of the target food. Here, we hypothesised
H2a: Participants who viewed a fake news story about contamination of nuts would eat less of the target nut than participants in the control condition, who did not view the fabricated story.H2b: Consumption of the non-target nut would not differ between the misinformed and control conditions.
Exposure to fake news can lead to the development of false memories for the events described in the news story^40^. Experiencing a false memory for a fake news story (e.g., a story suggesting that increased caffeine intake can reduce coronavirus symptoms) is associated with stronger intentions to engage in the targeted behaviour (drinking more coffee), over and above mere exposure to the story^29^. However, the formation of these false memories is especially likely if the content of the story is congruent with participants’ political orientation or beliefs^41–43^. Thus, the direction of effect is unclear, and it is possible that initial positive attitudes towards the targeted behaviour (e.g., preference for coffee) might result in both an increase in false memories for the pro-coffee story, and an increased willingness to drink more coffee. In the present study, we therefore preregistered an additional memory-related research question, hypothesising
H3a: Participants who reported less favourable initial attitudes towards the target nut would be more likely to form a false memory for the fake news story in the online study.H3b: Participants who formed a false memory for the fake news story would eat less of the target nut in the in-person study.H3c: The effect of false memories on nut consumption would be moderated by initial attitudes towards the target nut.
The hypotheses, research design and analysis for Study 1 were preregistered at https://aspredicted.org/V5H_QDW. There were minor deviations from the preregistration (see Supplementary Materials, Table S1 for details).
All experiments described in this paper received ethical approval from the research ethics committee of University College Cork, and were conducted in line with the Code of Professional Ethics of the Psychological Society of Ireland. Informed consent was obtained from all participants.
Participants were recruited via three methods; (1) traditional recruitment via personal networks and advertisements on campus; (2) direct advertisement on research-focussed forums on the social media platform Reddit; and (3) advertisement on the research recruitment platform Prolific. Participants were required to be over the age of 18. Two questions presented early in the survey asked participants to indicate any food allergies or other dietary restrictions. Participants who indicated a nut allergy were screened out at this stage and thanked for their willingness to participate. Participants were advised that the study would deal with issues pertaining to food, and were advised to self-exclude if they suffered from an eating disorder or anticipated that the topic of the study was likely to cause them distress. There were no other exclusion criteria relating to the online portion of the study.
Data collection was conducted in a cascading manner, whereby all participants who completed the online study via the traditional recruitment route and did not meet any of the exclusion criteria were invited to take part in the in-person study. We initially aimed to recruit a sample of 1,200 participants (following exclusions) in the online study, in order to detect an effect size of f = 0.1 in the primary analysis. We preregistered a target of 150 participants in the in-person study, which would provide 80% power to detect a small effect of f = 0.1 in the critical within-between subjects interaction. We stated that we would end data collection when we reached this target, or on 1st August 2023, assuming we had reached a minimum of 100 participants. In-person data collection proved challenging, and we did not reach our minimum sample size of 100 in-person participants by 1st August 2023. Data collection therefore continued into the next academic year, finishing on 25th April 2024. As a result of the extended data collection period, we far exceeded our target for online participants, recruiting a total of 2,673 participants. In line with our preregistration, we excluded participants who submitted incomplete responses or who indicated that they had a nut allergy, leaving a final sample of 2,397 participants (mean age = 36.05 years; SD = 13.575). Of this sample, 890 (37.57%) identified as male, 1,444 (60.95%) identified as female and 35 (1.48%) selected a third gender. This sample provided 95% power to detect very small effects (f = 0.03) in the interaction term of our preregistered analyses.
This study employed a 2 × 2 × 2 mixed within/between subjects design, in which the independent variables were misinformation condition (between misinformation/control), nut category (within target/non-target) and time (within pre/post misinformation exposure).
Participants were randomly assigned to receive misinformation about almonds, misinformation about cashews, or no misinformation (control). Participants in the control condition were also randomly assigned a target nut for the purposes of analysis, though they did not view the associated fake news story. Thus, the study included two experimental a full control condition in which participants were not exposed to any misinformation, and a control nut to allow for the possibility that exposure to the misinformation might reduce consumption of all nuts, not just the target nut. A schematic of the experimental design may be seen in Fig. 1.
All study materials may be viewed and downloaded at https://osf.io/nygd7/.
Participants in the misinformation condition were exposed to a fabricated news story suggesting that about 80% of bags of the target nut sold in supermarkets were contaminated by spider eggs. The story was accompanied by a close-up photograph of a spider laying eggs. Participants in both the misinformation and control conditions were also exposed to three true news stories on the topic of climate change and food production. The four news stories may be seen in Fig. 2.
Participants were invited to take part in a study on the topic of “Climate change, sustainability and food choices”, and informed that the purpose of the study was to assess how concerns about sustainability impact the food choices of consumers. After providing consent to participate, participants provided demographic information and details regarding allergies and dietary preferences that were used to screen out ineligible participants. They then answered some questions about their views on climate change, designed to mask the true purpose of the study. Next, participants were provided with a randomised list of 14 snack foods, including almonds and cashews, and asked to rate how much they liked each of the foods on a scale from 1 (not at all) to 7 (very much).
Fig. 1Schematic of experimental design from Study 1.
Fig. 2News stories presented to participants. (a) Fabricated news story presented to participants in the misinformation condition only. This story referred to the target nut, which was randomly assigned to be either almond or cashew. (b-d) True news stories presented to all participants. Photo (a) Roj Smith, CC BY-NC-SA 2.0; (b) Michael Hermann, CC-BY-SA-4.0; (c) Sgroey, CC BY-SA 4.0; (d) Hong Son, Creative Commons.
Participants were then presented with the news stories described above, including three true stories and one fake story (shown to participants in the misinformation condition only). The stories were presented in random order, and each story was followed by two questions. The first asked, “Have you come across this story before?”, and participants selected from the options, “Yes, I’ve previously come across this story in the media”; “Yes, I’ve previously heard about this story from a friend”; “Yes, I’ve heard this before but I can’t remember where”; “I’m not sure if I’ve seen this before”; and “No, I’ve never heard this before”. To maintain the ruse that the study was investigating marketing and sales of food products, the second question after each story asked whether information about the specific issue addressed in the story ought to be included on supermarket packaging; for example, “Do you think the risk of insect contamination for a given food should be included on supermarket packaging?”
In the next section, participants completed a data gathering bias task, designed to assess the amount of information an individual typically requires before coming to a decision. Research questions relating to this task were preregistered separately and will not be discussed further in this paper. This was followed by some more filler questions about marketing (indicating their preference between different food packaging types) before rating how much they liked each of a second set of 14 snack foods from 1 (not at all) to 7 (very much). This list included almonds and cashews as well as 12 other foods that were not included in the first list. The foods were listed in random order. Participants were then presented with a list of the news stories they had previously viewed and asked to rate the extent to which they believed each of these stories, on a scale from 0 (I don’t believe this at all) to 100 (I completely believe this).
After completing this section of the experiment, participants who were recruited via the Reddit and Prolific routes were immediately directed to the debriefing (see below). Participants recruited via the traditional route were asked if they were interested in attending an in-person lab session where they would be asked to taste some snack foods and rate them, assess their packaging, etc. Participants who indicated their interest in this session were asked to provide an email address for further contact and thanked for their time. Those who initially indicated an interest in the lab-based study but did not go on to complete it were sent the full debriefing survey via email.
Participants who were not invited to the in-person study or who declined the invitation were first asked to tell us what they believed the study was about, selecting from the following How sustainability-related information on food packaging may affect food preferences; how news stories shape food perceptions; how climate change awareness affects food choices; how food packaging affects portion sizes; how climate change deniers respond to news differently; other (please specify). The goal of this question was to establish whether our ruse was successful or if participants identified the focus of the study on fake news.
Participants were then fully debriefed, following procedures described in^44^. Finally, participants were asked to reconsent to the use of their data and thanked for their time.
All data and analysis scripts may be found online at https://osf.io/nygd7/.
Table 1 shows mean attitudes towards both almond and cashew nuts at Time 1 (before misinformation exposure) and Time 2 (after misinformation exposure). Our preregistration called for an analysis of raw rating scale data, however preliminary analysis indicated baseline differences in attitudes towards cashew and almond nuts, with participants generally preferring cashew over almond nuts, regardless of assignment to misinformation condition (t(2368) = 11.31, p < .001, d = 0.23). As the final number of participants in each target nut condition was not perfectly equal (as a result of participant attrition, exclusions due to nut allergies, and self-exclusions), these baseline differences had the potential to confound effects of the experimental conditions. We therefore elected to standardise the attitude variables by computing z scores within each nut type before subjecting the data to further analysis.
Table 1Raw attitude ratings (Mean (SD)) of almond and cashew nuts in the misinformation and control conditions at Time 1 (before misinformation exposure) and Time 2 (after misinformation exposure), on a scale from 1 to 7.Time 1Time 2MisinformationControlMisinformationControlAlmonds4.42 (1.92)4.53 (1.90)4.36 (1.96)4.54 (1.94)Cashews4.83 (2.00)4.95 (1.92)4.76 (2.01)4.89 (1.96)
To evaluate change in attitudes towards the target nut (Hypothesis 1), we conducted a 2 × 2 × 2 mixed ANOVA, in which the DV was standardised attitude towards nuts, and the IVs were condition (between-subjects: misinformation/control), nut (within-subjects: target/non-target) and time (within-subjects: pre/post misinformation exposure). The full results from this analysis are listed in Table 2; no significant effects were observed, though there was a borderline 3-way interaction. Exploratory Bayesian paired-tests were conducted to compare mean attitudes in the misinformation condition at Time 1 and Time 2. There was strong evidence in favour of the null hypothesis for both the target nut (BF01 = 10.07) and non-target nut (BF01 = 25.03), indicating no change in attitudes following misinformation exposure. See Fig. 3 for an illustration.
Table 2Repeated measures ANOVA evaluating effects of time (T1, T2), nut (target, non-target) and misinformation condition (misinformation, control) on standardised attitudes to nuts.CasesSSdfMSF p η²~p~Time0.021,23950.020.150.70< 0.001Nut1.521,23951.522.040.15< 0.001Misinformation Condition10.231,239510.233.360.070.001Time * Misinformation Condition0.161,23950.161.310.25< 0.001Nut * Misinformation Condition0.901,23950.901.210.27< 0.001Time * Nut0.021,23950.020.190.66< 0.001Time * Nut * Misinformation Condition0.291,23950.293.260.070.001
Fig. 3Mean standardised attitudes towards the target and non-target nut in the control and misinformation conditions.
Table 3Responses to the question “have you come across this story before?” for each of the four news stories.Memory responseNuts story (fake)Quinoa story (true)Avocado story (true)Packaging story (true) N
%
N
%
N
%
N
% Yes, came across this story in the media140.89552.2927811.6044018.36Yes, heard about this story from a friend60.38241.00572.38542.25Yes, but can’t remember where402.53933.8839316.3960925.41Not sure1197.5427011.2632813.6842717.81No, never heard it before140088.66195581.56134155.9486736.17Total N1579-2397-2397-2397- Responses deemed a memory for the story.
Responses to the question, “Have you come across this story before?” for each of the four stories are tallied in Table 3. In line with our preregistration, responses of “Yes, I’ve previously come across this story in the media”, “Yes, I’ve previously heard about this story from a friend” or “Yes, I’ve heard this before but I can’t remember where” in response to the fake story were deemed false memories. False memory was very rare in this sample – only 60 participants (3.8% of people in the misinformation condition) remembered having encountered the fabricated nuts story before.
To evaluate the effect of attitudes on the formation of false memories (Hypothesis 3), we conducted a binary logistic regression with standardised attitudes towards the target nut (assessed prior to fake news exposure) as the IV and false memory (yes/no) as the DV. This analysis was restricted to those in the misinformed conditions only. The logistic regression showed no effect of pre-existing attitudes to the target nut on the presence or absence of a false memory (X^2^ (1, N = 1602) = 1.32, p = .25).
Of the 2,003 participants who were not invited or indicated that they were not interested in the lab study, and who therefore completed the final part of the online survey, 923 (46.08%) responded that the study was about “how sustainability-related information on food packaging may affect food preferences”, 491 (24.51%) selected “how climate change awareness affects food choices”, 479 (23.91%) selected “how news stories shape food perceptions”; 55 (2.75%) selected “how climate change deniers respond to news differently” and 10 (0.5%) selected “how food packaging affects portion sizes”. Of the 45 participants (2.25%) who selected “other (please specify)” and provided a written response, one indicated that they believed the study to be about “how fake news can be packaged”; no other responses made any reference to fake news, misinformation or any related terms. Thus, we are confident that our ruse was successful and participants were not aware of the presence of misinformation in the study prior to being debriefed.
All participants who expressed an interest at the end of the online study were invited to participate in the laboratory study, with the exception of those who indicated that they were following a calorie-restricted diet and those who rated their initial attitudes towards almond or cashew nuts as 1 or 2 out of 7, as their willingness to eat the provided foods might be affected.
Participants were invited to participate via an email in which they were informed that the next phase of the research would involve coming in to the lab to taste and rate some foods and give us their opinions on some food packaging. The email reminded participants that the study was not suitable for those with nut allergies, and noted that all food options were appropriate for vegans/vegetarians. If participants responded to this initial invitation, they were asked which of the two laboratories they would like to attend (University College Dublin or University College Cork), and the lab session was scheduled for between one and three weeks after the completion of the online study.
In total, 176 participants completed the laboratory study, however 30 participants were included who should have been excluded in line with our preregistered criteria, as they had provided initial ratings of 1 or 2 out of 7 for almonds and cashews in the online assessment. In addition, three participants were excluded from analysis as, due to a data entry error, negative values for nut consumption were recorded. The final sample comprised 143 participants (52 male, 89 female, 2 other/prefer to self-identify; mean age = 29.27 years, SD = 9.70). We preregistered a target sample of 150 participants, with a minimum sample of 100; the final sample is therefore in line with our preregistered goals, and provided 80% power to detect small effects of f = 0.12 or larger.
The materials and apparatus used were identical across the two laboratories involved in the study. All foods provided to participants were purchased in a large chain supermarket, and comprised (1) roasted salted almonds; (2) roasted salted cashews; (3) lentil curls (a processed salty snack made from lentils); (4) raisins.
Prior to participants entering the lab, the experimenter poured the snack foods into single-use paper containers until they were reasonably full, and noted the weight of each filled container. They then covered a table with a clean white table cloth and placed the four food bowls in a row. The decoy foods (lentil curls and raisins) were randomly assigned to positions 1 and 3 in this row. Target foods (almonds and cashews) were placed in positions 2 and 4, with counterbalancing of position determined by whether the final digit of the participant ID code (a randomly assigned 5 digit number) was odd or even. Each bowl was accompanied by a paper sign indicating what food was in the bowl and providing information on food miles and plastic packaging usage for each food (see Figure S1). Drinking water, paper napkins and hand sanitizer were also provided on the table.
Copies of all forms and scales used in the experiment may be found in online materials at ttps://osf.io/nygd7/. These
Experiment record Used by experimenters to record food weights before and after the experiment and notes from the debriefing session.Current feelings Used by participants to record their current feelings of hunger, fullness, thirst and nausea on Likert scales ranging from 1 (not at all) to 7 (very).Food rating Used by participants to rate each food in terms of how much they like it, how much they would be willing to pay for it, and how much their decision to purchase would be influenced by the food miles or plastic packaging associated with the food.Cognitive Reflection Test [CRT^45,46^]: This list of seven brain-teaser questions was used as a filler task during the experiment (see procedure).Attitudes Towards Climate Change and Science Instrument [ACSI^47^]. Six items from this scale were used as a filler task during the experiment (see procedure).Debriefing sheet (see procedure).
The complete laboratory protocol, including sample scripts used by experimenters, may be found in online materials at https://osf.io/nygd7/. The same protocol was followed across both laboratories, and every care was taken to ensure that the experience was identical for all participants.
Participants were instructed to come to the lab slightly hungry, and advised not to eat for 1–2 hours prior to the study. On arrival, they were informed that they would be asked to taste and rate some foods, to consider whether issues relating to climate and sustainability would play a part in their decision to purchase those items, and to provide their opinions on some food packaging. Participants were provided with a clipboard and asked to complete the current feelings scale.
The experimenter then directed the participant’s attention to the table and explained the four foods and the information contained on the paper signs. Participants were instructed to taste each food in order, taking a sip of water first to cleanse their palate. They should then rate various features of the food on a paper form (taste, how much they would be willing to pay, the influence of food miles or plastic packaging on their decision to purchase etc.). They could taste as much of each food as they liked before making each rating, but were instructed that, after having moved on to the next food, they should not go back to a previous food or change their rating. Participants were assured that the food had been prepared hygienically, using gloves and facemasks, and were directed to the bottle of hand sanitiser in case they wished to clean their hands before beginning. The experimenter then left the room for exactly five minutes as the participant tasted the foods. If the participant was still tasting or rating the foods on the experimenter’s return, they were given as much time as they needed to complete the task.
The experimenter returned to the room and retrieved the completed rating forms. They then handed participants two more forms – the CRT and ACSI – and asked them to complete these forms before the next stage of the study. The experimenter then casually informed the participant that they should feel free to eat as much as they liked of the food in the bowls as, for hygiene reasons, it would have to be thrown out before the next participant arrived. The experimenter then left the participant alone in the room for exactly 10 minutes. On returning, the experimenter moved the bowls to another table so that the participant could not continue to eat after the 10 minutes had elapsed.
Experimenters then asked participants to sit, and engaged in a detailed debriefing conversation. As a starting point, participants were asked what they believed the purpose of the study to be, and the experimenters noted their response. The experimenters then explained the true goal of the study and clarified that the ‘spider eggs’ story had been fabricated. Experimenters briefly explained the concept of false memories, and assured participants that it was perfectly normal to have formed a memory of the fabricated story if they had done so. Participants were given an opportunity to ask questions and discuss the study goals before being directed to a computer to complete the post-debriefing questionnaire described in Study 1A. Finally, participants were thanked for their time and given a paper copy of the debriefing form to take away.
After the participant exited the lab, the experimenter reweighed all the food bowls and noted their final weights.
Consumption of the almond and cashew nuts was measured by subtracting the final weight of the food bowls from their starting weight. Descriptive statistics for raw grams consumed can be found in Table 4.
Table 4Raw grams of almond and cashew nuts consumed in the laboratory study, split by misinformation condition.MisinformationControlNMSDMinMaxNMSDMinMaxAlmonds9210.9511.230525112.9616.14067Cashews9219.3515.480675122.6920.11177
Regardless of experimental condition, participants ate more cashew (M = 20.54 g, SD = 17.28) than almond nuts (M = 11.65 g, SD = 13.17; t(142) = 7.37, p < .001, d = 0.61). We therefore standardised nut consumption within nut type (almonds, cashews) prior to analysis to account for baseline differences; values below are reported as z scores.
To evaluate effects of misinformation exposure on standardised consumption of the target nut, we conducted a 2 (condition: misinformation vs. control) x 2 (nut: target/non-target) ANOVA. The results of this analysis may be found in Table 5. Examination of the means indicates that participants ate fewer nuts in the misinformation condition (M = -0.06, SE = 0.09) than in the control condition (M = 0.11, SE = 0.12), and fewer of the target nut (M = -0.02, SE = 0.09) than the non-target nut (M = 0.07, SE = 0.09), but these effects did not reach statistical significance; see Fig. 4. Exploratory Bayesian t-tests indicated moderate evidence in favour of the null hypothesis, confirming that misinformation condition did not influence consumption of the target nut (BF01 = 3.10) or non-target nut (BF01 = 3.75).
Table 5Results of ANOVA evaluating effects of misinformation condition on consumption of the target and non-target nut.CasesSSdfMSF p η²~p~Nut (target/non-target)0.451, 1410.451.0900.300.01Misinformation condition1.961, 1411.961.240.270.01Nut * misinformation condition0.0251, 1410.0250.060.81< 0.001
Fig. 4Standardised consumption of the target and non-target nut in the misinformation and control conditions.
To evaluate predictors of eating behaviour, we conducted a linear regression in which the outcome variable was standardised grams of target nut consumed, and the predictor variables were (1) initial attitudes towards the target nut, (2) presence or absence of a false memory for the fake news story and 3) the interaction between these predictors. In line with our preregistration, this analysis was restricted to participants in the misinformation condition. The regression model was not significant (F(3,88) = 0.31, p 0.82; R^2^ = 0.01). Neither initial attitudes to the target nut nor the presence of a false memory significantly predicted nut consumption (all ps > 0.58).
In order to rule out the possibility that the null results in the laboratory study were due to selection bias in the sub-sample of participants from Study 1A who chose to participate in Study 1B, we conducted an exploratory analysis of attitudes to the target/non-target nut among the in-person sample, using the same 2 × 2 × 2 ANOVA described in Study 1A. Full results from this analysis may be found in supplemental materials Table S2, however the pattern of results was identical to that observed in the full online sample, with no significant effects. The sub-sample tested in the laboratory study does therefore appear to be representative of the larger sample in Study 1A.
Before concluding that one-off exposure to fake news about food contamination is largely ineffective, it is important to rule out alternative explanations. The null effects observed in Study 1 may have been an artefact of the specific materials used, if for example, participants found the spider-eggs story unconvincing, or were aware of contradictory information. In Study 2, we therefore replicated Study 1 A, but randomly assigned participants in the misinformation condition to view one of four fake news stories, variously suggesting that the target nut had been contaminated by (1) spider eggs (as in Study 1); (2) fungus; (3) E. Coli bacteria; or (4) rodent urine. To rule out effects owing to the potentially aversive nature of the spider photograph used in Study 1, we removed this image and instead paired all fake stories with a stock image of a pile of almond or cashew nuts, as appropriate.
As in Study 1, we asked, “Does exposure to a fake news story about food contamination change attitudes towards the target food?”, and tested the following
H1a: Attitudes towards the target nut will become less favourable after viewing the fake news story.H1b: Attitudes towards the non-target nut will not change after viewing the fake news story.H1c: Attitudes towards the target and non-target nut will not change for participants in the control condition.
The hypotheses, methods and analyses for this study were preregistered at https://aspredicted.org/Y1H_BF2.
Participants in this study were recruited via Prolific using the same inclusion and exclusion criteria as Study 1. In line with our preregistration, we recruited 420 participants to this study, though three were subsequently excluded because they indicated a nut allergy. The final sample comprised 417 participants (158 male, 259 female; mean age = 40.06 years, SD = 13.11).
All participants were exposed to the same three true news stories as in Study 1, which variously described pesticide usage in quinoa farming, the quantity of plastic packaging used in the sale of fruit and vegetables and the amount of water required to grow avocados. Participants in the misinformation condition were randomly assigned to view one fake news story from the following
A recent report from the World Wildlife Federation illuminates the shocking effects of deforestation and modern farming practices on the food we consume. A recent audit found that 80% of bagged [almond/cashew] nuts sold in European supermarkets contained spider eggs, as a result of devastation to natural spider habitats.Climate change is leading to more humid growing conditions that encourage the growth of a fungus inside commercially-grown [almond/cashew] nuts. The fungus is said to be particularly damaging to the human gut.Temperature changes as a result of climate change have contributed to a recent outbreak of E. Coli contamination of [almond/cashew] nuts. Food safety groups are calling for the nuts to be withdrawn from sale until the outbreak is under control.Increased rainfall affects the growing cycle of [almond/cashew] nut trees and results in more branches growing closer to the ground. As a result, a substantial proportion of commercially grown [almond/cashew] nuts are contaminated with urine from rodents who feast on the trees.
The fabricated stories were piloted extensively before being included in the main experiment, to assess plausibility and increase the likelihood that they would elicit behavioural reactions among participants. Pilot participants (N = 95) were asked, “would this story make you more or less likely to eat [almond/cashew] nuts?”, and responded on a scale from 1 (much less likely) to 7 (much more likely). The final version of each of the four stories received a mean rating of less than 1.5 out of 7, indicating that the pilot participants expected the story to substantially reduce their eating behaviour.
The procedure was identical to that described in Study 1 A, with the exception that participants did not complete the data gathering bias task.
To evaluate change in attitudes towards the target nut we conducted a 2 × 2 × 2 mixed ANOVA, in which the DV was standardised attitudes towards nut and the IVs were condition (between-subjects: misinformation vs. control), nut (within-subjects: target/non-target) and time (within-subjects: pre/post misinformation exposure). In this initial analysis, we averaged responses across all four fake news stories.
Overall, the pattern of results was very similar to that observed in Study 1, with no significant effects of time, nut or misinformation condition on attitudes towards nuts (see Tables 6 and 7). Exploratory Bayesian t-tests comparing responses at Time 1 and Time 2 in the misinformation condition provided strong evidence in favour of the null hypothesis for the target nut (BF01 = 13.61) and moderate evidence in favour of the null for the non-target nut (BF01 = 8.80), indicating no change in attitudes following misinformation exposure.
Table 6Descriptive statistics for the effects of time, nut and misinformation condition on standardised attitudes to nuts in study 2.TimeNutCondition N MeanSDT1Target nutcontrol1370.060.99misinformation280− 0.030.97Non-target nutcontrol1370.00.98misinformation280− 0.041.04T2Target nutcontrol1370.060.98misinformation280− 0.041.00Non-target nutcontrol1370.060.98misinformation280− 0.021.02
Table 7Results of ANOVA assessing effects of time nut and misinformation condition on attitudes towards nuts in Study 2.CasesSSdfMSF p η²~p~Time0.0031, 4150.0030.030.85< 0.001Nut0.061, 4150.060.080.78< 0.001Misinformation Condition3.711, 4153.711.220.270.003Time * Misinformation Condition0.031, 4150.030.290.59< 0.001Nut * Misinformation Condition0.021, 4150.020.030.87< 0.001Time * Nut< 0.0011, 415< 0.0010.010.94< 0.001Time * Nut * Misinformation Condition0.101, 4150.101.120.290.003
Preregistered follow-up ANOVAs evaluated the same effects separately for each fake story (contamination by spider eggs, fungus, E. Coli and rodent urine). Full analyses for each story may be found in supplemental materials (Table S3-S6), and means plots are shown in Fig. 5. To summarise, a main effect of misinformation condition was observed for the rodent urine story only. There were no interactions with nut (target/non-target) or time (before/after misinformation exposure) in any story. Thus, the negligible effects of fake news on attitudes towards the target nut do not appear to be specific to the spider eggs story described in Study 1, but extend to three other fake news stories.
Fig. 5Standardised attitudes to target and non-target nut as a result of exposure to each of the four fake news stories.
Studies 1 and 2 demonstrated minimal effects of a fake news story on attitudes towards nuts and actual nut consumption in the laboratory. This suggests that one-off exposure to misinformation is unlikely to have significant consequences for subsequent behaviour. There are however several outstanding questions arising from these results. We selected the low-stakes behaviour of nut consumption in Studies 1 and 2 both for ethical reasons and because the behaviour is not explicitly tied to any political, religious or moral belief systems. Real-world behaviours are, however, likely to be influenced by a combination of social, personal and informational factors; individuals are typically more likely to act on a piece of information if it is consistent with their pre-existing worldview and social identity^48,49^. We investigated this question by explicitly testing the effects of pre-existing attitudes towards nuts, but found no significant effects on behaviour. However, it is possible that participants do not hold particularly strong views about nuts, and that effects of pre-existing views would be stronger where participants have a more established ideological position. Thus, Study 3 investigated behavioural effects in the politically charged context of climate change. We expanded on the methods reported in Studies 1 and 2 by comparing effects of misinformation that might be expected to either increase or decrease the targeted behaviour. Participants were exposed to a fake news story that either supported the idea that climate change is a real and present threat, or suggested that the effects of climate change are exaggerated. We investigate effects of exposure to these stories on participants’ intentions to engage in a series of targeted behaviours as well as their actual engagement in those behaviours when given the opportunity to do so. This study was in part inspired by van der Linden^35^, who demonstrated that exposure to a two-minute video about climate change conspiracies reduced the proportion of participants who signed a “stop global warming” petition from 34% in a control condition to 23% in the conspiracy condition.
Our first research question asked how exposure to a fake news story would affect climate-related behaviours and behavioural intentions. We preregistered the following
H1 (Behavioural intention): Compared with a control condition, participants who see a fake news story reflecting belief in climate change story will report stronger intentions to engage in hypothetical pro-climate actions, while participants who see a fake story reflecting skepticism about climate change will report weaker intentions to engage in the same behaviours.H2 (Actual behaviour): Compared with a control condition, participants who see the climate-belief story will be more likely to engage in specific online behaviours relating to climate change, while participants who see the climate-skeptic story will be less likely to engage in the same behaviours.H3 (Intention vs. actual behaviour): The proportion of participants opting in to real online behaviours will be lower than the proportion who report being likely to engage in a hypothetical version of that behaviour.
Our second question asked how pre-existing opinions about climate change would influence (a) formation of false memory for a fake news story, (b) intentions to engage in the targeted behaviours, and (c) actual online behaviour. We predicted
H4: Participants who report stronger belief in climate change will be more likely to form a false memory for the climate-belief fake news story, while participants who report stronger belief in climate change will be less likely to form a false memory for the climate-skeptic fake news story.H5: Participants who form a false memory for the fake news story will report lower intentions to engage in the hypothetical behaviours.H6: Participants who form a false memory for the fake news story will be less likely to engage in the online behaviours.H7: The effect of false memories on behavioural intentions and actions will be moderated by initial attitudes towards climate change.
The hypotheses, research design and analysis for Study 3 were preregistered at https://aspredicted.org/rcry-yvc3.pdf. There were minor deviations from the preregistration (see Supplementary Materials, Table S1 for details).
This experiment received ethical approval from the research ethics committee of University College Cork, and was conducted in line with the Code of Professional Ethics of the Psychological Society of Ireland. Informed consent was obtained from all participants.
Participants in Study 3 were recruited from Prolific and paid at the rate of £6 per hour for their participation. We preregistered a sample of 414 participants, after exclusions. In total, 480 participants completed the experiment. In line with our preregistration, we excluded participants who failed two out of three attention and comprehension checks (n = 16), or who failed the check question associated with the fake news story (n = 49; see Procedure). Two participants withdrew consent following debriefing, leaving 413 valid participants. Of these, 223 (54%) were female, 183 (44.31%) were male and 7 (1.69%) identified as non-binary. The mean age was 42.10 years (SD = 13.19, range = 18–80). This sample provided 95% power to detect a small effect (f²(V) = 0.03) in the MANOVA analysis of behavioural intentions, and a small-moderate effect (w = 0.18) in the chi square analysis of online behaviours.
Participants were randomly assigned to one of three Pro-climate Participants were exposed to one of two fabricated news stories suggesting that climate change is real and is a serious problem, in addition to two true stories. Anti-climate Participants were exposed to one of two fabricated news stories suggesting that climate change is not real and that its effects have been exaggerated, in addition to two true stories. Control: participants viewed two true stories only.
All study materials may be viewed and downloaded athttps://osf.io/nygd7/.
The pro- and anti-climate change stories employed the same basic format with the specific content edited to suit a pro-climate change or anti-climate change agenda. The first story described a scientific scandal in which a prominent scientist had fabricated evidence suggesting that the global temperature either is or is not rising. The story noted that the data had been manipulated in various ways to drastically underestimate (pro-climate change version) or overestimate (anti-climate change version) the degree to which the climate has shifted over the course of the 21st century. The story was accompanied by a stock photograph of the Oxford skyline.
The second story described a new report claiming either that the ecological effects of climate change have been catastrophic, with more than 60% of plant and animal species having gone extinct in the last 30 years (pro-climate change version), or that increasing global temperatures have had very little impact on plant and animal life (anti-climate change version). The story was accompanied by a stock photograph of mushrooms growing in moss.
The two true stories shown to all participants were the avocado and plastic packaging stories used in Studies 1 and 2.
Study 3 was completed entirely online on the Qualtrics platform. Participants were informed that the purpose of the study was to assess media consumption, opinions on climate change, and concerns about sustainability; there was no mention of misinformation or fake news.
After providing demographic details, participants were asked to rank a list of sources from most to least likely to rely on them for climate-related news. This question was included to maintain the study ruse and was not relevant to our aims. Participants then rated their belief that “the earth’s climate is changing as a result of human activities” on a scale from 1 (no, definitely not) to 10 (yes, definitely).
In the next section participants viewed the news stories (experimental one fake story + two true stories; control two true stories) in random order. After each story, participants were asked “Have you come across this story before?” and answered using the same five options listed in Studies 1 and 2. An attention check was included after each story in which participants were presented with a list of four headlines and asked to choose the one which most accurately captured the content of the story. The attention check question was presented on the same page as the news story. In each case, there was one clearly correct answer and three incorrect answers.
The next section assessed the study outcomes. First, behavioural intentions were assessed via a Likert scale question in which participants were asked how likely they were to engage in a list of hypothetical climate-related behaviours over the next month, rating each on a five point scale from 1 (extremely unlikely) to 5 (extremely likely). The behaviours
Sign up to a mailing list to keep up to date with climate-related news.Donate money to a climate-change related organisation.Attend a climate-related protest.Sign a petition against climate change.Make climate-conscious purchases when grocery shopping.
To assess actual online behaviour, participants were next told that we were partnering with an umbrella network of climate activists and offering a chance to directly engage in supporting their work. Participants were asked to opt in or out of each of the following behaviours by selecting “Yes” or “No”:
Tick here to sign up for a climate change mailing list (we will contact you via Prolific messaging to sign you up).Tick here to sign a petition for your government to commit more money to climate change efforts (you can select your nationality and read the petition in full before signing).Tick here to donate 20% of your Prolific payment for this study to a climate change cause (you can select the specific charity before donating).
Finally, participants were asked to rate the extent to which each of the stories they had previously seen would motivate behaviours related to climate change, on a scale from 0 (this story would make people much less likely to take action to stop climate change) to 100 (this story would make people much more likely to take action to stop climate change).
Following the completion of the study, participants were fully debriefed. The truth of each of the fabricated stories was fully explained, and links to reliable information were provided. Participants were asked to re-consent to the inclusion of their data following the debrief; two participants withdrew consent and were removed from analysis.
To evaluate the effect of exposure to pro- and anti-climate change fake news stories, we conducted a multivariate analysis of variance in which the independent variable was news condition and the dependent variables were the Likert scale responses to the five hypothetical behaviours. There was no significant multivariate effect (F(2,410) = 1.19, p = .29, Wilk’ lambda = 0.97). Univariate ANOVAs showed a small but significant effect for the intention to make climate-conscious shopping decisions, but no significant effects for the other four intentions (see Table 8). Bayes factors are also reported for each analysis and indicate anecdotal evidence in favour of the null hypothesis for the shopping intention and moderate to strong evidence in favour of the null hypothesis for all other intentions. Separate MANOVAs for participants exposed to the scientist and ecology stories may be found in supplemental materials, Table S7, but revealed no significant effects.
Post hoc tests on the shopping intention variable revealed a significant difference between the anti-climate change and pro-climate change conditions, such that participants who viewed the anti-climate change stories reported reduced intentions to make conscious shopping decisions (M = 3.55, SD = 1.23) compared with the pro-climate change condition (M = 3.91, SD = 0.99). Bayesian analysis indicated anecdotal evidence in favour of the alternative hypothesis for this comparison, BF10 = 2.86. The mean for the control group (M = 3.77, SD = 1.07) was intermediate between the pro- and anti-climate change groups but did not differ significantly from either the anti-climate group (p = .22, BF10 = 0.45) or pro-climate group (p = .54, BF10 = 0.24).
Table 8Univariate ANOVAs examining effect of news condition (pro-climate change, anti-climate change, control) on intention to engage in hypothetical behaviours. Bayes factors indicate strength of evidence in favour of the alternative hypothesis (BF10) and the null hypothesis (BF01).Behavioural intentionSSdfMSF
p
η²~pBF10BF01~Sign up to a mailing list5.522, 4102.761.970.140.010.175.99Donate to a climate organisation1.852, 4100.930.690.500.0030.0519.43Attend a climate protest1.202, 4100.600.550.580.0030.0422.24Sign a petition against climate change6.102, 4103.051.820.160.010.156.82Make climate-conscious grocery purchases8.302, 4104.153.420.030.020.641.57**p < .05.
In an additional exploratory analysis, we included initial belief in climate change as a covariate in the analyses of variance. While climate change belief significantly predicted intentions to engage in all five behaviours, beliefs did not interact with exposure to pro- or anti-climate change messaging in their effects on behavioural intentions. See supplemental materials Table S8 for details of these analyses.
As predicted, the proportion of participants who opted into each target behaviour was significantly lower than the proportion who had previously stated an intention to engage in the hypothetical behaviour; details of these analyses may be found in supplemental materials Table S9.
To evaluate the effect of news exposure on actual online behaviour, we conducted a 2 × 3 chi square test for each behaviour to compare frequency of opt-in vs. opt-out responses across the three news conditions. There was a significant effect of news condition on the proportion of participants who opted in to sign a climate-change related petition; X^2^ (2) = 8.49, p = .01, Cramer’s V = 0.14). Post hoc pairwise chi square analyses (Table 9) revealed a significant reduction in the proportion of participants opting in in the anti-climate condition (23.4%), compared with the pro-climate condition (36.5%), or the control condition (39%). There was no significant difference between the pro-climate group and control; see Fig. 6. Bayes factors indicated strong evidence in favour of the alternative of the alternative hypothesis for the anti-climate vs. control comparison, and anecdotal evidence in favour of the alternative hypothesis for the anti-climate vs. pro-climate comparison. Separate analyses for each fake news story may be found in supplemental materials, Table 10; there were no significant effects.
There was no effect of news condition on real-world willingness to donate money (X^2^ (2) = 1.07, p = .59, Cramer’s V = 0.05) or sign up to a mailing list (X^2^ (2) = 3.24, p = .20, Cramer’s V = 0.09). See Table 9 for all pairwise comparisons.
Table 9Chi square comparisons of the proportion of participants who opted into each online behaviour, with Bayes factors indicating strength of evidence in favour of the alternative hypothesis.Online behaviourX^2^df
p
VBF10Sign up to a mailing listAnti-climate vs. pro-climate0.8010.370.060.26Anti-climate vs. control3.2110.070.110.85Pro-climate vs. control0.7510.390.050.26Sign a petition against climate changeAnti-climate vs. pro-climate5.1710.020.142.80Anti-climate vs. control7.8810.0050.1710.65Pro-climate vs. control0.1810.670.020.23Donate to a climate organisationAnti-climate vs. pro-climate1.0210.310.060.31Anti-climate vs. control0.1310.710.020.18Pro-climate vs. control0.4910.490.040.23
Fig. 6Percentage of participants opting in to each real online behaviour across news exposure conditions. *pairwise difference significant at p < .05. ns = pairwise difference not significant.
We predicted that participants who reported stronger belief in climate change would be more likely to form a false memory for the climate-belief fake news story, and less likely to form a false memory for the climate-skeptic fake news story. However, just 9% of misinformed participants formed a false memory for the fake story, and a logistic regression restricted to those in the misinformed conditions revealed no effect of climate change belief on false memory formation (X^2^ = 0.01, p = .94, N = 254, Nagelkerke R^2^ < 0.001).
To evaluate predictors of behavioural intentions, we conducted a linear regression for each intention rating with the predictor variables of (1) initial belief in climate change, (2) presence or absence of a false memory for the fake news story and (3) the interaction between these predictors. Coefficients from each regression are listed in the supplemental materials, Table S11. In summary, climate change belief significantly predicted all five behavioural intentions, with stronger beliefs leading to stronger intentions. There were no significant effects of false memory and no interactions between climate change belief and false memory.
Logistic regressions were subsequently conducted to evaluate predictors of actual online behaviour (see Table S12 for coefficients). Once again, climate change belief predicted all three actual behaviours (signing up to a mailing list, signing a petition and donating money) but there was no effect of false memory and no interaction. Thus, our final four hypotheses pertaining to false memory were not supported.
Finally, logistic regressions were conducted to examine whether climate change belief moderated the effect of misinformation condition. No significant interaction effect was observed for any of the three online behaviours (see Supplemental Materials Table S13).
The aim of the present set of studies was to investigate the effect of a single exposure to a fake news story on attitudes and real-world behaviour. Previous research suggests that social identity and political ideology have a significant effect on belief and memory for misinformation^18–20,49,50^. In Study 1, we hoped to isolate the effects of misinformation exposure from this wider political and social context. We therefore selected consumption of nuts as a target behaviour that is not typically linked with any political movement or ideology. A highly powered online study (Study 1A) revealed that one-off exposure to misinformation about food contamination did not significantly affect attitudes towards the target food (almond or cashew nuts). Although trends were observed towards a reduction in positive attitudes towards the nuts following exposure to misinformation, the effects were extremely small. A replication study (Study 2) demonstrated that these null effects were not an artefact of the specific story used, but were observed consistently across four fabricated news stories. Going further than any previous research in this field^3^, Study 1B examined effects of fake news exposure on real world behaviour in a well-powered and carefully designed laboratory study. We observed no significant effects on consumption of the target food, instead finding evidence in favour of the null hypothesis.
The news stories used in Studies 1 and 2 were designed to elicit disgust about the prospect of eating the contaminated food, an important factor in determining subsequent eating behaviour^51^. Indeed, the false memory studies on which the Study 1 paradigm was based had previously shown that implanting a false memory that a food had once made the participant sick resulted in a subsequent reduction in consumption of that food^36–39^. While those studies used a detailed personal profile to convince participants of their past experiences, it is possible that a single exposure to generic, non-personalised news was insufficient to produce behavioural effects. Although pilot data suggested that the stories in the present study were convincing and considered likely to affect behaviour, the actual impact of exposure to the story on attitudes towards and actual consumption of the targeted nuts was negligible. Note that, while pilot participants were asked whether they believed the information in the news stories would affect their behaviour, the behaviour itself was not assessed. The behaviour-intention gap is very well-documented^52–56^, and pilot participants’ estimates of what they might do in a hypothetical future situation should not be expected to correlate perfectly with actual behaviour. We might conclude from this that exposure to misinformation simply does not affect real-world behaviour, but it is important to consider some alternative explanations. First, it is possible that participants were not convinced by the specific news stories about nut contamination (though pilot data indicated that the stories were plausible and persuasive) or that they had forgotten the misinformation provided to them in the online survey by the time of the lab study one week later. Moreover, even if participants believed and remembered the news story, they may not have believed that academic researchers would knowingly give them contaminated food, and may have assumed that the nuts provided to them were safe to eat. Second, Studies 1 and 2 only included misinformation designed to decrease the targeted behaviour, and larger effects might have been observed with positive information (e.g., suggesting that nuts have health benefits). Finally, while we selected the ideologically unaligned behaviour of nut consumption in order to isolate the effect of misinformation exposure, stronger pre-existing beliefs and attitudes might be required for misinformation to have detectable effects.
To address these issues, Study 3 examined consequences of misinformation exposure in the highly polarising and politicised context of climate change, and measured participants pre-existing beliefs on this topic. To rule out effects having to do with a specific story, we randomised participants to receive one of four news stories which were designed to either increase or decrease their engagement in pro-climate behaviours. To ensure that any effects (or lack thereof) were not limited to a single outcome, we investigated effects on five behavioural intentions and three real online behaviours. We observed null effects (with strong Bayesian evidence in support of the null hypothesis) for four out of five intentions, and two out of three behaviours. The positive results were observed in respect of different participants were less likely to sign a pro-environmental petition after viewing anti-climate change misinformation, but there was no difference in their previously stated intention to engage in this behaviour. Instead, the only observed effect on behavioural intentions related to the intention to make climate-conscious shopping decisions after viewing pro-climate misinformation.
Interestingly, the only online behaviour to be significantly impacted by misinformation in the present study was signing the petition, a similar finding to that reported by van der Linden^35^. The other two behaviours assessed – signing up to a pro-climate mailing list, or donating a portion of the study payment to a climate change organisation – were not affected by exposure to either pro- or anti-climate change misinformation. The fact that a single exposure to misinformation was sufficient to shift the petition behaviour is interesting and potentially concerning with regard to the real-world effects of misinformation. It is perhaps notable that, regardless of experimental condition, participants were more likely to sign the petition than to engage in the other behaviours, with 33.4% of participants opting in compared with approximately 19% for the mailing list and donation behaviours. Thus, we may infer that signing the petition was seen as a relatively inconsequential behaviour in comparison with handing over hard-earned money or volunteering to receive an unspecified number of emails. This low-stakes behaviour may have been easier to manipulate than actions with perceived future consequences for participants, a finding with potential implications for more significant real-world behaviours such as voting or vaccination. On the other hand, it is possible that the manipulation described by van der Linden – in which participants viewed a two minute video about climate conspiracy theories – would have also affected the other two behaviours assessed in the present study. The conspiracy video manipulation was a relatively strong treatment in comparison with the present study, in which participants were exposed to a single fabricated news story. Future research should investigate this question in order to delineate the boundaries of misinformation how strong does a manipulation need to be for behavioural effects to be observed? Are particular behaviours easier to manipulate than others? Are multiple exposures to the misinformation required for behavioural effects to be observed, or does the information need to come from a trusted or reputable source? We strongly encourage other researchers to join us in attempting to answer these questions.
While the effects of misinformation exposure were rather weak and inconsistent, strong and consistent effects of climate change belief were observed. Unsurprisingly, participants who reported stronger initial belief in climate change reported stronger intentions to engage in all five behaviours and were considerably more likely to opt into the three real online behaviours. This result stands in contrast to the findings from Studies 1 and 2, where there was no effect of pre-existing attitudes towards nuts on eating behaviour. It seems reasonable to infer that beliefs and attitudes about climate change – a topic that is repeatedly discussed in media – were stronger and more deeply held than beliefs about nuts, and that these stronger beliefs were more likely to drive behaviour. There was however no interaction between climate change belief and misinformation condition, even for the petition behaviour where a significant main effect of condition was observed. A very important next step will be to identify the social, political and personal factors that moderate the effect of misinformation on behaviour. This is a question that must be addressed using experimental methods to ensure that the causal relationship is understood.
In summary, across three preregistered, well-powered experiments, the present work demonstrates limited and inconsistent real-world effects of one-off fake news exposure. While we are convinced that large-scale and consistent exposure to misinformation (especially from congenial or ideologically aligned sources) may result in significant harms, it is important to identify boundary conditions that may limit these effects. We echo previous calls [e.g.^57^] to temper our discussions of misinformation and avoid jumping to overblown conclusions about the effects of misinformation, however it is our hope that future research will build on the present findings to identify the specific conditions under which misinformation has clear and unambiguous effects on behaviour.
Below is the link to the electronic supplementary material.
Supplementary Material 1