Authors: Megan Shiroda, Clare G.-C. Franovic, Joelyn de Lima, Keenan Noyes, Devin Babi, Estefany Beltran-Flores, Jenna Kesh, Robert L. McKay, Elijah Persson-Gordon, Melanie M. Cooper, Tammy M. Long, Christina V. Schwarz, Jon R. Stoltzfus
Categories: General Essays and Articles
Source: CBE Life Sciences Education
Causal mechanistic reasoning is a thinking strategy that can help students explain complex phenomena using core ideas commonly emphasized in separate undergraduate courses, as it requires students to identify underlying entities, unpack their relevant properties and interactions, and link them to construct mechanistic explanations. As a crossdisciplinary group of biologists, chemists, and teacher educators, we designed a scaffolded set of tasks that require content knowledge from biology and chemistry to construct nested hierarchical mechanistic explanations that span three scales (molecular, macromolecular, and cellular). We examined student explanations across seven introductory and upper-level biology and chemistry courses to determine how the construction of mechanistic explanations varied across courses and the relationship between the construction of mechanistic explanations at different scales. We found non-, partial, and complete mechanistic explanations in all courses and at each scale. Complete mechanistic explanation construction was lowest in introductory chemistry, about the same across biology and organic chemistry, and highest in biochemistry. Across tasks, the construction of a mechanistic explanation at a smaller scale was associated with constructing a mechanistic explanation for larger scales; however, the use of molecular scale disciplinary resources was only associated with complete mechanistic explanations at the macromolecular, not cellular scale.
Fostering knowledge integration across disciplines and contexts is key to preparing students to be productive scientists and informed citizens following graduation (American Association for the Advancement of Science, 2011; Cooper et al., 2015; Ledford, 2015). The American Association for the Advancement of Science Vision and Change: Call to Action cited the importance of both core concepts and competencies, including the ability to leverage knowledge both within and outside of biology to interpret biological phenomena (2011). In prehealth fields, the Association of American Medical Colleges and Howard Hughes Medical Institute put forth a report stressing the importance of focusing on core scientific competencies rather than a progression of unconnected disciplinary coursework (2009). To make this goal more achievable, the National Research Council developed the Three-Dimensional Learning (3DL) framework for K–12 students (2012). 3DL incorporates core ideas, scientific practices, and crosscutting concepts to aid teachers in engaging students in meaningful, lasting learning experiences. 3DL has been successfully extended to undergraduate STEM courses, including introductory biology and chemistry (Cooper et al., 2015; Matz et al., 2018).
Constructing explanations is one of the seven scientific practices in 3DL (National Research Council, 2012). In our work, we focus on mechanistic explanations that require unpacking entities at scalar levels below the phenomenon and linking these entities to the phenomenon through causal chains (Krist et al., 2019). This type of mechanistic explanation includes cause and effect in a defined spatiotemporal system, includes causal connections that lead to the phenomenon, and requires deductive use of core disciplinary ideas (e.g., aligned with a Causal Mechanical Deductive Nomological Explanation in Alameh et al., 2023). Constructing these mechanistic explanations is three-dimensional because it requires using disciplinary core ideas and the crosscutting concept of cause and effect to perform the scientific practice of constructing explanations (Laverty et al., 2016).
When constructing mechanistic explanations, one must identify entities scalar levels below the phenomenon, unpack properties and interactions at that lower level, and link these unpacked factors into causal chains using causal mechanistic reasoning (Russ et al., 2008; Krist et al., 2019). This type of thinking allows one to connect and apply prior knowledge to explain or predict new phenomena and can help students prioritize and integrate important pieces of knowledge within and across their courses (Krist et al., 2019) and organize knowledge in more expert ways (Southard et al., 2016). For example, students can use causal mechanistic reasoning about protein structure and function to connect molecular and cellular biology and causal mechanistic reasoning about mutations to connect genetics and evolution (van Mil et al., 2013, 2016; Bray Speth et al., 2014; Villafañe et al., 2021; Bhatia et al., 2022). Thus, encouraging the construction of mechanistic explanations using causal mechanistic reasoning is an important instructional approach that fosters knowledge integration across contexts and can serve as an important sensemaking and problem-solving tool for STEM students.
In addition to using causal mechanistic reasoning within disciplines, the complex problems STEM graduates will face in their careers and society require that they move beyond the siloed use of causal mechanistic reasoning in one discipline and connect ideas across courses and disciplines into multifaceted mechanistic explanations that integrate core ideas from disparate disciplines into their explanations. Multifaceted mechanistic explanations differ from multilevel molecular explanations (van Mil et al., 2013), which move between entities at different scales (e.g., molecules through cells) in that they require leveraging of different core ideas to explain a phenomenon occurring at a single level. Multifaceted mechanistic explanations are also an extension of hybrid hierarchical explanations, which require mapping across ontologically distinct levels, such as informational and physical levels (Duncan and Reiser, 2007). In addition to mapping across these levels, multifaceted mechanistic explanations also require the use of disparate core ideas, often from different disciplines. For example, in the molecular genetics hybrid hierarchical explanations described by Duncan and Rieser (2007), students must connect the informational level (information in genes) with the physical level (amino acids in proteins), both ideas contained within the core biological idea of information flow (Laverty et al., 2016). To become a multifaceted mechanistic explanation, the hybrid hierarchical explanation described by Duncan and Rieser (2007) needs to be extended to include additional core ideas such as structure and properties, electrostatic interactions, and structure and function to explain how and why proteins create physical effects in cells. As an example of the importance and power of connecting ideas across disciplines, integrating electrostatic properties, a core idea from chemistry, into predictions regarding structure-function relationships in cell and molecular biology allows for accurate predictions of how mutations change protein structure and function and which binding sight is more likely to interact with a particular ligand (Halmo et al., 2018; Franovic et al., 2023a). Another example of a multifaceted mechanistic explanation is an explanation of how and why ATP serves as an energy source in cells. This explanation involves the core ideas of energy, electrostatic interactions, molecular structure and properties, and structure and function (Franovic et al., 2023b). Because the construction of multifaceted mechanistic explanations requires integrating important but disparate pieces of knowledge to explain complex phenomena, these types of explanations provide students opportunities to practice using causal mechanistic reasoning across disciplines and provide instructors insights into if and how students are connecting and integrating knowledge across courses in a STEM curriculum.
Previous work reveals that STEM students can and do construct mechanistic explanations but often encounter difficulties and need additional support. Biology students frequently struggle to construct mechanistic explanations in molecular genetics (Duncan and Reiser, 2007; Bray Speth et al., 2014), in cell biology (van Mil et al., 2013; Southard et al., 2017), and when explaining everyday phenomena (Scott et al., 2018). When constructing mechanistic explanations, students sometimes conflate aspects of mechanisms across similar biological phenomena and sometimes make associative connections that are a barrier to causal mechanistic reasoning. However, when they do make causal connections, they rarely incorporate inappropriate entities in their explanations. (Southard et al., 2016). Previous work from our group found the percentage of STEM students who identified appropriate chemistry ideas and used causal mechanistic reasoning to explain protein-ligand binding varied by course, with almost no complete mechanistic explanations from students completing the first semester of general chemistry and roughly half of students from upper-level biochemistry constructing complete mechanistic explanations (Franovic et al., 2023a). Importantly, this study also showed a high correlation between the construction of complete mechanistic explanations and correct predictions regarding protein-ligand binding, reinforcing the power of causal mechanistic reasoning as a predictive tool. Previous research also shows that connecting ideas across disciplines and coursework can be challenging for students (Villafañe et al., 2011; Haudek et al., 2012; Cooper and Klymkowsky, 2013b; Roche Allred et al., 2021). For example, when interviewed, students who are coenrolled in introductory biology and chemistry courses can synthesize connections between their courses for some crosscutting concepts, such as structure and function, but not others, such as energy transfer (Kohn et al., 2018a, 2018b). Taken together, these studies illustrate that while using causal mechanistic reasoning to construct mechanistic explanations is challenging, students can do this, and there are benefits when they do.
Fortunately, previous work also shows that instruction specifically aimed to improve causal mechanistic reasoning can do so (Bray Speth et al., 2014; Dauer and Long, 2015; van Mil et al., 2016; Hester et al., 2018; Crandell et al., 2019; Halmo et al., 2020), and it is likely that coordinated curricula can help students maintain the use of causal mechanistic reasoning across different courses (Crandell et al., 2019). Further development of instruction and curricula is needed to support students as they learn this critical scientific practice. This will require additional knowledge of how students construct multilevel mechanistic explanations, nested hierarchical mechanistic explanations, and multifaceted mechanistic explanations that span disparate core ideas.
Studying mechanistic explanations across courses and disciplines presents challenges for researchers. Challenges include differences in the expectations for causal mechanistic reasoning and in how each discipline frames shared ideas (Osborne et al., 2018; Cooper, 2020; Schwarz et al., 2020). For example, instructors from biology, chemistry, and biochemistry each had varying definitions and learning objectives surrounding structure and function in relation to enzyme binding (Yoho et al., 2019). Another set of related challenges to the use of causal mechanistic reasoning across disciplines is the presence of explanatory black boxes and bottoming out in many explanations, especially in nested hierarchical explanations (Machamer et al., 2000; Haskel-Ittah, 2022). Nested hierarchical explanations are those in which the entities, activities, and properties at lower levels produce higher-level phenomena that are themselves entities, activities, and properties that produce even higher-level phenomena. Explanatory black boxes acknowledge the causal connection between two entities in an explanation but do not explore the underlying mechanism. Which aspects of a mechanism must be unpacked and which can be left in a black box depends on the purpose of the explanation and varies by discipline and context. For example, van Mill (2013, 2016) acknowledges the potential importance of chemistry in molecular biology mechanistic explanations, but they use a machine metaphor for describing protein function instead of unpacking the relevant chemistry factors in the mechanism. The use of such metaphors or other black boxes is common in science as not every idea needs to be unpacked while constructing an explanation (Haskel‐Ittah, 2022). However, while it may be appropriate and even necessary to black box chemistry-related content in some introductory biology contexts, undergraduate cell and molecular biology instruction for STEM majors will likely benefit from removing this black box (Halmo et al., 2018). Some of this difference in acceptable black boxing is due to the lowest-level mechanism considered interesting for the person evaluating the explanation, a phenomenon termed “bottoming out” (Machamer et al., 2000) Because chemists, molecular biologists, and organismal biologists often focus on different scales of shared phenomena, their explanations often bottom out differently. For example, a cellular biologist may be interested in the presence or absence of a magnesium uptake phenotype, while a molecular biologist is interested in the protein that allows the magnesium uptake and a chemist is interested in the molecular properties that allow the protein to bind the magnesium ion. This creates challenges both for students and instructors as they try to create connections across courses because expectations around acceptable black boxing may be unclear to instructors and are rarely articulated to students.
Differences in how members of our research team with backgrounds in chemistry, molecular biology, and organismal biology approached causal mechanistic reasoning (Schwarz et al., 2020) led our team to focus on the Essential Epistemic Heuristics for Guiding Mechanistic Reasoning developed by Krist et al. (2019) which incorporated ideas from Russ et al. (2008) and van Mil et al. (2013) and has been used by others to study mechanistic explanations generated by undergraduate STEM students (Uhl et al., 2024). This framework allowed us to examine causal mechanistic reasoning of different but related phenomena through different disciplinary lenses and categorize student explanations as described by Noyes et al. (2022). At the holistic level, student responses fall into three general those that include appropriate factors and link them to the phenomenon in appropriate mechanistic ways (complete mechanistic explanations); those that are missing key factors or do not link them in an appropriate mechanistic way (partial mechanistic explanations); and finally, those that do not include any relevant factors (nonmechanistic explanations; Table 1). These categories are similar to Alameh et al.’s (2023) quality continuum in that disciplinary experts deemed the complete mechanistic explanations to contain the required relevant components, to accurately align with canonical scientific understanding of the mechanism, and to have appropriate interconnections. Partial mechanistic explanations are mostly or partially adequate while nonmechanistic explanations are inadequate. Our categories also align well with segments from Southard et al.’s (2016) theoretical knowledge integration continuum for undergraduate cellular and molecular biology, with complete mechanistic explanations falling in the connected segment of the continuum, partial mechanistic explanations falling in the transitional segment of the continuum, and nonmechanistic explanations falling in the fragmented segment of the continuum. Because the key factors are based on core disciplinary ideas and the linking is based on causal mechanistic reasoning, these categories provide insights into whether and how students are using disciplinary resources and epistemic resources (Hammer, 2000; Hammer et al., 2005). Therefore, each category has important implications for instruction as they demonstrate whether students are leveraging disciplinary resources appropriately and whether they connect these resources using the epistemic resource of causal mechanistic reasoning. This provides insight into the types of additional support students may need to construct mechanistic explanations (Franovic et al., 2023a). In addition, these categories allow the examination of relationships between successful construction of a mechanistic explanation at one scale with the successful construction of a mechanistic explanation at another scale within a multiscale nested hierarchical mechanistic explanation.
Our research team designed a prompt to elicit a multiscale nested hierarchical mechanistic explanation that included the opportunity to construct a multifaceted mechanistic explanation, collected responses from undergraduate students enrolled in chemistry, biology, and biochemistry courses, and analyzed responses to understand how students in different disciplinary contexts and course levels construct mechanistic explanations. Leveraging our crossdisciplinary expertise and experience, we asked the following research
To gain a better understanding of how students use resources to construct mechanistic explanations and multifaceted mechanistic explanations that rely on mechanistic understanding of electrostatic interactions, protein structure and function, transcription and translation, and genotype-phenotype relationships, we designed a scaffolded, three-part prompt that requires productive use of causal mechanistic reasoning (an epistemic resource) and multicourse content knowledge (disciplinary resources) to explain protein-ligand binding at the scales commonly studied in chemistry (molecular), molecular biology (macromolecular), and organismal biology (cellular). This multiscale nested hierarchical prompt focuses on protein ligand-binding and requires students to leverage disciplinary resources based on core ideas from chemistry (electrostatic interactions and structure and properties) and biology (structure and function, information flow, and evolution; Laverty et al., 2016). Each task in the prompt has unique disciplinary perspectives on mechanistic reasoning brought by three different disciplinary experts who teach chemistry, cell and molecular biology, and organismal biology courses. The scaled structure of the tasks allows us to study how an individual student explains related phenomena at different scales. This prompt extends previous work by other groups on causal mechanistic reasoning in similar contexts (Bray Speth et al., 2014; Southard et al., 2016, 2017; van Mil et al., 2016; Halmo et al., 2020; Noyes et al., 2022; Franovic et al., 2023a) by explicitly exploring connections between mechanistic explanations using electrostatic interactions and those using information flow across molecular, macromolecular, and cellular scales.
Our work relies on the resource perspective, a constructivist approach to learning that recognizes fine-grained cognitive units or pieces of knowledge (Hammer, 2000; Hammer et al., 2005), as a lens to guide our exploration of whether and how students construct mechanistic explanations and multifaceted mechanistic explanations across a STEM curriculum. Resources needed for causal mechanistic reasoning include both disciplinary and epistemic resources (Krist et al., 2019). Disciplinary resources include the knowledge of entities and the activities and relationships of those entities needed for unpacking them. Epistemic resources include using causal mechanistic reasoning to link the disciplinary resources coherently and logically into a mechanistic explanation of how and why a phenomenon occurs. The resources approach also appreciates the dynamic connections that students form among these resources by examining whether and how students use these resources productively in certain contexts (Hammer, 2000; Hammer et al., 2005). This perspective has been particularly useful in understanding student thinking across disciplines as it recognizes students call upon or activate related resources when learning in related contexts and asserts that students do not “transfer” intact ideas, but instead, students activate different resources in different contexts (Hammer, 2000; Hammer et al., 2005; Noyes et al., 2022). The resources perspective is consistent with the “Preparation for Further Learning” perspective (Bransford and Schwartz, 1999) and the knowledge integration perspective (Clark and Linn, 2013; Southard et al., 2016). Taken together, these perspectives suggest that learning requires opportunities to activate, connect, and use knowledge and provide a framework for designing effective instruction and curricula.
The framework we used to understand and analyze explanations generated by our prompt is based on the Essential Epistemic Heuristics for Guiding Mechanistic Reasoning proposed by Krist et al. (2019) and aligns with several overlapping and complementary frameworks used by others. Machamer et al. (2000) focused on entities and their activities and described the importance of nested hierarchies and bottoming out. Russ et al. (2008), building on the work of Machamer (2000), created a coding scheme for discourse analysis of first-grade students discussing falling objects. van Mil et al. (2013), also building on the work of Machamer (2000), created a framework that emphasized spatial, temporal, and hierarchical organization and separated activities and interactions to describe multilevel mechanisms present in cell and molecular biology and used this to analyze explanations from high school students. Building on the work of van Mil, Southard et al. (2017) interviewed undergraduate STEM majors and analyzed their explanations for generative mechanistic reasoning. In a separate study, Southard et al. (2016) used a mixed methods approach and knowledge integration theory (Clark and Linn, 2013) to analyze the identification of mechanistic entities and the nature of connections in students’ explanations. Alameh et al. (2023) used structural elements to develop a framework for distinguishing types of explanations. Our group settled on the Krist framework as it was simple enough to use across our disciplinary perspectives while still capturing aspects of causal mechanistic reasoning that all members of the research team felt were essential in mechanistic explanations.
We developed a 10-question prompt using an iterative, evidence-centered design process (Mislevy et al., 2003), which we have previously described in detail (Noyes et al., 2022). Briefly, our collaborative group of biologists, chemists, and teacher educators discussed topics that could provoke key ideas across introductory chemistry, cell and molecular biology, and organismal biology courses in a scaffolded set of tasks that require related content knowledge (disciplinary resources) and use of causal mechanistic reasoning (an epistemic resource) to answer. Protein Structure Function is an important core concept that spans chemistry and biology and provides the opportunity to link phenomena at different scales to examine student thinking. The prompt we designed contains three tasks at three different molecular, macromolecular, and cellular. Questions 1 to 3 present students with two versions of “Protein M” that have two slightly different binding sites with relevant amino acid side chains shown (Supplemental Figure 1). Students are asked which binding site is more likely to bind to a magnesium ion and to explain their answer by drawing the Mg^2+^ in the better site and providing a written explanation. This task has been previously described by Noyes et al. (2022) and Franovic et al. (2023a). Here, we refer to this task as the molecular task. Questions 4 to 6 focus on the macromolecular scale and ask students to consider a different molecule (Molecule X) that binds a different protein (Protein X; Supplemental Figure 2). Students draw a potential binding site for the molecule, showing how the molecule binds, and then explain why a different gene is needed to produce the differences in Protein X compared with Protein M. Finally, students are asked to explain how and why these DNA differences lead each protein to bind either Molecule X or a magnesium ion. We refer to this task as the macromolecular task. Finally, in questions 7 to 10 students consider two bacteria (Type A and B) that have two different versions of Protein M (1 and 2) in their membrane. In these questions, students must explain how the DNA between Bacteria A and B would differ in comparison to each other and how that causes Bacteria B to more effectively take up magnesium ions (Supplemental Figure 3). We refer to this task as the cellular task. In comparison to the macromolecular task, this task focuses on the organism’s phenotype and not just the preferential binding of the ligand. Importantly, as explanations moved up a scale, students were explicitly asked to “Include ideas from your answers to the previous questions” in their explanations in hopes of helping them connect to and use disciplinary resources from the lower level in their explanation at the higher level.
In the spring semester of 2020, we administered the 10-question prompt to students in seven different courses at a large, public, research-intensive midwestern university in accordance with IRB protocol (IRB#x17-793e). Students were provided with a small amount of course credit for completing the task but were assured grading was effort-based. Participation remained voluntary as students could opt out of the study and still complete the task for credit. Responses were given a random identifier using a random number generator to remove any identifying information from the responses. The six courses included in this study are general chemistry 1, general chemistry 2, molecular biology, organismal biology, organic chemistry 2, and biochemistry 2. Because students do not necessarily take these courses in a specified order and could be coenrolled in multiple courses, their pathway through the curriculum varies. Students are required to take general chemistry 1 before general chemistry 2. They are also required to either have already completed or concurrently enroll in general chemistry 1 to enroll in molecular biology. They are required to take molecular biology before organismal biology. To take organic chemistry 2, students need to have taken general chemistry 1 and the first semester of organic chemistry (not included in this work) but have no requirements for a biology course. To be in biochemistry 2, students must have taken biochemistry 1, molecular biology, organic chemistry 2, and at least be concurrently enrolled in organismal biology. The most likely progression of course work is general chemistry 1, general chemistry 2 and molecular biology, organismal biology, organic chemistry 2, biochemistry 2; however, the order of courses does vary by student.
Responses were collected at the beginning and/or end of the semester, depending on the course instructor. In March 2020, all courses switched to an online format due to the COVID–19 pandemic. We recognize that the transition to online work likely impacted student performance in comparison to other semesters. All responses, whether collected before or after the COVID transition, were collected virtually using an online assessment platform called beSocratic (Bryfczynski et al., 2015), which allows students to draw or provide written explanations depending on question requirements. This collection resulted in 4092 responses after the removal of incomplete or uncodable responses. From this larger data set, a random sample of 395 responses that equally represented the courses and time points available were examined for a cross-sectional study (Franovic et al., 2023a). This previous work demonstrated this subset was not different from the larger data set based on cumulative GPA. In this study, we used only the responses given at the end of the semester so that responses were indicative of student thinking at the end of the associated course. We also excluded any responses that were not a student’s first attempt at the task to make the responses more comparable across courses (n = 105). Finally, a closer examination of the data set revealed that some students had copied and pasted responses from the same unknown source (n = 18). After the removal of these responses, the data set contains 272 responses from six general chemistry 1 (n = 48), general chemistry 2 (n = 49), molecular biology (n = 49), organismal biology (n = 29), organic chemistry 2 (n = 47), and biochemistry 2 (n = 50).
For each task in the larger prompt, student responses were analyzed using a combination of analytic (Table 1) and holistic (Table 2) coding using an approach similar to that of Jescovitch et al. (2021). In-depth descriptions of the task and coding scheme development for the molecular task have already been published (Noyes et al., 2022; Franovic et al., 2023a), and the same approach and methodology were used to develop the macromolecular and cellular tasks and coding schemes at the same time the molecular task and coding schemes were being developed. Briefly, categories for analytic coding were determined by a combination of inductive and deductive approaches, with each category representing an important factor in the mechanistic explanation. Initially, the research team created ideal mechanistic explanations for each task and created analytic categories based on the factors (entities and their unpacking) present in these ideal explanations. Each task and the ideal explanation was created by members of disciplinary subgroups (Authors C.G.C.F., K.N., and M.M.C. – chemistry; Authors J.K., R.L.M., and J.R.S. – molecular biology; Authors J.D.L., E.B.F., E.P.G., and T.M.L. – organismal biology) and discussed by the entire research team to reach a common understanding of how each discipline thought about mechanistic explanation for their task and why the ideal explanation was mechanistic. This was followed by iterative rounds of deductive coding and discussion by the disciplinary subgroups and larger discussions by the entire research team that added and eliminated analytic categories based on what was seen in student responses. Holistic coding determined whether students linked the factors in the analytic categories together to explain how and why the phenomena occurred. Initially, the criteria for a complete mechanistic explanation were based on how the factors present in the ideal explanations developed by the research team were linked together. During coding, what counted as a complete mechanistic explanation was expanded to include additional ways students linked together factors to create coherent mechanistic explanations that agreed with a canonical disciplinary understanding of the phenomenon but that were not initially anticipated by the research team. Common factors and examples of complete mechanistic explanations for each prompt are provided in Table 1. In the macromolecular task, there were two ways, information flow or protein properties, for a response to be characterized as a complete mechanistic explanation, as described in Table 1. In addition, explanations that contained complete mechanistic explanations for both information flow and protein properties and linked these together were coded as multifaceted mechanistic explanations. These three types of responses were all grouped as complete mechanistic explanations for the analysis unless otherwise noted; however, we provide example responses to show the distinctions between each. Explanations that included at least one factor in the analytic coding but that were not holistically coded as a complete mechanistic explanation were holistically coded as partial mechanistic explanations, and those that did include any factors in the analytic coding were holistically coded as nonmechanistic. See Table 2 for examples of non-, partial, and complete mechanistic explanations for each task. Thus, the holistic aspect of the coding scheme reflects aspects of the framework presented in Krist et al. (2019). Responses that included all three steps are complete mechanistic explanations, responses that completed the first or second steps of identifying important entities underlying the phenomenon a scalar level below and unpacking these but did not complete the third step of linking the factors were coded as partial mechanistic explanations, and responses that did not identify and unpack any appropriate entities were holistically characterized as nonmechanistic explanations because the response did not complete the first step of the framework. This coding approach was previously described by Franovic et al. (2023a) and Noyes et al. (2022) for the molecular task.
In the initial round of coding, each task was coded by at least two coders from the disciplinary subgroup responsible for that task. Each set of coders first examined responses separately, compared codes, and resolved any disagreements. Interrater reliability (IRR) was calculated using Cohen’s Kappa (Cohen, 1960) based on individual coder responses before resolving disagreements. Following the initial coding, experts from different disciplines (authors C.G.C.F. and M.S.) conducted a second round of coding. In this second round, coders evaluated responses to the Macromolecular and Cellular tasks for the molecular factors of Charge/Polarity and Attraction using the same rubric used to code responses to the molecular task to better understand whether students continued to leverage these factors at larger scales and how this impacted the construction of complete mechanistic explanations across the tasks. Example responses for each task are included in Supplemental Table 5.
In the initial round of coding, Cohen’s Kappa (κ) was 0.814 for molecular, 0.807 for macromolecular, and 0.858 for cellular. For individual chemistry ideas, κ was 0.91 for Attraction and 0.88 for Charge/Polarity. Though not standardized, a κ of 0.8 or greater is generally considered an “almost perfect” agreement, while 0.61-0.80 is a “substantial” agreement (Landis and Koch, 1977).
We calculated quantitative statistics associated with Cohen’s kappa, Pearson’s χ2, χ2, and Fisher’s Exact Test in IBM SPSS Statistics Version 27.34. For these tests, we used a significance threshold of 0.05. We also calculated Cramer’s V, which is a modified version of φ for contingency tables with more than two rows or columns, to determine the effect size (Green and Salkind, 2011). Previous work suggests that Cramer’s V values can be interpreted as small (0.1), medium (0.3), or large (0.5; Cohen, 1960). We used Fischer’s Exact Test when we were unable to use χ2 due to a small sample size for one observation (McHugh, 2013). To ensure our analysis is reflective of course and not other student demographics, we also examined whether the demographic information we had access to contributed to differences in the construction of mechanistic explanations. We did not observe a significant impact based on gender, previous achievement as determined by cumulative GPA quartile, ethnicity (White non-Hispanic vs. Other), Major, or Course grade for the molecular or cellular tasks (Supplemental Table 1). That is, the percentages of non-, partial, and complete mechanistic explanations were similar across these demographics. Within the macromolecular task, we did observe that higher course grades and higher cumulative GPAs correlated to an increased rate of complete mechanistic responses. While both correlations were significant it had a very small effect size (p < 0.05; Cremer’s V < 0.2). As we only saw these effects in certain courses, it is likely that the course and these specific demographics are interacting.
(RQ1) In what ways do students construct mechanistic explanations for three different, but related, phenomena in a multiscalar task about protein-ligand binding?
Here, we provided examples of student responses to illustrate the entities that students included in their explanation and how they unpacked and linked these entities to construct partial and complete mechanistic explanations. We also describe the most common combination of entities and linkages for each task and generalize these patterns across the three tasks.
The first task focuses on the binding site of two different versions of a protein. Analysis of a subset of the data presented here can be found as part of an in-depth analysis of this task (Franovic et al., 2023a). For this task, students can explain the preferential binding of a Mg^2+^ ion to one binding site or another by leveraging disciplinary resources that are commonly taught in chemistry courses, including electrostatic forces and interactions. Specifically, to construct a complete mechanistic explanation, students must identify and unpack entities in a way that provides evidence of an understanding 1) oppositely charged species are attracted to each other (italics), 2) polar or charged atoms and amino acids attract or bind charged ligands (bold), and 3) a larger negative charge in one binding site results in stronger attraction and preferential binding of the positive ion (underlined; Table 2). These three factors make up the three analytical codes for this task. For example, student 1051 writes, “The partial negative charge on the nitrogen of the amine group attracts the positive charge of the magnesium ion, causing an interaction that results in the binding of the ion to the protein. version 2 has more electronegative charge overall, with an amine group and an alcohol group, compared with version 1, which has just one alcohol group. The electronegativity of version 2 of the protein would be more likely to attract the magnesium ion.” In general, students who constructed complete mechanistic explanations linked the factors of Charge/Polarity and Attraction to explain protein binding and then incorporated Magnitude to specify the preferential binding (Figure 1).
FIGURE 1. Representation of mechanistic explanations in the molecular task. Student explanations could include all, some, or none of the factors (black boxes) to explain the preferential binding of Mg^2+^. Students were most likely to link Charge/Polarity and Attraction (solid line) and a subset of explanations would then also link these to Magnitude (dotted arrow). Complete mechanistic explanations linked all three factors to logically explain differential binding. An example response is included with the relevant excerpts from the response shown below each factor box.
Student responses that identify and unpack entities in at least one of the three analytical codes but that did not include all three factors or did not link them appropriately were considered partial mechanistic explanations (Tables 1 and 2). It was most common for these responses to only contain Attraction and Charge/Polarity and not Magnitude. For example, response 1149 states, “The positive charge of the magnesium and the negative charge on the oxygen attract and allow it to bind. Version 1 has structural differences which allow it to bind better.” In this example, the student does not specify what they mean by “structural differences” and omits the importance of the Magnitude of the negative charge in the binding of Mg^2+^. This type of response made up 72.8% of the partial mechanistic responses. The final common partial mechanistic response (22.2%) only included the idea of Attraction without identifying the importance of Charge/Polarity or Magnitude in the phenomenon.
The second task focuses on how two different proteins bind two different ligands. This phenomenon can be mechanistically explained using two different disciplinary resources that are commonly taught in biology Information Flow and Protein Properties. In comparison to the first task, there was more than one way that students could construct a mechanistic explanation. These fell into three general categories based on which factors the students leveraged (Tables 1 and 2; Figure 2).
FIGURE 2. Representation of mechanistic explanations in the macromolecular task. Student explanations could include all, some, or none of the factors (black boxes) to explain the that the proteins bind different ligands. Responses were equally likely to connect the factors of Genetic Information to Protein Properties or Protein Properties to Protein Function (arrows). We observed mechanistic explanations based only on Information Flow, based only on Protein Properties, or connecting all three factors into a multifaceted mechanistic explanation (bracket). An example response of a multifaceted mechanistic explanation is included with the relevant excerpts of the response with each factor box.
The first group of complete mechanistic explanations focused on disciplinary resources related to Information Flow and connected factors from Genetic Information (underlined) to factors from Protein Properties (bolded; Table 2). Identifying and unpacking entities related to Genetic Information took two forms. In one form, students would unpack that the sequence of bases holds information or describe a “code” that is responsible for the information. In the other form, students abstractly refer to the information without explaining exactly what the information is. The term “code” is commonly used in the classroom setting, but it is unclear if students understand how the information is encoded; however, the movement from physical (amino acids) to abstract (information) has been previously used to demonstrate mechanistic thinking within genetics (Uhl et al., 2024). Factors related to Genetic Information were then linked to factors related to Protein Properties (bolded). Some students described the overall shape of the protein, including student 1365, who connected the amino acids to protein folding and wrote, “The sequence of amino acids in the polypeptide determines the folding of the protein and the binding properties of the active site.” Other students connect gene sequences to specific structures in the protein. For example, student 1700 writes, “The difference in genes causes the proteins to be different due to the coding of different amino acids. These differences allow a specific binding to occur between a protein and the corresponding bonding site because of a difference in interactions between the site and the protein. These interactions are different due to the different folding that occurs when a protein is made and allows different functional groups to be exposed leading to different interactions.” Overall, 26.8% of responses were coded as complete Information Flow mechanistic explanations.
The second group of complete mechanistic explanations focused on disciplinary resources related to Protein Structure and Function. These responses mechanistically connected factors from Protein Properties (bold) to factors from Protein Function (italics) to explain how two proteins bind two different ligands. These responses were similar except in how students unpack protein structure. Broadly, students leveraged the ideas of shape, molecular structures, molecular properties including charge or polarity, and intermolecular forces such as hydrogen bonding, hydrophobic clustering, or ionic bonding. For example, student 13457 uses molecular structure and properties to explain differences in “So because molecule X is comprised of different amino acids with different tails groups, their binding affinities will vary in both what they have affinity for and how strong their affinity is. For example, Protein M has a carboxylate group that is very favored by Mg^2+^, but protein X has hydroxyl and methyl groups coming off of it so Mg^2+^ will have little if any affinity to bind these, but other molecules might, which means that protein X likely binds somewhere else.” Another student (1167) focuses on intermolecular forces, writing, “Different amino acids can bond to certain molecules and chains through varying intermolecular forces, such as hydrogen bonds, dipole bonds, hydrophobic bonds, etc. Protein X has the amino acids that would be able to bind to similar molecules or chains in molecule X but would not be able to bind to a Mg ion because the amino acids do not line up.” Overall, 24.6% of responses were coded as Protein Properties mechanistic explanations.
Finally, the third group of complete mechanistic explanations included disciplinary resources from Information Flow and Protein Structure and Function, connecting the three factors of genetic information, protein properties, and protein function, and therefore constitute a multifaceted mechanistic explanation. We found that these responses reflected the diversity we observed in the individual mechanistic explanations. Some of these responses detailed how proteins are created through transcription and translation before linking the amino acids to protein structure. For example, student 1365 writes, “Different genes contain different sequences of nucleotides that can be transcribed to mRNA. There are 61 codons (set of three nucleotides) that code for 20 amino acids and three codons that signal the end of transcription. Each codon is complementary to an anticodon on a tRNA that brings in the corresponding amino acid during translation. The sequence of amino acids in the polypeptide determines the folding of the protein and the binding properties of the active site.” The student then continues to link the binding properties to the binding function of the “Protein X binds to molecule X because the active site of Protein X has binding affinity for the shape and chemical properties of molecule X whereas Protein M binds magnesium ions for the same reason. For example, if Protein X contains lysine within the active site with an amine group in the side chain, the N of the amine would be attracted to the O of the hydroxyl group on molecule X to form a hydrogen bond.” This particular student is also very detailed in their description of the molecular properties and structures that lead to ligand binding, including the idea of attraction. Other students are more general. For example, student 1226 linked the genetic code to amino acids with different molecular properties that lead to “Different genes code for different amino acids in each protein. Different amino acids have different side chains so that different things can be attracted and bind to the proteins. The different amino acids involved in different proteins will create different attraction and repulsion patterns so that different proteins and different molecules to be able to bind to them.” Overall, 8.5% of responses were coded as multifaceted mechanistic explanations.
Responses that contained some but not all the necessary factors or did not link the factors appropriately to explain the phenomenon were coded as partial mechanistic explanations. Most commonly, these responses only included factors related to Protein Function. For example, response 1980 states, “Different genes are required to produce the difference seen in Protein M and Protein X because they exhibit different properties and structures. The different genes cause these differences by their various interactions with each other and their environment. The different gene x gene interactions and the different gene x environment interactions both result in different structures and different properties. Protein M has more positively charged (fully or partially) particles for magnesium to be electrostatically attracted to. On the other hand, Protein X has more negatively charged (fully or partially) particles for molecule X to interact with. Both of these proteins are probably present in different environments, which may also contribute to this phenomenon.” This response talks generically about the relationship between genes and proteins and protein structure but never identified any unpacked entities underlying information flow or protein structure, and therefore, was assigned neither the genetic information nor the protein properties code. It does identify the importance of charge and interactions as important in protein function (differential ligand binding) but does not explain why different proteins have different charges or how these differences arise, nor do they unpack how or why the differences in charges cause differences in binding and therefore was not coded as a mechanistic explanation. This type of response made up 34.5% of the partial mechanistic responses for the macromolecular task. A second large proportion of the partial mechanistic responses (26.1%) included both Protein Structure and Protein Function. For example, response 1333 says, “Genes cause proteins to become different shapes and bind to different things… Protein X can attract whole molecules, whereas protein M can only attract ions.” This response unpacks protein structure by describing proteins as having different shapes and includes protein function by describing differential binding. However, there is no explanation of how or why genes would cause proteins to have different shapes, nor are there any connections linking the shape of the protein to how or why this would cause the proteins to bind to different ligands and, therefore, was not coded as a mechanistic explanation. The remaining partial mechanistic explanations included all three factors (13.5%), only Genetic Information & Protein Function (12.6%), or only Genetic Information (11.8%) but did not mechanistically link them to the phenomenon. For example, students may have mentioned the idea that the two proteins have different DNA sequences but do not explain how that leads to differences in protein structure or function.
In the final task, students are asked to explain differences in Mg^2+^ uptake between two different bacterial populations. This is an expansion of the disciplinary resource of Information Flow to the organismal level, while the focus of the macromolecular task ends at protein function. Most commonly, students constructed a mechanistic explanation that connected Genetic Information (underlined), Protein Structure or Properties (bold), and organismal Phenotype (italics; Tables 1 and 2; Figure 3). We observed students using many of the same ideas that were used in the previous tasks, including ideas about the genetic code and molecular structures. For example, student 1051 writes, “DNA for type B bacteria would code for more amide groups than type A bacteria. Amide groups are more able to bind to magnesium ions than methyl groups and alcohol groups, so this would cause the type B bacteria to take in more magnesium.” Other students focused on the organismal level, describing the differences between the two bacteria using genotype and protein phenotype. Student 2036 writes, “The DNA between Type A and Type B must be different due to the phenotypes of the two respective proteins. Two different phenotypes must have two different genotypes. The proteins are the same except for the amino acid residues on the far left. So, the DNA sequences are similar except for those. The differences in the DNA sequence for version 2 allow the protein to take in more magnesium.” Overall, 28.3% of the responses were this type of complete mechanistic explanation.
FIGURE 3. Representation of mechanistic explanations in the cellular task. Student explanations could include all, some, or none of the factors (black boxes) to explain the that two bacteria have different Mg^2+^ uptake; however, responses holistically coded as complete mechanistic explanations typically included all three. It was more common for students to only link Protein Structure or Properties to Phenotype (thicker arrow). An example response is included with the relevant excerpts of the response with each factor box.
Responses that did not mechanistically link factors to the phenomenon or were missing some factors were considered partially mechanistic in the cellular task. The most common pattern (57%) for partial mechanistic responses was only creating a mechanistic link between protein structure to phenotype. For example, response 1025 uses a nonmechanistic factor, the amount of DNA, to explain the role of Genetic Information and also uses a mechanistic factor, identifying the importance of polar side chains and their impact on phenotype, to explain the role of “In comparing the two versions of Protein M, there would be an exceptionally higher amount of DNA in bacteria B. Because Version 2 of protein M has more polar side chains, it is twice as likely to pull in more magnesium to add to the protein.” Another common theme (26%) in partial mechanistic responses was including factors related to Protein Structure but not linking this to Genetic Information or Phenotype. For example, response 1040 says, “Type B bacteria probably has a structure with more bonds than type A bacteria/Stronger bonds allow for the protein to have a stronger electrostatic pull” While this response identifies the importance of electrostatic pull it is not linked to the increased uptake of Mg^2+^ by Type B bacteria. Finally, another group of partial mechanistic responses (17%) only linked Genetic Information to Protein Structure but did not continue on to connect this back to the phenomenon of increased Mg^2+^ uptake. For example, response 1163 reads, “The DNA in the bacteria will be very similar because they code for the same protein, but there will be a small difference in nucleotides that code for the functional group on the far-left side.” As this response does not unpack the importance of the functional group in increased Mg^2+^ binding, protein structure is not being used to mechanistically explain the phenomenon.
This data set includes 816 opportunities for students to construct a mechanistic explanation (272 students x 3 tasks per prompt). Of these opportunities, 165 (20%) were coded as nonmechanistic explanations, 391 (48%) were coded as partial mechanistic explanations, and 260 (32%) were coded as complete mechanistic explanations. The relatively large percentage of partial or complete mechanistic explanations shows that most responses included some component of a mechanistic explanation. Across the three tasks, responses that were only partially mechanistic were most likely missing the link between two important entities or between the entities and the mechanism.
(RQ2) How does student construction of mechanistic explanations for each task vary across courses?
To better understand how students construct mechanistic explanations at different points in a STEM curriculum, we examined the proportion of student responses that were holistically coded as complete, partial or nonmechanistic explanations in different STEM courses. (Table 1; Franovic et al., 2023a). Given that each task was constructed and coded by experts in different disciplines and was meant to reflect those course contexts, one might expect that certain classes would construct more mechanistic explanations on the task designed by faculty who teach in that course. One might also expect that student responses would progressively include more mechanistic explanations in courses that are further along in the curriculum. Instead, we observed similar patterns across the courses for all three tasks and no clear stepwise increase in complete mechanistic explanations when comparing from a lower-level through upper-level courses (Figure 4). Indeed, the observed differences among the courses were rarely statistically different (Supplemental Tables 2–4). The largest and most consistent difference is between all the other courses and the first course in the series, general chemistry 1, in which a significantly higher proportion of students construct nonmechanistic explanations in all three tasks. The second notable difference is that students in the final course in the series, biochemistry 1, constructed complete mechanistic explanations more frequently than students in lower-level courses, but this varied by task and course. No other clear patterns emerged. It is important to note that because this is a crosssectional study where the data were collected at a single time point, the data do not indicate how an individual student would perform as they progress through the curriculum.
FIGURE 4. Causal mechanistic reasoning across courses. Responses for the (A) molecular, (B) macromolecular, and (C) cellular tasks were holistically categorized as a complete mechanistic explanation whether it included and linked appropriate factors to explain the phenomenon (yellow), as a partial mechanistic explanation whether it was missing important factors or failed to link factors appropriately (blue), or as a nonmechanistic explanation if it did not contain any relevant factors (red). Courses are ordered in their most likely order through the STEM curriculum.
(RQ3) How does the construction of a complete mechanistic explanation at a smaller scale or the inclusion of smaller-scale factors in larger-scale explanations impact the construction of mechanistic explanations at the larger scales?
We examined how individual students constructed mechanistic explanations across the three scales and contexts in our multiscale nested hierarchical prompt. We analyze this data in four different ways. First, we look at patterns from individual students in the holistic codes across the three tasks. Second, we compare the probability that students who did or did not construct a complete mechanistic explanation at a lower-scale task would construct a complete mechanistic explanation for a higher-scale task. Third, we examine the probability that the construction of a complete mechanistic explanation at one scale results in the construction of a complete mechanistic explanation at other scales. Finally, we analyze whether and how the use of lower-scale factors in higher-scale explanations impacted the construction of complete mechanistic explanations at those larger scales.
Because there are three tasks each with three possible holistic codes, there are 3^3^ or 27 possible patterns in the response across the tasks. For example, a student who constructed a Complete mechanistic explanation on the molecular task might also construct a Complete mechanistic explanation on the macromolecular task and a Complete mechanistic explanation on the cellular task, a CCC pattern. A different student who constructed a Complete mechanistic explanation on the molecular task might construct a Partial mechanistic explanation on the macromolecular task and a Nonmechanistic explanation on the cellular task, a CPN pattern. These patterns are represented in the Sankey plot in Figure 5. Exact numbers are provided in Supplemental Table 6 along with individual z-statistics based on the expected versus observed incidence of each pattern. If the type of explanations students constructed in one task was completely independent from the type of mechanistic explanation they constructed in the other parts of the task, we would expect patterns in which students maintained the same type of explanation across all three tasks, CCC, PPP, or NNN, to represent 3/27, or 11%, of the responses. We found that students constructed the same type of explanation across all three tasks 29% of the time, an overrepresentation of this type of pattern. Other overrepresented patterns included PCP, NPP, CCP, and PCC. Each of these patterns are those in which students are performing similarly across the three tasks. Patterns that include both complete and non mechanistic explanations, such as PNC, CNN, and CNC were significantly underrepresented. This indicates that it is rare for students who do not leverage any mechanistic factor in one task to construct a complete mechanistic explanation in another task.
FIGURE 5. Individual student construction of mechanistic explanations across tasks. Each student’s responses for each task was holistically categorized as a complete mechanistic explanation whether it included and linked appropriate factors (yellow), as a partial mechanistic explanation whether it was missing important factors or failed to link factors appropriately (blue), or as a nonmechanistic explanation whether it did not contain any relevant factors (red). The color of each flow (yellow, blue, or red) is determined by the explanation constructed for the molecular task so that the level of each student’s responses can be traced across each task. The percentages of each category within each task are included in the nodes (white boxes).
To better understand how to aid students in constructing mechanistic explanations, we asked whether constructing a complete mechanistic explanation in the smaller scale tasks impacted the construction of a complete mechanistic explanation at a larger scale. We found that constructing a complete mechanistic explanation in the smallest scale task, the molecular task, was correlated to constructing a complete mechanistic explanation in both larger scale tasks (macromolecular Figure 6A; cellular Figure 6B). Likewise, constructing a complete mechanistic explanation at the intermediate scale of the macromolecular task was correlated to constructing a complete mechanistic explanation in the largest scale of the cellular task (Figure 6C). This indicates that students who construct a complete mechanistic explanation at a smaller scale task are more likely to construct a complete mechanistic explanation at larger scale tasks than students who did not construct a complete mechanistic explanation at the smaller scale.
FIGURE 6. Relationship between construction of complete mechanistic explanations at smaller scales and complete mechanistic explanations at larger scales. The percentage of students who constructed a complete mechanistic explanation at the smaller scale correlated to the completeness of their explanation at larger scales. The difference between students who construct complete versus partial or nonmechanistic explanations on the smaller scale molecular task is significant for A) the larger scale macromolecular task (Χ^2^ = 31.44; p < 0.00001) and B) the larger scale cellular task (Χ^2^ = 14.79; p < 0.001). Likewise, the differences between students who construct complete versus partial or nonmechanistic explanations on the macromolecular task is significant for C) the larger scale cellular task (Χ^2^ = 28.5; p < 0.0001).
To further clarify the relationship between constructing complete mechanistic explanations across tasks, we asked whether constructing a complete mechanistic explanation in any task is associated with constructing a complete mechanistic explanation in another task. By necessity, this analysis includes only students who constructed at least one mechanistic explanation (n = 133). This allows us to explore whether successfully using the epistemic resource of causal mechanistic reasoning to construct a complete mechanistic explanation at one scale leads to successful use of causal mechanistic reasoning to construct a complete mechanistic explanation at other scales. Of the students who constructed at least one complete mechanistic explanation, 60.2% constructed at least one more mechanistic explanation in another task (Figure 7). If students included two mechanistic explanations across the tasks, it was most likely the molecular and macromolecular tasks or macromolecular and cellular tasks. Students who constructed complete mechanistic explanations for both the molecular and cellular tasks but not the macromolecular task were rare. We only observed statistical evidence for an association between constructing a complete mechanistic explanation for the molecular and macromolecular tasks (Fischer’s Exact Probability; p < 0.0001), but not molecular and cellular tasks or macromolecular and cellular tasks. This indicates that constructing a mechanistic explanation in one task does not necessarily mean that a student will construct another mechanistic explanation in a different context. This reiterates that, to apply causal mechanistic reasoning to related phenomena, students still need support to leverage appropriate disciplinary resources that are required to identify, unpack, and link entities into a complete mechanistic explanation. We likely observed a connection between the molecular and macromolecular tasks because the explanation and content resources for a complete mechanistic explanation in these tasks overlap.
FIGURE 7. Students who constructed complete mechanistic explanations by task. The Venn diagram shows students who constructed complete mechanistic explanations in molecular (blue), macromolecular (orange), and cellular tasks (green). Overlapped areas are students who constructed complete mechanistic explanations in more than one task. Numbers are the count of students in each group. Percentages are out of the total number of students who constructed at least one mechanistic explanation across the three tasks.
Finally, we asked whether including smaller scale factors from the molecular task is associated with constructing complete mechanistic explanations for the macromolecular and cellular tasks. The mechanism in the cellular task bottoms out (Machamer et al., 2000) before the molecular factors of Attraction and Charge/Polarity become useful. For the macromolecular explanation, student could leverage a larger scale protein property such as shape or molecular structures to mechanistically explain the phenomenon. If student explanations included the molecular factors of Attraction or Charge/Polarity, they were grouped together with the Protein Properties analytical code (Table 2). Thus, the factors of Attraction and Charge/Polarity from the molecular coding scheme are below the lowest level mechanism considered to be relevant for the macromolecular and cellular tasks and were not a required part of the coding scheme. Therefore, in addition to the coding done using the disciplinary coding schemes, responses from the macromolecular and cellular tasks were also coded for Attraction and Charge/Polarity using the coding scheme developed for the molecular task (Supplemental Table 5). An example of a student response to the macromolecular task that would be coded as leveraging both Attraction and Charge/Polarity can be found the complete Protein Properties mechanistic explanation as the response unpacks the importance of the polar and non-polar amino acids (Table 2). In contrast, the Complete response for the cellular task mechanistic explanation in Table 2 would not be coded for Attraction or Charge/Polarity as the response only uses the molecular structure of amide group to explain the difference in Mg^2+^ uptake and does not unpack the importance of Attraction or Charge/Polarity.
Students who used one of these molecular scale factors in their macromolecular explanation were more likely to construct a complete mechanistic explanation than those who did not include a molecular factor (Figure 8; Table 3). This was true for construction of a mechanistic explanation overall or for each individual type of mechanistic explanation; however, the molecular resources of Attraction and Charge/Polarity was more strongly associated with the protein properties mechanistic explanation and the multifaceted mechanistic explanation than the information flow mechanistic explanation. In contrast, there was no correlation between including one of these molecular factors and constructing a complete mechanistic response in the cellular task (Figure 8; Table 3).
FIGURE 8. Relationship between the use of molecular factors and construction of a complete mechanistic explanation in the macromolecular and cellular tasks. The percentages of students who constructed complete mechanistic explanations when they included at least one molecular factor (black) or did not include any molecular factors (white) in their response to the macromolecular and cellular tasks. Supporting statistics are in Table 3.
In this work, we explore the use of causal mechanistic reasoning in student responses to a multiscale nested hierarchical prompt focused on protein-ligand binding. This work extends previous work on causal mechanistic reasoning by exploring whether and potentially how constructing a mechanistic explanation at smaller scales supports construction of mechanistic explanations at larger scales that require use of overlapping disciplinary resources. Our cross-disciplinary team examined differences in construction of mechanistic explanations at different scales for students in both lower- and upper-level undergraduate chemistry and biology courses. Our results provide insights into if and how students unpack important factors from both chemistry and biology to construct mechanistic explanations, how this varies at different points in a STEM curriculum, and whether and how constructing complete mechanistic explanations at one scale relates to construction of complete mechanistic explanations at other scales. These insights have important implications for instructional practices within individual courses and for building connected STEM curricula that foster integrated causal mechanistic reasoning across courses.
Student responses to our prompt included a range of disciplinary resources deemed important for constructing mechanistic explanations by disciplinary experts who teach a range of courses in the STEM curriculum at our institution (Tables 1 and 2). Using the essential epistemic heuristics for causal mechanistic reasoning identified by the Krist framework (2019), we were able to categorize student responses as nonmechanistic, partial mechanistic, or complete mechanistic explanations (Table 2). We also identified common patterns in how students use and link disciplinary resources (factors) in their responses (Figures 1–3).
One important finding from this study is that, in the context of our prompt, students do use causal mechanistic reasoning to construct mechanistic explanations. These results add to previous work focused on whether and how undergraduate STEM students use causal mechanistic reasoning. In responses to our prompt from students enrolled in courses that focus on 3DL, which includes constructing explanations using cause and effect (Matz et al., 2018), disciplinary experts who teach courses in an undergraduate stem curriculum recognized elements of causal mechanistic reasoning in 80% of responses, coding 651 of the 816 opportunities to create a mechanistic explanation as either a partial or complete mechanistic explanation with 32% (260 of the 816) of the opportunities coded as complete mechanistic explanations. Looking at our data from a different perspective shows that over 60% of students were able to construct at least one complete mechanistic response across the three tasks. These results indicate that undergraduates can and do use causal mechanistic reasoning and agree with results from Southard et al. (2017) who found that most students included elements of mechanism in their explanations, 70% of students developed a mechanistic explanation for at least part of the prompts used in their study, and almost 30% of students developed a mechanistic explanations across all parts of their prompts. This contrasts with the work of Scott et al. (2018) who found that students rarely (less than 15% of discourse turns) used a causal mechanistic framework. While methodologies in these three studies vary, based on work on framing (Berland and Hammer, 2012), we suggest the critical difference between these studies is the contexts of the prompts with some prompts activating causal mechanistic reasoning epistemic resources while others do not. The prompts used in the Scott study were “real-world” phenomenon, while the prompts in the Southard study and our study more closely reflect the type of assessments students are likely to encounter in in STEM courses and therefore appear more likely to activate the epistemic recourse of causal mechanistic reasoning and disciplinary resources associated with “schoolwork.”
While our data show that students can and do use causal mechanistic reasoning, it also reveals aspects of causal mechanistic reasoning that students find challenging. Our data show that many students struggle to identify the relevant factors needed for causal mechanistic reasoning. In 20% (165 of 816) of the opportunities to create a mechanistic explanation in our study, students failed to include any relevant mechanistic factors. Looking at the data from a different perspective, 5% of students did not include appropriate mechanistic factors in any task. Within the partial mechanistic explanations, a substantial number of students omitted important factors. For example, in the molecular task, students noted the importance of Attraction without unpacking the related factor of Charge or Polarity. Similarly, we observed responses in both the macromolecular and cellular tasks that did not include the important factors of Genetic Information or Protein Structure. Without further probing, we cannot know whether an individual student lacked the disciplinary resources or did not realize the importance of including that resource in the explanation. However, given the higher proportion of non- and partial mechanistic explanations in early coursework such as general chemistry 1, it is likely that some of these responses were written by students who do not yet have the disciplinary resources to mechanistically explain the phenomenon.
Other students include the appropriate disciplinary resources needed to construct a mechanistic explanation but do not use causal mechanistic reasoning to link them together and explain the phenomena. This was true at all three scales, but particularly at the molecular scale where students failed to use the idea of Magnitude to link resources related to Attraction and Charge/Polarity to explain differential Mg^2+^ binding. Similarly, at the cellular scale, students would connect Genetic Information to Protein Structure or Function but not link these factors to the Phenotype of one bacterium having better uptake of Mg^2+^. Only 32% (260 of the 816) of the possible mechanistic explanations linked resources together appropriately to create a complete mechanistic explanation. Our data further illustrate how difficult linking becomes when connections are required across disparate disciplinary resources to create a multifaceted mechanistic explanation. Only 8.5% of students in this study connected genetic information, protein properties, and protein function into a multifaceted mechanistic explanation. This is not surprising given previous studies illustrating the challenges students face using these disciplinary resources to develop mechanistic explanations (Duncan and Reiser, 2007; van Mil et al., 2013; Southard et al., 2016; Halmo et al., 2018).
Our data also add to the larger understanding of how construction of mechanistic explanations vary at different stages in the curriculum. In all three tasks, students from the first course in the series, general chemistry 1, had the highest rate of nonmechanistic explanation for each task (Figure 4). A plausible explanation for this result is that these students, who are typically just starting the university-level STEM curriculum, have not yet acquired the disciplinary resources needed to construct mechanistic explanations. In comparison, students from the last course in the series, biochemistry 2, were generally more likely to construct mechanistic explanations suggesting that many students who persist to this level either had, or have acquired, these resources. The remaining courses (general chemistry 2, molecular biology, organismal biology, and organic chemistry 2) had more differences among the three tasks, and we found no clear, statistically significant progression or pattern. However, even in upper-level courses, including biochemistry 2, a substantial percentage of students did not construct complete mechanistic explanations in response to the tasks. This aligns with previous findings showing trends toward increased construction of mechanistic explanations by upper-level students but that substantial percentages of upper-level students are not constructing mechanistic explanations (Southard et al., 2016, 2017; Halmo et al., 2018).
Our holistic coding (Table 2) aligns with Southard et al.’s (2016) theoretical model of knowledge integration in undergraduate molecular and cellular biology and adds additional support to the idea that student explanations of cell and molecular biology phenomena vary along a spectrum that represents fragmented to transition to connected and finally nuanced understanding of cell and molecular biology concepts. Responses coded as nonmechanistic explanations are similar to the fragmented section of the continuum where students struggle to identify appropriate entities and are unable to unpack the properties or activities of the entity in that mechanism. Responses codes as partial mechanistic explanations are similar to the transitional section of the continuum where students make errors in inclusion/exclusion of mechanism appropriate entities and have not yet firmly connected mechanisms to appropriate biological contexts. Responses coded as complete mechanistic explanations are similar to the connected section of the continuum where students identify mechanism-appropriate entities and create mechanistic chains of molecular events. Responses that contained a multifaceted mechanistic explanation move into the nuanced section of the continuum where students integrate several ideas to describe a nuanced and overarching biological principle. Results from analysis of holistic code patterns from individual students (Figure 5) add to this support. The two most common patterns in our data were students who had all partial (15%) or all complete (10%) mechanistic explanations across the three tasks. Two other overrepresented patterns included one partial mechanistic explanation and two complete mechanistic explanations or two partial mechanistic explanations and one complete mechanistic explanation, consistent with students in the segment between transitional and connected. A third overrepresented pattern was one non- and two partial mechanistic explanations, consistent with the segment between transitional or fragmented sections of the continuum. Of the students who constructed multifaceted mechanistic explanations in our study, 35% constructed complete mechanistic explanations on all three tasks and an additional 57% constructed complete mechanistic explanations in two of the three categories suggesting that these students do have a connected and integrated understanding of these disciplinary resources and are able to consistently use them in mechanistic ways. Additional support for this continuum is the lack of students who constructed two nonmechanistic explanations and one complete mechanistic explanation or one nonmechanistic explanation and two complete mechanistic explanations. Only 3% of students in this study had these patterns. Both our data and the continuum are consistent with the resource perspective (Hammer, 2000; Hammer et al., 2005) that, until one gains significant disciplinary expertise, the types of tasks used in these studies activate fine-grained resources in a context dependent manner rather than brining forward intact mechanistic chunks.
Our results highlight that, while disciplinary resources are essential for constructing the type of mechanistic explanations expected by instructors in undergraduate STEM courses, connecting these disciplinary resources using causal mechanistic reasoning is also critical. Using the theoretical model of knowledge integration in undergraduate molecular and cellular biology proposed by Southard et al. (2016), students whose disciplinary resources fall in the fragmented and transitional section of the continuum are unlikely to construct complete mechanistic explanations. However, students in the connected and nuanced sections are more likely to do so as constructing a complete mechanistic explanation requires both specific disciplinary resource and the epistemic resource of causal mechanistic reasoning. When moving up scales, the nested hierarchical nature of the tasks means students in the fragmented and transitional sections of the continuum will still be less likely to have the disciplinary resources needed to construct a complete mechanistic explanation while those who fall in the connected and nuanced sections will be more likely to have the disciplinary resources to do so at this higher level as well. Our comparison of students who do and do not construct complete mechanistic explanations across levels (Figure 6) support this as our data show that students who construct a complete mechanistic explanation at a lower scale are more likely to construct a complete mechanistic explanation at a higher scale than those who do not construct a complete mechanistic explanation at the lower scale. This is true for all three possible comparisons (molecular to macromolecular, molecular to cellular, and macromolecular to cellular).
At the same time, when we analyze only students who constructed at least one complete mechanistic explanation (Figure 7), we find an association between construction of complete mechanistic explanations at the molecular and macromolecular levels but not at the molecular and cellular or the macromolecular and cellular levels. While these results may seem contradictory from those discussed in the previous paragraph, a plausible explanation is that disciplinary resources play a larger role in constructing a complete mechanistic explanation than the epistemic resource of causal mechanistic reasoning. Students who construct a complete mechanistic explanation at one scale have demonstrated that, in that context, that they can use causal mechanistic reasoning. Constructing a complete mechanistic explanation at a different scale in the nested hierarchical prompt requires the use of causal mechanistic reasoning and some of the same disciplinary resources but also requires use of additional disciplinary resources. Students in the connected section of the continuum for one set of disciplinary resources may fall into the transitional section for other disciplinary resources and thus be unable to construct a complete mechanistic explanation at a different scale even though they have demonstrated the use of causal mechanistic reasoning at another scale. The association between the molecular and macromolecular levels but not the other levels is consistent with this interpretation, as the disciplinary resources needed at these two scales have the most overlap (Table 1). If the epistemic resource of causal mechanistic reasoning was the major driver of constructing complete mechanistic explanations, one would expect to see strong associations across all three comparisons. We cannot eliminate the possibility that the different contexts in the different tasks differentially activated epistemic resources related to causal mechanistic reasoning, but the tasks were purposely constructed using parallel language asking students to explain how and why, a strategy shown in other studies to encourage construction of mechanistic explanations (Cooper et al., 2016; Crandell and Cooper, 2022). We also recognize that the nature of the coding scheme contributes to the importance of disciplinary resources in constructing a complete mechanistic explanation. However, as previously stated, the prompts and coding schemes were developed by disciplinary experts who teach in STEM courses and thus represent what they value in mechanistic explanations, supporting the importance of disciplinary resources in mechanistic explanations in undergraduate STEM courses.
Finally, in our second set of coding we specifically looked for the use of molecular factors of Attraction or Charge/Polarity in responses to the macromolecular and cellular tasks. This analysis illustrates a place where mechanistic explanations will likely bottom out in many STEM curricula. Responses from the macromolecular and cellular tasks coded as including the molecular factors of Attraction or Charge/Polarity indicate that students have unpacked entities at the molecular scale in their response at these higher scale tasks. Our results show that inclusion of these molecular factors was not associated with construction of complete mechanistic explanations at the cellular scale (Figure 8; Table 3). These findings are consistent with the idea of bottoming out where the lower-level mechanism is not considered relevant at this scale (Machamer et al., 2000). Not surprisingly, students who use molecular factors in the macromolecular portion of the task were more likely to construct a complete mechanistic explanation (Figure 8; Table 3) as these factors directly support unpacking of macromolecular factors required for the macromolecular task. However, because these same molecular factors are at least two scales below the factors being unpacked in the cellular task, they do not directly support the unpacking of factors required for the cellular task and including the molecular factors did not support construction of a complete mechanistic explanation. This suggests Attraction and Charge/Polarity are below the level considered relevant or interesting in the cellular mechanism and black boxing these factors is unlikely to impact the explanation (Haskel‐Ittah, 2022).
This analysis provides some insight into how inclusion of smaller scale factors aids in construction of mechanistic explanations at higher levels. As seen in Figure 8, the inclusion of the molecular factors is strongly correlated to the construction of complete Protein Properties and multifaceted type macromolecular explanations. One potential way inclusion of these factors may support this increase in complete explanations is it moves students away from more generic explanations coded as non- or partial. Attraction is an activity of relevant entities, and Charge/Polarity is a property of relevant entities for the Protein Properties type explanation. As such, they support unpacking of entities relevant for this explanation, even if they were not explicitly required in the macromolecular coding scheme. We speculate that including these lower-level factors may increase student’s inclusion of relevant entities. For example, student 2110 uses molecular factors to unpack both how the amino acids vary and why this is important. “The different amino acids in the proteins are made up of side chains which have different attractive properties based on their make-up. If a side chain is polar, it will attract other polar molecules.” This contrasts with the nonmechanistic response from student 1738, “Different genes are required to produce the differences seen in protein M because each protein has a different structure and bind to different things” in which they do not unpack what the differences in protein structure are or why they are important for the phenomenon. The weak correlation between inclusion of Attraction or Charge/Polarity to complete Information Flow type macromolecular explanations could support this idea as the lower-level factors of Attraction and Charge/Polarity do not directly relate to entities important for the information flow mechanistic explanation. Further, the lack of correlation to complete cellular explanations at the larger scale indicates that, to impact the explanation’s quality at a larger scale, the factors should be in the level directly below the phenomenon. Additional studies, including student interviews, are needed to better understand how inclusion of these smaller scale factors support mechanistic explanations at the larger scale.
This work examines a relatively small sampling of student responses in a single time point at one university. We took this approach to obtain a broad understanding of the student thinking across multiple chemistry and biology classes; however, these data are representative of our institution and may not be representative of student thinking at other institutions. In particular, we have a relatively low representation of students of who do not identify as White (non-Hispanic; 21%) and first-generation students (15%). While our work did not find any effect of these demographics, our results will not necessarily apply to other student populations. Another consideration of this work is that the courses examined in this work have been transformed to actively target causal mechanistic reasoning and three-dimensional assessments (Matz et al., 2018). General chemistry 1, general chemistry 2, and organic chemistry 2 use a transformed curricula called CLUE and OCLUE that practice causal mechanistic reasoning in homework, in-class activities and assessments (Cooper and Klymkowsky, 2013a; Cooper et al., 2019). Similarly, the molecular biology and organismal biology instructors encourage students to explain how and why phenomena occur throughout the courses. Therefore, the students in this study may be better able to leverage causal mechanistic reasoning than others.
The prompt itself is also an important factor when considering the limitations of the study. Extensive work has found that even small wording changes in a prompt can affect the amount of mechanistic reasoning demonstrated in student responses (McNeill et al., 2006; Crandell and Cooper, 2022; Noyes et al., 2022; Uhl et al., 2024). This prompt was designed to elicit mechanistic explanations based on previous work within our group (Bray Speth et al., 2014; Cooper et al., 2016; Crandell and Cooper, 2022) and may overrepresent the percentage of mechanistic explanations that students would construct for prompts that contain less scaffolding. Specifically, the nesting of the prompt questions encourages students to consider the scalar level below which is an essential aspect for how we defined a mechanistic explanation (Krist et al., 2019). Despite these limitations, the work provides insight into how we can foster the construction of mechanistic explanations in undergraduate STEM Curricula.
Finally, the coding schemes themselves are a limitation as they focused on canonical disciplinary resources and did not code or measure other types of resources that students used to respond to these prompts. The coding scheme required inclusion of appropriate disciplinary resources to be considered a complete mechanistic explanation and did not measure the use of the epistemic resource in the absence of the disciplinary ideas. The research team discussed these limitations as we developed the coding schemes and came to consensus that, in our courses, inclusion of canonical disciplinary resources was essential to constructing mechanistic explanations and we were comfortable with these limitations in the coding scheme. However, we argue that our results are still valid and contribute to our understanding of causal mechanistic reasoning in undergraduate courses because the coding schemes reflect the goals of the disciplinary experts who teach courses in the STEM curriculum and therefore reflect the objectives students will likely need to meet in assessments in these courses.
Taken together, our results provide several insights that can help shape teaching and learning in a STEM curriculum.
First, our courses and curriculum need to ensure students have the relevant disciplinary resources needed to construct mechanistic explanations, that students recognize when to use which disciplinary resources, and that students can link disciplinary resources using causal mechanistic reasoning. Accomplishing this will likely require reducing the vast amount of information included in introductory stem courses and instead focusing on those core ideas at the heart of the discipline which are useful in explaining a wide range of phenomena. This will allow students to reduce the number of ideas they need to consider when constructing mechanistic explanations. It will also help them to better understand those ideas as more time will be spent learning about and using those ideas. This is in no way a novel idea, but changing how STEM courses are taught is a slow process (Stains et al., 2018), and this work supports the need for this type of change. 3DL accomplishes this change by focusing on a few core ideas rather than the breadth of disciplinary knowledge (National Research Council, 2012). The core ideas for undergraduate chemistry and biology established at MSU are well aligned with the disciplinary resources underlying the factors needed to construct the mechanistic explanations in this study (Laverty et al., 2016) and make a good starting point for conversations around changes to courses and curricula.
Second, our courses and curricula must help students recognize the importance of using causal mechanistic reasoning strategies they incorporate into “schoolwork” when approaching problems and explaining phenomena they encounter in “real-world” contexts. In other words, we need to help students activate the epistemic resource of causal mechanistic reasoning in a broader range of situations. We also need to change the curriculum so that as students move into higher-level courses, more of them construct mechanistic explanations more frequently. Based on the tenets of preparation for further learning and knowledge integration (Bransford and Schwartz, 1999; Clark and Linn, 2013), if students are repeatedly asked to construct explanations across contexts and are given appropriate feedback and opportunities for revision, their understanding of how to identify, unpack, and link factors appropriately and to recognize a complete causal mechanistic chain will likely improve. Previous work shows that instruction that specifically asks students to construct mechanistic explanations can improve causal mechanistic reasoning (Bray Speth et al., 2014; Dauer and Long, 2015; van Mil et al., 2016; Hester et al., 2018; Crandell et al., 2019; Halmo et al., 2020). MSU has been working to transform its STEM courses using 3DL, giving students more opportunities to engage in scientific practices like constructing explanations (Matz et al., 2018) and this transformation likely contributes to the types of explanations constructed by students in our study. However, as we see from our data, we need to do more if we expect most students in upper-level courses to construct complete mechanistic explanations using these core disciplinary ideas. Previous work also shows that if students are not asked to continue using causal mechanistic reasoning to construct mechanistic explanations in subsequent classes, the frequency at which they create mechanistic explanations declines (Crandell et al., 2019; Crandell and Cooper, 2022). Therefore, it is critical that as students move through the curriculum, they are continually asked to use and refine the scientific practice of constructing mechanistic explanations in most or all of their courses rather than encountering a hodge-podge of instructional approaches. And we need to support their constructed explanations in “real world” situations such as those from the Scott et al.’s study (2018), not just scenarios they likely see as “schoolwork” that are disconnected from their “everyday life.”
Third, as instructors design this integrated curriculum, they need to intentionally plan connections between courses in ways that help students understand how core ideas connect across courses and at the same time help students understand when explanations bottom out and when black boxes are appropriate and when they are not (Machamer et al., 2000; Haskel‐Ittah, 2022). For example, instructors in molecular biology courses may find it useful to explicitly require students to incorporate ideas from chemistry into their explanations while instructors in organismal biology are less likely to find this strategy useful. This type of redesign will require significant rethinking of how we teach so that we focus on the larger curriculum rather than just on individual courses.
In this work, we sought to broadly understand the ways students construct mechanistic explanations of protein-ligand binding across three scales that represent contexts presented in both chemistry and biology courses. Our analysis of student responses shows that undergraduate STEM students can and do engage in causal mechanistic reasoning and create mechanistic explanations using core ideas from chemistry and biology and that identifying and unpacking appropriate core ideas is critical to the successful construction of mechanistic explanations. However, our current curriculum only helps a minority of students developed the nuanced understanding of the core chemistry and biology ideas needed to connect unpack these ideas and engage in causal mechanistic reasoning across the scales encountered in molecular biology, indicating that additional work is needed if we want graduates who can use causal mechanistic reasoning in their lives and careers. Construction of mechanistic explanations at smaller scales can support construction of mechanistic explanations at larger scales when there is significant overlap of the core ideas used in the mechanistic explanations, suggesting that carefully designed curricula that intentionally connect these ideas may help students develop better connections between courses and construct a more nuanced understanding of the causal mechanisms underlying molecular biology. We hope this work will encourage both biology and chemistry instructors to build on student knowledge to foster mechanistic thinking across disciplines and in multiple contexts.
In this study, we grouped partial mechanistic responses into a single group. While we did describe the most common patterns we observed, these responses can be analyzed for additional difference in how students use core ideas. In future work, we intend to parse out the different patterns of factors students used in each course to better understand these facets of student thinking. We also plan to analyze additional responses to better understand how instruction and course context influence use of core ideas and construction of mechanistic explanations. This will allow for a deeper understanding of how instructors can build on the partial mechanistic explanations students already construct to encourage them to use causal mechanistic reasoning across all courses in their STEM curriculum and in the “real world” contexts they encounter in their everyday lives and careers.
This material is based upon work supported by the National Science Foundation (Grant: 1725521). Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the supporting agencies.