Optimizing clinical scientific the cohort intervention random sampling study with historical controls

Authors: Nadine D de Klerk, Phebe BQ Berben, Myrthe van der Ven, Annemarie F Fransen, M Beatrijs van der Hout-van der Jagt, S Guid Oei, Judith OEH van Laar, Edwin R van den Heuvel

Abstract

Randomized controlled trials (RCTs) are regarded as the highest level of evidence in medical research, but RCTs also have their drawbacks. Over the years, several alternative study designs have been introduced to address these problems. However, many of the alternative designs are often regarded as inferior to RCTs or currently not suitable for widespread implementation due to, for example, ethical or statistical problems. Thus, there is a need for study designs that have the same level of validity as RCTs, but are also suitable for large-scale implementation. The cohort intervention random sampling study (CIRSS) with historical controls meets these requirements, by combining the strengths of abovementioned designs. The CIRSS with historical controls has the potential to optimize implementation of promising new treatments as fluidly and rapidly as possible, representing real-world clinical population. Further research is required to address the range of analyses, implementation, issues, barriers and facilitators and ethical questions related to CIRSS.

Randomized controlled trials (RCTs) are regarded as the highest level of evidence in the field of comparative effectiveness research [1,2], but RCTs also have their drawbacks [3]. Limitations include lack of external validity (generalizability) [4], high costs [5,6], selection bias due to for instance loss to follow-up and a long recruitment period [7]. Over the years, alternative study designs have been introduced or used to improve the validity of comparative effectiveness of interventions [2], such as Zelen’s designs [8], pragmatic trials [9], cohort multiple randomized controlled trials (cmRCTs) [10], and even observational studies [11]. However, many of these alternative designs are currently not considered suitable for widespread implementation due to, for example, ethical or statistical problems. Compared with RCTs, observational studies are generally quicker, less expensive, simpler to conduct and include a broader range of patients [12]. However, due to the limitations of observational studies (e.g., unmeasured confounding factors), results of these studies are often viewed with suspicion and their use is regarded as inferior to RCTs [13], while well-designed observational studies have demonstrated the ability to reproduce RCT outcomes [13–15]. Pragmatic trials have been described as alternative to RCTs to obtain real-world evidence of an intervention in clinical practice. However, they have not been without criticism, because they are mostly applied for complex treatments [16], may result in ethical issues [17,18], have less control than RCTs (e.g., comorbidities, other treatments, no unique control treatment) [19], and they frequently use cluster randomization [20,21]. In the past years, RCTs within cohorts, as alternative to or form of pragmatic trials [22,23], have been introduced to combine the strengths of both trial designs and observational studies. Most of the research using cmRCT designs has been carried out in the field of oncology [24]. In order to improve and unite scientific research and routine clinical practice, this article describes a new study the cohort intervention random sampling study (CIRSS) with historical controls, that also combines the strengths of both RCTs and observational cohort studies.

Current study benefits & limitations

The currently most commonly used study designs for comparative effectiveness (RCT and observational cohort) as well as previously introduced alternative study designs (i.e., cmRCT and Zelen’s designs) form the foundation of our new study design. A visualization of these study designs are shown in Figure 1. Before a detailed description of the new study design is given, the benefits and limitations of these existing study designs will be described in the following five recruitment and informed consent (IC), internal and external validity, ethics, statistics and logistics.

Recruitment & IC

In explanatory and pragmatic RCTs, potential participants are informed about both the experimental and control group before randomization (pre-consent conversation). The pre-consent conversation in an RCT could potentially be an unnecessary barrier to recruitment in RCTs for both clinician and potential participant [25,26]. Clinicians may be reluctant to adequately inform the potential participant because they are hesitant to acknowledge uncertainties about alternative treatments [27] or do not (completely) inform them that the treatment they will receive is randomly chosen [10]. For participants, the decision not to participate in an RCT may stem from both a reluctance to engage in scientific research and resistance to the experimental treatment [6]. Therefore, different consent models have been suggested for pragmatic trials [26].

Furthermore, the decision to participate in the trial (apart from altruism) is to receive the experimental treatment, with a 50% certainty, because standard care is always available outside the trial [10,28]. If then placed in the control arm (the nonpreferred arm), this can result in emotional distress which can damage the physician–patient relationship [8,29], disappointment bias and noncompliance in the study arm [28,30,31].

The pre-consent conversation with abovementioned negative consequences was for the first time eliminated in Zelen’s single consent design, introduced in 1979. It is characterized by randomization without prior IC with the goal of improving clinicians’ and patients’ trial participation and to minimize bias [8]. IC for trial participation is only obtained from participants who are allocated to the experimental group. Potential participants who are allocated to the experimental treatment have 100% certainty, instead of 50% in an RCT, to receive the experimental treatment if they agree, without the uncertainty of randomization. However, the fact that control group patients are unaware of study participation is considered a major ethical concern [32,33]. Therefore, the Zelen double consent design was introduced [34]. In this ameliorated design, IC is obtained for both the experimental and control group after randomization. If a control group patient declines to receive standard care, the patient can cross over to alternative therapies, including the experimental treatment [34]. Nevertheless, Zelen’s double consent design fully eliminates the whole purpose of randomization, since each participant gets the opportunity to change to their favorite treatment.

To further ethically optimize the IC procedure, the cmRCT design was introduced [10]. This design exists of an ongoing cohort with the possibility for multiple RCTs over time. Each participant in this study design provides IC for the ongoing cohort and for potential future randomization, knowing that they might end up in an RCT. In each RCT though, only the patients randomly sampled into the intervention group and who accept this intervention, are asked to sign an additional IC.

When an RCT might be unethical or impossible, an observational cohort study is frequently performed. In a cohort study, participants, with varying exposures, are followed over a period of time and the outcome of interest is identified and treatment is allocated on the basis of what a patient needs. This follow-up can be retrospective or prospective and is sometimes combined [35]. No randomization or random sampling is applied in observational cohort studies. As a result, recruitment for observational cohort studies is often less selective and easier than for RCTs, for instance. An overview of the key differences and similarities of the study designs is given in Table 1.

Internal & external validity

Although RCTs maximize their internal validity [36], many RCTs have problems with generalizability (external validity) [37]. The study population is a poor representation of the real-world clinical population since many RCTs exclude clinically important subgroups who are hard to reach by clinicians or because of narrow entry criteria [38,39]. The goal is to determine evidence that the intervention may have benefits. With the introduction of pragmatic RCTs this limitation is often eliminated, but randomization in pragmatic trials are most often executed on groups of patients, introducing potentially statistical issues (confounding factors for groups) and losing power due to the group structure (heterogeneity of groups). Due to the reduction of selection bias in the Zelen designs and the cmRCT design, the study population is more diverse and representative [10,25,34]. Likewise, well-conducted observational cohort and case–control studies also include a broad representation of the population and have shown to provide the same level of internal validity as RCTs [13–15].

Ethics

In the pre-consent conversation in RCTs, all patients are informed about the experimental treatment with 50% chance of not receiving the treatment. Although it is beneficial to provide all patients upfront information, clinical care often seems to deviate from this since patients are rarely told about treatment options that their physicians cannot guarantee to provide [40]. However, for many years it is considered ethically required to complete counseling before randomization [10]. Furthermore, randomization implies that patients are allocated and cannot choose between treatment options due to randomization, and therefore imposes ethical challenges.

Despite improvement in recruitment and external generalizability in Zelen’s single consent design, some ethical concerns have been raised [34,41]. The participants who are randomly assigned to the control group without giving IC, are unaware of being in a trial. This is resolved in Zelen’s double consent and cmRCT designs to receive IC from all the participants, although the participants in the control group for the cmRCT may not know at what moment in time the intervention is being implemented.

A cohort study design is often performed when a RCT would be impossible or unethical [42]. Compared with an RCT, observational studies have less ethical concerns since participants have 100% certainty of the treatment due to the absence of randomization. Treatment is allocated by caregiver. In some retrospective cohort studies (e.g., large cohorts where asking consent is not practicable, provided assurance that the patient’s privacy will not be compromised) IC for trial participation is not always necessary [43].

Statistics

An advantage of an RCT is that randomization reduces bias and confounding, both known and unknown [44]. Common pitfalls in statistical analyses of RCTs are managing missing data, applying the wrong statistical test, misreporting of results of the statistical analysis, and ad hoc reporting of subgroup analysis [45]. Noncompliance issues with the new treatment are typically addressed by an intention-to-treat approach. In pragmatic trials, additional (bias) issues may arise due to the cluster structure of randomization and researchers have proposed to use a per-protocol analysis to be able to estimate the effect of treatment instead of the treatment assignment [46].

The possibility of a high number of participants rejecting the intervention when they were randomly sampled for the new treatment raises a potential bias issue with the cmRCT design. In the double Zelen design, crossing over in the double consent design reduces noncompliance with the participants allocated to standard of care, but eliminates the effect of randomization, and will lead to loss to follow-up if the experimental treatment is preferable. This biases the true treatment effect compared with the single consent design. To solve these potential bias issues of the cmRCT and double Zelen design, an intention-to-treat analysis is recommended. However, research regarding trials that used the double consent design showed that 28% of the trials did not use an intention-to-treat analysis [34]. When looking at cohort designs, the analysis of data from cohort studies is often complex due to the large numbers of possible confounding variables [35], but cohort designs may offer a wider variety of collected variables and may be more representative to the population of interest.

Sample size

Calculation of an exact sample size for medical research is imperative [47,48] in particular for clinical trials. The sample size is typically calculated to achieve a minimal effect size that is considered clinically relevant. Not having the correct sample size is considered unethical [48]. Not including enough participants may risk a failure of the medical study to demonstrate the research goals. For instance, a true clinical benefit of the new treatment may not be detected due to lack-of-power of a too small sample size, while in prospective cohort studies specific associations may not be detected either. On the other hand, including too many participants may cause too much unnecessary burden on participants. This implies that too many participants may have been treated with a less beneficial treatment (depending on which treatment is clinically best), whereas a smaller trial could already have shown which treatment would be best. Also in cohorts, participant burden could be high due to the large number of variables that are being collected. Nevertheless, it should be noted that literature shows that studies are typically underpowered, because initial information is often too limited or too promising [49]. A disadvantage of clinical trials is that there are often difficulties to achieve the planned sample size due to lower than anticipated accrual rates [50]. In some cases, the clinical trial needs to be terminated before the final sample size can be achieved. In prospective cohort studies, participation rates are also declining [51,52]. However, within the cmRCT design, sample sizes may be attained more quickly due to the continuous inclusion of participants in the cohort.

Logistics

Due to RCT being the gold standard in current clinical research practices, clinicians are familiar and conversant with the logistics of the study design. As a result, the study tasks are easier adapted into clinical practice and are carried out quicker. On the other hand, the majority of RCTs initially have a large sample size to achieve the intended treatment effect. This is accompanied by a long recruitment period and high costs [5,39]. It has been shown that only 55% of the studies reach their recruitment target [53]. In addition, 45% of trials had to request for extension to reach recruitment targets [53]. Due to the long recruitment period and follow-up time of a RCT, it takes years to implement a new treatment. As a result, RCTs are less able to quickly follow clinical innovations [5]. During RCTs, treatment guidelines are sometimes modified and it is likely for the new intervention to be overtaken by a competing intervention or by a technical improved version of itself [6]. Due to time and/or monetary constraints, RCTs may also rely on surrogate markers because of feasibility, and those do not always correlate well with the outcome of interest [5].

Within the cmRCT design, numbers required for RCTs are attained more quickly because of the continuous inclusion of participants in the cohort. Also, the expenses of organizing and carrying out RCTs can be greatly decreased by centralizing the data process. Furthermore, it allows for multiple interventions to be tested simultaneously or sequentially within the same cohort [10]. The challenges of this design are the significant organizational efforts required to manage a large ongoing cohort and simultaneously coordinating multiple RCTs. Additionally, researchers, clinicians and participants in the intervention group(s) may find it difficult to comply with the multiple consent procedures.

The logistics of the Zelen design could also be doubtful. The goal of the Zelen design was to improve clinicians’ and patients’ trial participation and to minimize bias [8]. A review of trials using the Zelen design by Adamson et al. [34] described two studies with different effects on recruitment rates. Due to the poor study qualities, these results were considered questionable [34].

Data collection in a prospective cohort could also be challenging, due to the time-consuming, expensive and complex nature caused by confounders, selection-bias and loss-to-follow-up [35,54]. A retrospective cohort can be used as study design to shorten the time to complete the study and is therefore cheaper in comparison to a prospective cohort. However, additional problems such as recall bias and missing data may be associated with the use of a retrospective cohort [35,54].

The CIRSS with historical controls

We suggest a novel strategy that combines the benefits of the (cm)RCT and cohort study design with several objectives. First and foremost, the aim is to accurately reflect the real patient population and to evaluate treatment effects within this population as part of routine practice. Additionally, and equally crucial, the objective is to implement new treatments or interventions as patient-centered and effective as possible. See Figure 2 for an overview of the CIRSS, including a large prospective cohort alongside historical controls (retrospective cohort).

Recruitment

Prospective recruitment

After the patient’s eligibility is established (i.e., any patient who would in potential be able to receive the treatment), the patient is added to the prospective cohort (C). All patients will be part of this cohort without their consent, in which treatment A (standard care) will be administered. A random sample of patients (sC) is collected from the prospective cohort (C) after which the group of randomly sampled patients will be asked to give their consent for their data to be used for research purposes (i.e., if they wish to participate in the prospective cohort). If the patient declines participation in the cohort, the patient will be excluded from the study. These patients will receive treatment A (standard care) and their data is not used. Those who agree to participate in the prospective cohort (i.e., agree to sharing data) are subsequently asked if they are willing to receive a new treatment (treatment B). All treatment options, potential benefits and risks, are explained. If the patient consents to participate in the cohort and to the new treatment (B), the new treatment (B) will be administered; if the patient consents to participate in the cohort but declines the new treatment (B), the patient will receive the standard treatment (A), and his/her data will be used in the analysis.

The group of eligible patients who are not sampled is asked to participate in the prospective cohort, and will all receive the standard treatment (A). No information concerning the new treatment (B) is provided. Those who provide consent to join the cohort, form part of the control group and will be used in the analysis. Those who decline to participate in the cohort will receive standard treatment (A) and will not submit any data.

Retrospective recruitment

In order to finish the CIRSS as soon as possible, historical controls receiving standard care (treatment A) from a retrospective cohort (rC) will be supplemented to the prospective cohort.

Results of both groups – the control group (C and rC) and the intervention group (sC) – from the prospective and retrospective cohort are then used to determine the effectiveness of the new treatment. See the Figures 1 & 2 for an overview of these groups.

The recruiting process in the CIRSS resembles the process in the cmRCT design, but there are a two exceptions. First, in CIRSS there is no consent requested for sampling, while cmRCT requests approval for sampling. This may lead to a larger control group in CIRSS than in cmRCT when participants would not like to participate in a future trial. Together with the historical controls, more patients can be immediately allocated to the new intervention (a sampling proportion beyond 50%). Having a part of the control group (the prospective control group) parallel to the intervention group, makes it possible to investigate large temporal changes. Second, the cmRCT is intended for multiple RCTs, possibly for different treatment and diseases, while CIRSS creates a cohort of patients eligible for the single intervention being studied. This implies that the control group will be better defined in CIRSS than in cmRCTs, potentially generating a less biased treatment effect.

Random sampling & patient-centered IC

The objective is to include all eligible patients in a prospective cohort that is used for both clinical evidence of standard care and intervention study at the same time. From the eligible patients, a random sampling procedure is implemented that samples patients for the new treatment. By employing this random sampling technique, prior consent is not required, unlike the standard randomization process used in RCTs. As ‘random allocation of all’ and something that is ‘done’ to all patients in RCTs, randomization is typically understood to require prior consent of the patient. The decision to receive treatment A or B in the study lies with the randomization procedure. In the CIRSS, participants in the prospective cohort (i.e., agree to sharing data) are asked if they are willing to receive treatment B when they fall in the sample. If they decline treatment B, they will receive treatment A and remain part of the study. Obviously, patients that have not been sampled, do not need IC for not being sampled. Additionally, after the random sampling procedure, IC of all sampled patients is asked. They have the full right to decline the new treatment and receive standard care within or outside of the cohort. Subsequently, the nonsampled group receives information about standard care as is done in normal practice. An overview of the key features of our study design is provided in Table 2.

Internal & external validity

We anticipate that more patients will consent to participate in the CIRSS (due to a better recruitment strategy) in comparison to – for example – RCTs [10]. There is less exclusion criteria and all cohort participants will play a role in the analysis of the study, resembling the cmRCT design. Therefore, this would most likely lead to a better representation of the general population (external validation) [10,55]. Additionally, standard care in the nonsampled group is not affected by the possibility of the use of the new treatment option and will therefore better resemble routine standard care [10]. The sampling approach aligns very much with the randomization and therefore mimics a clinical trial (internal validity), although the sampled patients are aware of the experimental setting but the control group is not.

Ethics

There are some major ethical advantages to this study design. First, by including the retrospective cohort as part of the control group, the intervention can be made available to more patients in the prospective cohort. This will lead to less burden to the patients (i.e., a smaller number prospective participants needed to reach the sample size) and a possible quicker implementation of new interventions or treatments. Second, the IC procedure is patient-centered, also contributing to as little stress as possible on the patient and caregiver. Additionally, randomly sampled patients are free to decline the intervention and have the opportunity to still take part in the study while receiving standard treatment. The sampling approach being fully random also gives every patient the same chance of being part of the intervention.

However, this CIRSS with historical controls poses a few ethical quandaries. The notion that the control group is unaware of the intervention (group) may be the leading issue, as discussed for Zelen’s single consent design, but 50 years has passed after his introduction of postrandomization and new insights, discoveries and advancements in the ethical domain may be needed. One of the compelling arguments for the uninformed control group is that this group offers a more accurate representation of reality (current clinical care). In addition, since not everyone will be sampled for the intervention, it can be more ethical to spare them from unneeded information and avoid putting them through possibly emotional distress from hearing they will not receive the intervention [28,29]. Additionally, this could avoid any potential strain on the caregiver–patient relationship, as reported in RCTs [8,29]. However, the discovery of the existence of an intervention (group) within the CIRSS by the patients in the prospective control group could lead to another ethical dilemma. The healthcare professional may find themselves in a difficult position as a result. Prior to the start of the study, this must be properly considered and its risk must be estimated.

Statistics

In the CIRSS, participants who receive the experimental treatment are typically randomly sampled from the cohort and this splits the cohort in two representative sub cohorts (of possibly different size). Like in RCTs, this also addresses the issue of confounding, in particular when all sampled participants accept to undergo the experimental treatment. Selection bias in the CIRSS may occur when the sampled participants choose the control treatment, but this seems of a similar bias problem as the selection bias that comes from noncompliance [22]. Noncompliant participants are often a nonrandom group of participants. Additionally, the selection bias due to the refusals of the experimental treatment is compensated by an unbiased estimate of the clinical effect for control treatment obtained in the nonsampled participants. These nonsampled participants (and the historical controls) use the control treatment without knowing about the existence of other experimental treatments and are therefore unaffected by their knowledge of being present in a clinical trial. This provides an opportunity to investigate the potential of selection bias by comparing the clinical outcome of the refusals with clinical outcome of the nonsampled patients, but it requires some caution since historical controls may also introduce a temporal bias due to time-related factors (e.g., different patient population, seasonal effects, other innovations that affect the clinical outcome) compared with the nonsampled participants.

The possible biases obtained in CIRSS are not unsolvable, since they can be corrected by making use of causal inference [56,57] and hybrid sampling [58] approaches specifically developed for cohort studies. Indeed, the analysis of the CIRSS with historical controls will be analyzed as a single cohort study, with one treatment group and three different control groups (a proper control, a historical control, and a group of refusals). Potential imbalances and confounders may be corrected with inverse probability weighing approaches, assuming that the possible selection and temporal biases can be corrected with the collected participant information. Thus to correct for biases, a per-protocol analysis is proposed, following the advice for pragmatic trials from causal inference researchers [46], instead of an intention-to-treat analysis. In this respect, the CIRSS also deviates from the cmRCT, where an intention-to-treat analysis is suggested [59,60], although researchers are criticizing for this approach for cmRCTs as well [58,61]. More specifically, two weighting approaches are used to correct for possible biases that may occur due to the group of patients who are offered the new treatment B but receive current treatment A (either by choice or specific circumstances), referred to as the group of refusals in CIRSS and the retrospective and prospective nature of the study, referred to as a possible temporal bias or cohort effect. Weights (based on propensity score analyses using the observed information on participants unrelated to clinical outcomes) are created to correct for imbalances in patient characteristics between the four treatment groups. First a propensity score is determined between the retrospective and prospective patients to eliminate the cohort effect from temporal differences. This will lead to an inverse probability weight for each patient in the full study. Then secondly, the calculated weights for the group of refusals are corrected by the observed patient information from the nonsampled patients in the CIRSS (i.e., the proper control group) using inverse weighting probability that is common in the field of hybrid sampling [62]. This makes the group of refusals representative in characteristics to the nonsampled control group (creating one representative control group). After establishing the weights for all patient in the full cohort study (possibly rescaled to make the weights add up to the size of the full cohort study), the clinical outcomes can be analyzed by standard weighted (logistic) regression approaches where the intervention is compared with the combined control group. Similar causal inference analyses approaches (instrumental variables) have been described for cmRCT designs, where the same type of selection may occur by the design [22,59].

Sample size

Clinical trials (both superiority and noninferiority trials) require a proper sample size calculation for a well-defined effect size, since the goal is to demonstrate a clinically relevant benefit of the new treatment. Similarly, CIRSS requires a sample size calculation when the study must empirically establish a relevant benefit for e.g., a new treatment compared with an existing treatment. The common approach is that the sample size calculation follows the proposed statistical analysis for the primary outcome. However, since the statistical analysis for CIRSS is rather complex due to the proposed data analytic approaches to accommodate potential biases. Thus, an exact sample size calculation for CIRSS, when a well-defined clinical effect is provided, is not so straightforward and requires additional research. Therefore, sample size calculation should at least be conducted under certain ideal situations, where it is assumed that selection biases are not present. The sample size calculation would then follow existing approaches for sample size calculations. Note that literature shows that covariate adjusted analyses (like the causal inference approaches proposed) in RTC’s may also lead to an increase in power [63].

Logistics

Well-conducted observational cohort and case–control studies can provide the same level of internal validity as RCTs [13–15]. Furthermore, by adding a retrospective cohort to reinforce the prospective cohort, it reduces the number of prospective patients who need to receive standard care. This relatively increases the number of patients who could benefit from the new treatment and fasten the enrolment of the required number of patients. Additionally, it could lead to a faster implementation of new treatments, preventing the new intervention to be overtaken by a competing intervention of by a technically improved version of itself. Moreover, there could be financial benefits from higher effectiveness, which are especially interesting for specific patient populations and therapies, such as expensive interventions. An applied example of this study design, including operational aspects, including infrastructure, data integration and clinical implementation, are detailed in the NIEM-II study protocol [64]. Prior to the start of a study with CIRSS design, it is recommended to inform and train care givers about the IC procedure (approaching every potential participant, separate counseling for intervention group and control group and ensure that the control group remains unaware of the intervention).

Conclusion & future perspective

The CIRSS (with historical controls) aims to unite scientific research and routine clinical practice, and optimize implementation of promising new treatments as fluid, patient-centered and rapidly as possible. CIRSS thus combine the strengths of current study designs, e.g., (pragmatic) RCTs, cmRCTs, and Zelen’s designs, aiming to overcome some of the challenges presented using these designs. The CIRSS has several distinctive a large observational cohort of target patients is recruited and better resembles the true population; randomly sampling participants takes place (which avoids randomization conducted in standard clinical trials); and ‘patient-centered’ information and consent is applied.

No outcome of studies using the CIRSS design are published yet. Two large clinical trials in obstetrics in the Netherlands (NIEM-II study (NCT06135961) and the NIEM-O study (NCT06151613)) using this new study design, are ongoing. Both studies received approval from the Medical Ethics Committee of Máxima MC, Veldhoven, the Netherlands with respectively number W22.071 and W22.070. The CIRSS design aligns with international research ethics guidelines (e.g., Declaration of Helsinki, principle 11, 25 and 26). As an illustration, the clinical implementation of the CIRSS design in the NIEM-II study is presented in Box 1. Further methodological details, including some statistical considerations and design rationale, are described in the corresponding study protocol [64]. CIRSS is an experimental study design in which new treatments can be evaluated against existing or current treatments. It is similar in spirit as cohort multiple randomized control trials where it shares many features of the cmRCT, but it has also some peculiarities. For instance, it allows the use of historic controls easier than cmRCT’s. In combination with proper statistical analysis from causal inference theory and hybrid sampling it is expected that an unbiased treatment effect can be obtained. The execution of CIRSS requires proper planning and organization, but in combination with historic controls, implementation of new treatments can be fast or rejected timely. We believe the CIRSS design is broadly applicable, although it might pose challenges in rare disorders, due to the necessity of large sample sizes or long accrual times. Further research is required to address the range of analyses, implementation, issues, barriers and facilitators and ethical questions related to this cohort intervention study design.