Philosophy

On this page, you can find examples of previous natural experiment studies conducted in the field of emergency and (orthopedic) trauma surgery. Also, various meta-analyses from this field of research are presented, where results from randomised trials are complemented by results of natural experiments. Recent articles by members of the NEXT study group about key methodological of observational studies are listed too.

The publication policy of the NEXT study group can be found here


Examples of natural experiments

A common threat to the validity of observational studies of medical treatment is the impossibility to correct for inherent incomparability between the groups of patients who receive different treatments. This is referred to as confounding.

Natural experiments are observational studies where natural variation in treatment allocation is utilized to increase the validity of the results of observational studies of treatment effects. There are various example of natural experiment studies listed below. What these examples have in common is that the differences in the treatments that patients receive do not depend so much on differences in patients characteristics, but much more on for example the hospital they visit or the preferences of the doctor who treats them. Observational comparisons grounded in natural experiments are likely to be less prone to bias due to confounding.

Examples:

  • Luke Leenen et al. performed this study to determine the effect of intracranial pressure (ICP) and targeted management of cerebral perfusion pressure (CPP) intensive care on functional outcome and therapy intensity levels after severe head injury. In center A (supportive intensive care), mean arterial pressure was maintained at approximately 90 mm Hg, and therapeutic interventions were based on clinical observations and computed tomography findings. In center B (ICP/CPP-targeted intensive care), management was aimed at maintaining ICP <20 mm Hg and CPP >70 mm Hg. Allocation to either trauma center was solely based on the site of the accident. Read more
  • Cumhur Oner et al. described an observational study design with clinical equipoise as an inclusion criterion. The proposed methodology is a cohort study with head-to-head comparison of operative and nonoperative treatment regimens for thoracolumbar spine fractures in an expertise-based trial fashion. Patients were selected retrospectively by an expert panel and clinical outcomes were assessed to compare competing treatment regimens. Surgeon equipoise served as an inclusion criterion. Read more
  • Marijn Houwert et al. performed this study to compare complications after intramedullary fixation and plate fixation for dislocated midshaft clavicle fractures. Surgeon A (EJV) always performed plate fixation whereas surgeon B (KO) always performed intramedullary fixation. Allocation to either surgeon was a random process solely based on the surgeon performing the procedure. Read more
  • Frank Beeres et al. evaluated the number of complications following locking plate fixation of proximal humeral fractures in country X and in country Y. Differences in both countries exist regarding timing of the procedure and number of procedures per surgeon. The secondary aim was to identify risk factors for complications. Read more
  • Marijn Houwert, Luke Leenen, Rolf Groenwold et al. performed a study to compare results of surgical ribfixation for patients with multiple ribfractures or a flail chest. In Hospital A, ribfixation was performed as a standard treatment option whereas in hospital B patients were treated non-operatively. Read more
  • Frank Beeres and Marijn Houwert et al. performed this study to compare outcomes of geriatric care pathways (GCPs. A comparison was made between two traumageriatric care models, one Swiss (CH) and one Dutch (NL), to assess whether these models would perform similarly despite the possible differences in local clinical practices. Read more

Meta-analyses

Members of the NEXT study group advocate including observational studies in systematic reviews and meta-analyses of the effects of medical treatments. The strength of including observational studies in systematic reviews and meta-analyses is that all available evidence on a given topic is presented, including the real-world variety of study populations and long-term outcome effects. In such reviews, data of RCTs and observational studies can be presented separately which allows readers to draw their own conclusions. On top of that, for each study, the quality can be appraised using available methodology (such as the MINORS criteria, the ROBINS-I tool, or the Cochrane Risk of Bias assessment tool), to provide readers with key information to appraise the validity of the included studies (RCTs and observational).

Our research group published several systematic reviews and meta-analyses including observational data for (orthopaedic) trauma. These studies show the added value of including observational evidence: including more studies leads to larger sample sizes, which allows for more detailed investigation of subgroup effects. What is more, observational studies appear to be more representative of daily practice, e.g., in terms of patient and surgeon characteristics. Provided that the included observational studies are of sufficient quality, their results complement those of RCTs.

Examples:

  • Bleeker NJ, van de Wall BJM, IJpma FFA, Doornberg JN, Kerkhoffs GMMJ, Jaarsma RL, Knobe M, Link BC, Babst R, Beeres FJP. Plate vs. nail for extra-articular distal tibia fractures: How should we personalize surgical treatment? A meta-analysis of 1332 patients. Injury. 2021 Mar;52(3):345-357. https://doi.org/10.1016/j.injury.2020.10.026
  • Ochen Y, Beks RB, van Heijl M, Hietbrink F, Leenen LPH, van der Velde D, Heng M, van der Meijden O, Groenwold RHH, Houwert RM. Operative treatment versus nonoperative treatment of Achilles tendon ruptures: systematic review and meta-analysis. BMJ. 2019 Jan 7;364:k5120. https://doi.org/10.1136/bmj.k5120
  • Beks RB, Ochen Y, Frima H, Smeeing DPJ, van der Meijden O, Timmers TK, van der Velde D, van Heijl M, Leenen LPH, Groenwold RHH, Houwert RM. Operative versus nonoperative treatment of proximal humeral fractures: a systematic review, meta-analysis, and comparison of observational studies and randomized controlled trials. J Shoulder Elbow Surg. 2018 Aug;27(8):1526-1534. https://doi.org/10.1016/j.jse.2018.03.009
  • Beks RB, Peek J, de Jong MB, Wessem KJP, Öner CF, Hietbrink F, Leenen LPH, Groenwold RHH, Houwert RM. Fixation of flail chest or multiple rib fractures: current evidence and how to proceed. A systematic review and meta-analysis. Eur J Trauma Emerg Surg. 2019 Aug;45(4):631-644. https://doi.org/10.1007/s00068-018-1020-x
  • Ochen Y, Peek J, van der Velde D, Beeres FJP, van Heijl M, Groenwold RHH, Houwert RM, Heng M. Operative vs Nonoperative Treatment of Distal Radius Fractures in Adults: A Systematic Review and Meta-analysis. JAMA Netw Open. 2020 Apr 1;3(4):e203497. https://doi.org/10.1001/jamanetworkopen.2020.3497
  • van de Wall BJM, Ochen Y, Beeres FJP, Babst R, Link BC, Heng M, van der Velde D, Knobe M, Groenwold RHH, Houwert MR. Conservative vs. operative treatment for humeral shaft fractures: a meta-analysis and systematic review of randomized clinical trials and observational studies. J Shoulder Elbow Surg. 2020 Jul;29(7):1493-1504. https://doi.org/10.1016/j.jse.2020.01.072
  • Beeres FJ, Diwersi N, Houwert MR, Link BC, Heng M, Knobe M, Groenwold RH, Frima H, Babst R, Jm van de Wall B. ORIF versus MIPO for humeral shaft fractures: a meta-analysis and systematic review of randomized clinical trials and observational studies. Injury. 2021 Apr;52(4):653-663. https://doi.org/10.1016/j.injury.2020.11.016
  • van de Wall BJM, Baumgärtner R, Houwert RM, Link BC, Heng M, Knobe M, Groenwold RHH, Babst R, Beeres FJP. MIPO versus nailing for humeral shaft fractures: a meta-analysis and systematic review of randomised clinical trials and observational studies. Eur J Trauma Emerg Surg. 2021 Jan 15. https://doi.org/10.1007/s00068-020-01585-w
  • Smeeing DPJ, van der Ven DJC, Hietbrink F, Timmers TK, van Heijl M, Kruyt MC, Groenwold RHH, van der Meijden OAJ, Houwert RM. Surgical Versus Nonsurgical Treatment for Midshaft Clavicle Fractures in Patients Aged 16 Years and Older: A Systematic Review, Meta-analysis, and Comparison of Randomized Controlled Trials and Observational Studies. Am J Sports Med. 2017 Jul;45(8):1937-1945. https://doi.org/10.1177/0363546516673615
  • Houwert RM, Smeeing DP, Ahmed Ali U, Hietbrink F, Kruyt MC, van der Meijden OA. Plate fixation or intramedullary fixation for midshaft clavicle fractures: a systematic review and meta-analysis of randomized controlled trials and observational studies. J Shoulder Elbow Surg. 2016 Jul;25(7):1195-203. https://doi.org/10.1016/j.jse.2016.01.018

Methodology

Below is a list of papers that describe some of the challenges of natural experiments and how to mitigate those challenges. Together with the literature cited in these papers, they could serve as a starting point to explore the vast amount of literature on this topic.

  • Using equipoise subgroups in observational research: Valid results, but to whom do they apply? Presentation at annual NEXT meeting 2023 Read more
  • Houwert RM, Beks RB, Dijkgraaf MGW, Roes KCB, Öner FC, Hietbrink F, Leenen LPH, Groenwold RHH. Study methodology in trauma care: towards question-based study designs. Eur J Trauma Emerg Surg. 2021 Apr;47(2):479-484. Read more
  • Beks RB, Bhashyam AR, Houwert RM, van der Velde D, van Heijl M, Smeeing DPJ, Hietbrink F, Leenen LPH, Groenwold RHH. When observational studies are as helpful as randomized trials: Examples from orthopedic trauma. J Trauma Acute Care Surg. 2019 Sep;87(3):730-732. Read more
  • Groenwold RHH. Trial Emulation and Real-World Evidence. JAMA Netw Open. 2021 Mar 1;4(3):e213845. Read more
  • Beks RB, Houwert RM, Groenwold RHH. Meerwaarde van observationeel onderzoek in chirurgie [Added value of observational studies in surgery: the hierarchical structure of study designs requires a more refined approach]. Ned Tijdschr Geneeskd. 2017;161:D1493. Read more
  • Groenwold RHH, Brenkman H, Houwert RM. Trials, observationeel onderzoek en de echte wereld [Trials, observational studies and the real world]. Ned Tijdschr Geneesk 2020:D5514. Read more