Exploring the Fragility of Meta-Analyses in Ophthalmology: A Systematic Review

Theme: Paediatric ophthalmology and strabismus
What: Paediatric ophthalmology and strabismus
Part of: Paediatrics III: Retinopathy of Prematurity - Pathophysiology and Best Evidence for Prevention and Treatment / Pédiatrie III: Rétinopathie du prématuré - Pathophysiologie et meilleures données probantes pour la prévention et le traitement
When: 6/2/2024, 02:00 PM - 03:30 PM
Where: Room | Salle 714 A

Abstract

NOTE: This abstract will be presented via video recording. 

 

Purpose: The quality of a meta-analysis can vary widely; not all meta-analyses can provide the same level of evidence. The fragility index (FI) of a meta-analysis evaluates the extent to which the statistical significance of a meta-analysis can be changed by modifying the event status of individuals from the included trials. An event status modification is defined as changing non-events to events or events to non-events for the outcome examined in the meta-analysis. Understanding the fragility of meta-analyses aids with the interpretation of the results and can help to inform changes to clinical practice. The purpose of this review was to determine the fragility of published meta-analyses in ophthalmology.

Study Design: Systematic review and meta-analysis.

Methods: This systematic review was registered in PROSPERO (CRD42022377589). Meta-analyses of randomized controlled trials with a binary outcome published prior to September 1^st, 2022, in a journal classified as ‘Ophthalmology’ according to the 2022 Journal Citation Report or an Ophthalmology-related review published in the Cochrane Database of Systematic Reviews were included. An iterative process was utilized to determine the minimal number of event-status modifications from individual trials that would change the statistical significance of the pooled treatment effect of the meta-analysis. A linear regression model evaluated the associations between the FI and the following characteristics: a) whether the outcome was the primary outcome, b) whether the study had a significant result, c) the sample size, d) the number of events, e) the impact factor of the journal that the study was published in, f) the subspecialty most relevant to the published report and g) whether the included outcome was related to safety or efficacy. 

Results: 175 meta-analyses were included. The median FI was 6 (Q1-Q3: 3-12). This meant that modifying the event status of 6 individuals from one of the included studies would reverse the findings of the meta-analysis. The FI was 1 for 18 (10.2%) of the included meta-analyses and was < 5 for 75 (42.4%) of the included meta-analyses. Only the number of outcome events was associated with the FI (p=0.037). 

Conclusions: The statistical significance of meta-analyses in ophthalmology often hinges on the outcome of a few patients. The number of outcome events in a review is the single most important factor in determining the fragility of the evidence. Clinicians are encouraged to interpret the results of meta-analyses in ways beyond the p-value. The FI can supplement the reader’s understanding of the strength of the evidence being presented. 

 

 

Presenter(s)

Presenting Author: Keean Nanji

Additional Author(s):

Jim Xie, McMaster University, Department of Surgery, Division of Ophthalmology, 2757 King Street East, Hamilton, Ontario, Canada

Amin Hatamnejad, McMaster University, Department of Surgery, Division of Ophthalmology, 2757 King Street East, Hamilton, Ontario, Canada

Daiana Pur, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada

Mark Phillips, McMaster University, Department of Health Research Methods, Evidence and Impact, 1280 Main Street West, Hamilton, Ontario, Canada

Dena Zeraatkar, McMaster University, Department of Health Research Methods, Evidence and Impact, 1280 Main Street West, Hamilton, Ontario, Canada

Sobha Sivaprasad, NIHR Moorfields Biomedical Research Centre, Moorfields Eye Hospital, London, UK

Mohit Bhandari, McMaster University, Department of Health Research Methods, Evidence and Impact, 1280 Main Street West, Hamilton, Ontario, Canada

David Steel, Bioscience Institute, Newcastle University, Newcastle Upon Tyne, UK

Charles Wykoff, Retina Consultants of Texas, Houston, Texas, Blanton Eye Institute, Houston Methodist Hospital, Houston, Texas, USA

Varun Chaudhary, McMaster University, Department of Health Research Methods, Evidence and Impact, 1280 Main Street West, Hamilton, Ontario, Canada

---