gms | German Medical Science

Introduction: In previous studies the standard beta-binomial (BB) model has shown good statistical properties for meta-analyses of binary outcomes, especially in challenging situations with a low number of studies or events [1], [2]. The standard BB-model (CR) assumes a common correlation (rho) of observations within study groups across treatment. Recently, we have introduced a common-beta model (CB) and a common-beta model with an additional multiplicative overdispersion term (CBO).

Our objective was to compare these models to standard methods for estimating the summary odds-ratio.

Methods: We compared the BB models to the following standard methods for meta-analysis in a simulation study:

Generalized linear mixed model with a fixed intercept and random treatment effects (GLFR)

Random-effects model using a refined version of Hartung-Knapp-Sidik-Jonkman (RHKSJ) 95% confidence intervals (CIs) that is the modification is only applied if the 95%CIs of the original HKSJ-method are smaller than Wald-type CIs. Heterogeneity variance estimated with the Paule-Mandel method.

The parameters of the simulation were informed by actually performed meta-analyses [3]. We considered scenarios with median study numbers of 8 and 3, respectively, which mirrors non-Cochrane and Cochrane meta-analyses [3], [4]. Results are only reported separately for the scenarios if the results differ. To compare the methods we considered median bias and mean coverage to the 95%CI.

In addition, we performed a meta-regression on the simulated data-set to analyse the impact of the number of studies in meta-analyses, event probability in the control group, and heterogeneity on coverage of the 95%CIs.

Results: None of the methods had problems with convergence. Median bias was small for all methods. Only GLFR fell relevantly below the nominal coverage probability. Meta-regression suggested that the risk of unsatisfying coverage for GLFR heavily increases with increasing heterogeneity. In the Cochrane scenario, the risk of not satisfying coverage increased with increasing event probability for the CR and increased with decreasing event probability for the RHKSJ. Coverage of CBO was least affected by different data situations.

Discussion: This study confirms previous findings that beta-binomial models work well for meta-analyses in the case of (very) low event probability and (very) few studies in the meta-analysis. Our simulation suggests that the performance of CBO is least sensitive for changes in event probability and heterogeneity. A practical advantage of the new BB models is that they are easy to implement because they can be estimated with standard software packages for count panel data from econometrics. RHKSJ also performed satisfactorily but the probability of coverage below the nominal level increased in very challenging data situations. One reason for this finding is probably, that in contrast to the other models, this model requires a continuity correction, when there are studies with zero events.

The finding that coverage probability of GLFR decreases with increasing heterogeneity of the treatment effect was also observed in previous simulations and raises some concerns for applying this model in practice [5].

Conclusion: BB models should be considered for meta-analyses of few studies and in the case of low event probabilities.

The authors declare that they have no competing interests.

The authors declare that an ethics committee vote is not required.