gms | German Medical Science

62. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e. V. (GMDS)

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie

17.09. - 21.09.2017, Oldenburg

Article

Send article

Comparison of the multinomial generalized linear mixed model with three alternative generalized mixed models in the meta-analysis of diagnostic accuracy trials with non-evaluable index test results

Meeting Abstract

Search Medline for

  • Jenny Pham
  • Peter Schlattmann - Universitätsklinikum Jena, Jena, Deutschland

Deutsche Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie. 62. Jahrestagung der Deutschen Gesellschaft für Medizinische Informatik, Biometrie und Epidemiologie e.V. (GMDS). Oldenburg, 17.-21.09.2017. Düsseldorf: German Medical Science GMS Publishing House; 2017. DocAbstr. 200

doi: 10.3205/17gmds046, urn:nbn:de:0183-17gmds0465

Published: August 29, 2017

© 2017 Pham et al.
This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 License. See license information at http://creativecommons.org/licenses/by/4.0/.

Outline

Text

For the meta-analysis of diagnostic test accuracy trials with a dichotomous understanding of the disease, one state-of-the-art model is the bivariate Generalized Linear Mixed Model (GLMM) which assumes the true positive and true negative counts to follow a binomial distribution drawn from the diseased and non-diseased population, respectively. Thus, the two outcomes sensitivity (=true positive rate) and specificity (=true negative rate) can be simultaneously modelled and the heterogeneity between studies as well as their correlation across studies be incorporated.

However, if the index test yields non-evaluable results, no standard procedure exists.

A novel idea is the modelling of unevaluable results as separate category, thereby extending the binomial distribution of the bivariate GLMM to the multinomial.

A simulation study was implemented to validate the performance of four competing GLMMs in terms of bias, mean squared error (MSE), coverage probability and convergence frequency: the multinomial, the excluding, the worst case and the extended trivariate GLMM. The excluding GLMM discarded the unevaluable results while the worst case model treated them as false results. The extended trivariate GLMM (TGLMM) modelled the unevaluable results as missing data with a missing at random structure and modelled the prevalence as third outcome which is adjusted for the non-evaluable results.

In the estimation of pooled sensitivity and specificity, the multinomial GLMM outperformed the remaining models in terms of bias, MSE and coverage probability. In the mean, the relative bias was about -0.5%, the MSE 0.002 and the coverage probability ~93%. All performance parameters improved considerably with a larger number of primary studies.

The worst case GLMM yielded estimates that came closest to the multinomial GLMM (mean relative bias ~ -2.0%, mean coverage probability ~ 90%) with an MSE nearly identical to the multinomial GLMM. Again, a larger number of primary studies improved the bias and the MSE. The coverage probability however decreased.

Estimates for sensitivity and specificity of the excluding GLMM and the extended TGLMM were markedly worse. The bias was theoretically derived and amounted to the odds of the non-evaluable results for the relative bias and to the product of the odds and the respective accuracy parameter for the absolute bias. Observed and theoretical bias were nearly identical. The MSE was also highly dependent on the true values. The coverage probability varied between 0% and 80%, with lower coverage probabilities in scenarios with larger number of studies. Regarding the convergence frequency, the worst case GLMM yielded the best results (mean: 99.9%± 0.4), followed by the excluding GLMM (mean: 99.0%± 2.2) and the multinomial GLMM (mean: 97.5% ± 3.5). The extended TGLMM failed to converge in > 50% of the cases.

The multinomial GLMM is an intuitive extension of the bivariate GLMM that models non-evaluable results as separate category. This approach satisfies the intention-to-diagnose principle and provides promising candidates for unbiased and efficient estimates of sensitivity and specificity, which consistently approach the true values with increasing number of primary studies.



Die Autoren geben an, dass kein Interessenkonflikt besteht.

Die Autoren geben an, dass kein Ethikvotum erforderlich ist.

Outline

References

1.
Begg C, Greenes R, Iglewicz, B. The influence of uninterpretability on the assessment of diagnostic tests. Journal of Clinical Epidemiology. 1986;59:575-584.
2.
Borenstein M, Hedges L, Higgins J, Rothstein H. Introduction to Meta-Analysis. s.l.: John Wiley; 2009.
3.
Borenstein M, Hedges L, Rothstein H. 2Meta-Analysis Fixed effect vs random effect. 2007. https://www.meta-analysis.com/downloads/Meta-analysis%20fixed%20effect%20vs%20random%20effects.pdf External link
4.
Burton A, Altman D, Royston P, Holder R. The design of simulation studies in medical statistics. Statistics in Medicine. 2006:4279-4292.
5.
Carter W, et al. Relative accuracy of two diagnostic schemes for detection of pulmonary embolism by ventilation-perfusion scintigraphy. Radiology. 1982:447-451.
6.
Chu H, Cole S. Bivariate meta-analysis of sensitivity and specificity with sparse data: A generalized linear mixed model approach. Journal of Clinical Epidemiology. 2006;59:1331.
7.
Chu H, Nie L, Cole S, Poole C. Meta-analysis of diagnostic accuracy studies accounting for disease. Statistics in Medicine. 2009:2384-2399.
8.
Collins LM, Schafer JL, Kam CM. A comparison of inclusive and restrictive strategies in modern missing data procedures. Psychol Methods. 2001;6(4):330-351.
9.
Glass G. Primary, secondary and meta-analysis of research. Education Research. 1976;5:3-8.
10.
Hamza TH, Reitsma JB, Stijnen T. Meta-analysis of diagnostic studies: a comparison of random intercept, normal-normal, and binomial-normal bivariate summary ROC approaches. Medical Decision Making. 2008;28:639-649.
11.
Jackson D, Riley R, White I. Multivariate meta-analysis: Potential and promise. Statistics in medicine. 2011;30:2481-2498.
12.
Kateri M. Contingency Table Analysis: Methods and Implementation Using R. New York: Springer; 2014.
13.
Kiemann K, Tao J, Gibbs P. SAS Global Forum. Cary NC: SAS Institute Inc.; 2012.
14.
Klein G. Bivariate meta-analysis of sensitivity and specificity: A comparison of the normal-normal and the binomial-normal model with a finite mixture model of bivariate normal distributions. s.l.:s.n.; 2015.
15.
Leeflang MMG, Deeks JJ, Gatsonis C, Bossuyt P.Systematic reviews of diagnostic test accuracy. Annals of Internal Medicine. 2008:889-897.
16.
Ma X, Fareed M, Chu H. A trivariate meta-analysis of diagnostic studies accounting for prevalence and non-evaluable subjects: re-evaluation of the meta-analysis of the coronary CT angiography studies. BMC Medical Research Methodology. 2014;14:128.
17.
Ma X, Nie L, Cole S, Chu H. Statistical methods for multivariate meta-analysis of diagnostic tests: An overview and tutorial. Statistical Methods in Medical Research. 2013:1–24.
18.
Menke J. Bivariate Random-effects Meta-analysis of Sensitivity and Specificity with SAS PROC GLIMMIX. Methods of Information in Medicine. 2010:54–64.
19.
Menke J. Bivariate Random-Effects Meta-analysis of Sensitivity and Specificity with the Bayesian SAS PROC MCMC: Methodology and Empirical Evaluation in 50 Meta-analyses. Med Decis Making. 2013:692−701.
20.
Menke J, Kowalski J. Diagnostic accuracy and utility of coronary CT angiography with consideration of unevaluable results: A systematic review and multivariate Bayesian random-effects meta-analysis with intention to diagnose. Eur Radiol. 2016;26:451-458.
21.
NIH. MedlinePlus [Online]. 2016. Available at: https://medlineplus.gov/coronaryarterydisease.html External link
22.
Peng D, MacKenzie G. Discrepancy and choice of reference subclass in categorical regression models. In: Statistical Modelling in Biostatistics and Bioinformatics: selected papers. Munich, Germany: Springer; 2014.
23.
Reitsma J, et al. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58:982-990.
24.
Roche [Online]. 2016. Available at: https://www.roche.de/diagnostics/krankheiten-erkennen/frauengesundheit/chlamydien/diagnostik-chlamydien.html External link
25.
SAS Institute Inc. SAS/STAT 9.3 User's Guide. Cary, NC: s.n.; 2011.
26.
Schuetz G, Schlattmann P, Dewey M. Use of 3x2 tables with an intention to diagnose approach to assess clinical performance of diagnostic tests: meta-analytical evaluation of coronary CT angiography studies. BMJ. 2012;345:6717-6727.
27.
Severini TA. Elements of Distribution Theory. Cambridge: Cambridge University Press; 2005.
28.
Simel D, Feussner J, Delong E, Matchar D. Intermediate, indeterminate and untinterpretable diagnostic test results. Med Decis Making. 1987:107-114.
29.
Stroup W. Generalized Linear Mixed Models. Boca Raton: CRC Press; 2013.
30.
Takwoingi Y, Guo B, Riley RDJ. Performance of methods for meta-analysis of diagnostic test accuracy with few studies or sparse data. Statistical Methods in Medical Research. 2015:1–19.
31.
Ulimoen G, et al. Noninvasive coronary angiography with 64-channel multidetector computed tomography in patients with acute coronary syndrome. Acta Radiol. 2008:1140-1144.
32.
Zapf A, Hoyer A, Kramer K, Kuss O. Nonparametric meta-analysis for diagnostic accuracy studies. Statistics in Medicine. 2015:3831-3841.
33.
Zhou XH, Obuchowski N, McClish D. Statistical Methods in Diagnostic Medicine. Hoboken, New Jersey: Wiley; 2011.