Multiple imputation for validating prediction models
Summary
Measuring the calibration is an essential part of the validation of clinical prediction models. However, missing data is a common issue which can complicate the measurement of calibration statistics like the observed to expected (O/E) ratio. By performing a simulation study, this work investigates whether transformations of the O/E ratio improve the performance of the statistic in settings of missing data handled by multiple imputation (MI). MI replaces missing values, thereby creating multiple different data sets. The O/E statistics are calculated for each imputed data set. Subsequently, the O/E statistics are pooled according to Rubin’s rules. As Rubin’s rules are based on asymptotic theory, transformations of the O/E estimates prior to pooling may improve the performance of the statistics. The simulation study considers the untransformed, natural log, and square root transformation methods to compare their performance to the true O/E, in terms of bias and coverage probability. The O/E estimates of all methods show a small positive bias and a coverage probability below the nominal. Moreover, the performance of the O/E estimates is better in the simulation condition of O/E < 1 compared to O/E > 1. With respect to the different transformation methods, the results show no significant difference in the performance. This indicates that none of the investigated transformation methods is appropriate to improve the performance.