ABSTRACT
Lack of trial reproducibility has gained widespread attention, and even randomised controlled trials (RCT) are not immune. This paper examines the accuracy of the Berger–Exner test to detect third-order selection bias in the presence of true treatment effects. RCTs with various parameter settings were simulated and subjected to bias-free/biased scenarios. The test’s sensitivity (SS), specificity (SP), diagnostic odds ratio (DOR), negative/positive predictive values (NPV/PPV) and negative/positive likelihood ratios (LR−/LR+) for alpha levels of 1% and 10% were computed. The relationships between biasing of different intervention groups and treatment effect differences between groups with test accuracy were explored. The SS and NPV were 1.00; LR− was zero and DOR was infinite for all parameter sets and alphas levels. The SP ranged between 0.74 and 0.81 for alpha 10% and between 0.95 and 0.98 for alpha 1%. The PPV ranged between 0.79 and 0.84 for alpha 10% and between 0.95 and 0.98 for alpha 1%. The LR+ ranged from 3.86 to 5.32 for alpha 10% and from 21.28 to 40.00 for alpha 1%. Neither biasing of the different intervention groups nor treatment effect differences between the groups affected test accuracy. In the presence of true treatment effects, the Berger–Exner test may detect third-order selection bias at high accuracy.
