Sequential Monitoring of Randomization Tests

CONTENTS 9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262

9.1.1 Sequential Monitoring under Population Model . . . . . . . . . . . . . 262 9.1.2 Randomization Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265 9.1.3 Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267

9.2 Score Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267 9.3 Asymptotic Results for the Unconditional Randomization Test . . . . 269

9.3.1 Asymptotic Results in the Nonsequential Case . . . . . . . . . . . . . . . 269 9.3.2 Asymptotic Results for K Inspections . . . . . . . . . . . . . . . . . . . . . . . . . . 270

9.4 Information Fraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 9.4.1 Complete Randomization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273 9.4.2 General Urn Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274

9.5 K-Inspection Unconditional Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 9.6 Discussion of Calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276

9.6.1 R Calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278 9.6.2 SAS Calculations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 278

9.7 Unconditional Test for Three Common Designs . . . . . . . . . . . . . . . . . . . . . . . 280 9.7.1 Properties of Randomization Tests in the Nonsequential

Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 281 9.7.2 Development of a Monitoring Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

9.7.2.1 Permuted Block Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 9.7.2.2 Stratified Block Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283 9.7.2.3 Stratified General Urn Design . . . . . . . . . . . . . . . . . . . . . . . . 284

9.8 Conditional Tests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287 9.8.1 Sequential Monitoring Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287 9.8.2 Asymptotic Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 9.8.3 Information Fraction for Conditional Tests . . . . . . . . . . . . . . . . . . . . 292

9.9 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293 Appendix A . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294 Appendix B . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

C5777: “c5777_c009” — 2007/10/27 — 13:03 — page 262 — #2

in

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 294

Sequential monitoring has become a hallmark of the well-conducted randomized clinical trial. The statistician computes interim values of the test statistic for the primary outcome, and a decision is made to continue the trial or stop, based on these interim assessments of efficacy. There is a large literature on group sequential monitoring of population-based inference procedures; see Jennison and Turnbull (2000) for a comprehensive overview of the subject. There is no corresponding literature on sequential monitoring using randomization-based inference, which is rooted historically and arises naturally from the randomized nature of the clinical trial. Here we attempt to rectify that, by presentingwhatwe know (so far) about sequentialmonitoring of randomization tests. This work represents a sketch of the doctoral thesis of Zhang (2004), and focuses not on the theory, but on how toperform sequential monitoring of randomization tests in practice. A sequential monitoring plan necessarily involves computing the joint asymptotic distribution of sequentially computed statistics. The mathematical formulation of the procedure therefore is complicated and involves multidimensional integration and test statistics that are messy to compute. Although we try to minimize the mathematical content of this paper, it is necessary to present the formulas for what needs to be computed and the conditions under which such formulas are appropriate. It is our contention (and perhaps a controversial one) that although ran-

domization has become an entrenched part of the biostatistician’s culture, it is unfortunately often treated in a lackadaisical manner. In particular, great arguments over the appropriateness of incorporating randomization in the analysis have been subsumed by standard analyses using SAS. Here, we take the approach that randomization tests provide a nonparametric, assumption-free test of the treatment effect in a clinical trial, and indeed arise naturally from the structure of the clinical trial. Thus, randomization-based analyses should be conducted as a matter of course, either as a complement to population-based analyses or as a stand-alone primary analysis.