PubMed ID: 21660119
Author(s): Oden N, Vanveldhuisen PC, Scott IU, Ip MS; the SCORE Study Investigator Group. SCORE study report 8: closed tests for all pair-wise comparisons of means. Drug Inf J. 2010 Jul 1;44(4):405-420. PMID 21660119
Journal: Drug Information Journal, Volume 44, Issue 4, Jul 2010
We compare five closed tests for strong control of family-wide type I error (FWE) while making all pair-wise comparisons of means in clinical trials with multiple arms such as the SCORE Study. We simulated outcomes of the SCORE Study under its design hypotheses, and used p-values from chi-squared tests to compare performance of a “pairwise” closed test described below to Bonferroni and Hochberg adjusted p-values. “Pairwise” closed testing was more powerful than Hochberg’s method by several definitions of multiple-test power. Simulations over a wider parameter space, and considering other closed methods, confirmed this superiority for p-values based on normal, logistic, and Poisson distributions. The power benefit of “pair-wise” closed testing begins to disappear with 5 or more arms, and with unbalanced designs. For trials with 4 or fewer arms and balanced designs, investigators should consider using “pair-wise” closed testing in preference to Shaffer’s, Hommel’s, and Hochberg’s approaches when making all pairwise comparisons of means. If not all p-values from the closed family are available, Shaffer’s method is a good choice.