SCORE Study Report 8: Closed tests for all pairwise comparisons of means

Neal Oden, Paul C. VanVeldhuisen, Ingrid Scott, Michael S. Ip

Research output: Contribution to journalReview article

4 Citations (Scopus)

Abstract

We compare five closed tests for strong control of family-wide type I error while making all pairwise comparisons of means in clinical trials with multiple arms such as the SCORE Study. We simulated outcomes of the SCORE Study under its design hypotheses, and used P values from chi-squared tests to compare performance of a pairwise closed test described below to Bonferroni and Hochberg adjusted P values. Pairwise closed testing was more powerful than Hochberg's method by several definitions of multiple-test power. Simulations over a wider parameter space, and considering other closed methods, confirmed this superiority for P values based on normal, logistic, and Poisson distributions. The power benefit of pairwise closed testing begins to disappear with five or more arms and with unbalanced designs. For trials with four or fewer arms and balanced designs, investigators should consider using pairwise closed testing in preference to Shaffer's, Hommel's, and Hochberg's approaches when making all pairwise comparisons of means. If not all P values from the closed family are available, Shaffer's method is a good choice.

Original languageEnglish (US)
Pages (from-to)405-420
Number of pages16
JournalDrug Information Journal
Volume44
Issue number4
DOIs
StatePublished - Jan 1 2010

Fingerprint

Testing
Poisson Distribution
Poisson distribution
Normal Distribution
Logistics
Research Personnel
Outcome Assessment (Health Care)
Clinical Trials

All Science Journal Classification (ASJC) codes

  • Pharmacology (nursing)
  • Drug guides
  • Public Health, Environmental and Occupational Health
  • Pharmacology (medical)

Cite this

Oden, Neal ; VanVeldhuisen, Paul C. ; Scott, Ingrid ; Ip, Michael S. / SCORE Study Report 8 : Closed tests for all pairwise comparisons of means. In: Drug Information Journal. 2010 ; Vol. 44, No. 4. pp. 405-420.
@article{bd028c6566024dc0a1b8a663863b4b45,
title = "SCORE Study Report 8: Closed tests for all pairwise comparisons of means",
abstract = "We compare five closed tests for strong control of family-wide type I error while making all pairwise comparisons of means in clinical trials with multiple arms such as the SCORE Study. We simulated outcomes of the SCORE Study under its design hypotheses, and used P values from chi-squared tests to compare performance of a pairwise closed test described below to Bonferroni and Hochberg adjusted P values. Pairwise closed testing was more powerful than Hochberg's method by several definitions of multiple-test power. Simulations over a wider parameter space, and considering other closed methods, confirmed this superiority for P values based on normal, logistic, and Poisson distributions. The power benefit of pairwise closed testing begins to disappear with five or more arms and with unbalanced designs. For trials with four or fewer arms and balanced designs, investigators should consider using pairwise closed testing in preference to Shaffer's, Hommel's, and Hochberg's approaches when making all pairwise comparisons of means. If not all P values from the closed family are available, Shaffer's method is a good choice.",
author = "Neal Oden and VanVeldhuisen, {Paul C.} and Ingrid Scott and Ip, {Michael S.}",
year = "2010",
month = "1",
day = "1",
doi = "10.1177/009286151004400405",
language = "English (US)",
volume = "44",
pages = "405--420",
journal = "Drug Information Journal",
issn = "0092-8615",
publisher = "Drug Information Association",
number = "4",

}

SCORE Study Report 8 : Closed tests for all pairwise comparisons of means. / Oden, Neal; VanVeldhuisen, Paul C.; Scott, Ingrid; Ip, Michael S.

In: Drug Information Journal, Vol. 44, No. 4, 01.01.2010, p. 405-420.

Research output: Contribution to journalReview article

TY - JOUR

T1 - SCORE Study Report 8

T2 - Closed tests for all pairwise comparisons of means

AU - Oden, Neal

AU - VanVeldhuisen, Paul C.

AU - Scott, Ingrid

AU - Ip, Michael S.

PY - 2010/1/1

Y1 - 2010/1/1

N2 - We compare five closed tests for strong control of family-wide type I error while making all pairwise comparisons of means in clinical trials with multiple arms such as the SCORE Study. We simulated outcomes of the SCORE Study under its design hypotheses, and used P values from chi-squared tests to compare performance of a pairwise closed test described below to Bonferroni and Hochberg adjusted P values. Pairwise closed testing was more powerful than Hochberg's method by several definitions of multiple-test power. Simulations over a wider parameter space, and considering other closed methods, confirmed this superiority for P values based on normal, logistic, and Poisson distributions. The power benefit of pairwise closed testing begins to disappear with five or more arms and with unbalanced designs. For trials with four or fewer arms and balanced designs, investigators should consider using pairwise closed testing in preference to Shaffer's, Hommel's, and Hochberg's approaches when making all pairwise comparisons of means. If not all P values from the closed family are available, Shaffer's method is a good choice.

AB - We compare five closed tests for strong control of family-wide type I error while making all pairwise comparisons of means in clinical trials with multiple arms such as the SCORE Study. We simulated outcomes of the SCORE Study under its design hypotheses, and used P values from chi-squared tests to compare performance of a pairwise closed test described below to Bonferroni and Hochberg adjusted P values. Pairwise closed testing was more powerful than Hochberg's method by several definitions of multiple-test power. Simulations over a wider parameter space, and considering other closed methods, confirmed this superiority for P values based on normal, logistic, and Poisson distributions. The power benefit of pairwise closed testing begins to disappear with five or more arms and with unbalanced designs. For trials with four or fewer arms and balanced designs, investigators should consider using pairwise closed testing in preference to Shaffer's, Hommel's, and Hochberg's approaches when making all pairwise comparisons of means. If not all P values from the closed family are available, Shaffer's method is a good choice.

UR - http://www.scopus.com/inward/record.url?scp=77954909443&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77954909443&partnerID=8YFLogxK

U2 - 10.1177/009286151004400405

DO - 10.1177/009286151004400405

M3 - Review article

AN - SCOPUS:77954909443

VL - 44

SP - 405

EP - 420

JO - Drug Information Journal

JF - Drug Information Journal

SN - 0092-8615

IS - 4

ER -