Problems in the estimation and interpretation of the reliability of survey data

Duane F. Alwin

Research output: Contribution to journalArticlepeer-review

42 Scopus citations

Abstract

In this paper I discuss several of the difficulties involved in estimating the reliability of survey measurement. Reliability is defined on the basis of classical true-score theory, as the correlational consistency of multiple measures of the same construct, net of true change. This concept is presented within the framework of a theoretical discussion of the sources of error in survey data and the design requirements for separating response variation into components representing such response consistency and measurement errors. Discussion focuses on the potential sources of random and nonrandom errors, including "invalidity" of measurement, the term frequently used to refer to components of method variance. Problems with the estimation of these components are enumerated and discussed with respect to both cross-sectional and panel designs. Empirical examples are given of the estimation of the quantities of interest, which are the basis of a discussion of the interpretational difficulties encountered in reliability estimation. Data are drawn from the ISR's Quality of Life surveys, the National Election Studies and the NORC's General Social Surveys. The general conclusion is that both cross-sectional and panel estimates of measurement reliability are desirable, but for the purposes of isolating the random component of error, panel designs are probably the most advantageous.

Original languageEnglish (US)
Pages (from-to)277-331
Number of pages55
JournalQuality and Quantity
Volume23
Issue number3-4
DOIs
StatePublished - Sep 1989

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Social Sciences(all)

Fingerprint Dive into the research topics of 'Problems in the estimation and interpretation of the reliability of survey data'. Together they form a unique fingerprint.

Cite this