### Abstract

Two new methods are proposed for linear regression analysis for data with measurement errors. Both methods are designed to accommodate intrinsic scatter in addition to measurement errors. The first method is a direct extension of the ordinary least squares (OLS) estimator to allow for measurement errors. It is quite general in that (1) it allows for measurement errors on both variables (2) it permits the measurement errors for the two variables to be dependent, (3) it permits the magnitudes of the measurement errors to depend on the measurements, and (4) other "symmetric" lines such as the bisector and the orthogonal regression can be constructed. We refer to this method as BCES estimators (for bivariate correlated errors and intrinsic scatter). The second method is a weighted least squares (WLS) estimator, which applies only for the case in which the "independent" variable is measured without error and the magnitudes of the measurement errors on the "dependent" variable are independent of the measurements. Several applications are made to extragalactic astronomy: the BCES method, when applied to data describing the color-luminosity relations for field galaxies, yields significantly different slopes than OLS and other estimators used in the literature. Simulations with artificial data sets are used to evaluate the small sample performance of the estimators. Not surprisingly, the least-biased results are obtained when color is treated as the dependent variable. The Tully-Fisher relation is another example for which the BCES method should be used because errors in luminosity and velocity are correlated because of inclination corrections. We also find, via simulations, that the WLS method is by far the best method for the Tolman surface-brightness test, producing the smallest variance in slope by an order of magnitude. Moreover, with WLS it is not necessary to "reduce" galaxies to a fiducial surface-brightness, since this model incorporates intrinsic scatter.

Original language | English (US) |
---|---|

Pages (from-to) | 706-714 |

Number of pages | 9 |

Journal | Astrophysical Journal |

Volume | 470 |

Issue number | 2 PART I |

DOIs | |

State | Published - Jan 1 1996 |

### Fingerprint

### All Science Journal Classification (ASJC) codes

- Astronomy and Astrophysics
- Space and Planetary Science

### Cite this

*Astrophysical Journal*,

*470*(2 PART I), 706-714. https://doi.org/10.1086/177901

}

*Astrophysical Journal*, vol. 470, no. 2 PART I, pp. 706-714. https://doi.org/10.1086/177901

**Linear regression for astronomical data with measurement errors and intrinsic scatter.** / Akritas, Michael G.; Bershadv, Matthew A.

Research output: Contribution to journal › Article

TY - JOUR

T1 - Linear regression for astronomical data with measurement errors and intrinsic scatter

AU - Akritas, Michael G.

AU - Bershadv, Matthew A.

PY - 1996/1/1

Y1 - 1996/1/1

N2 - Two new methods are proposed for linear regression analysis for data with measurement errors. Both methods are designed to accommodate intrinsic scatter in addition to measurement errors. The first method is a direct extension of the ordinary least squares (OLS) estimator to allow for measurement errors. It is quite general in that (1) it allows for measurement errors on both variables (2) it permits the measurement errors for the two variables to be dependent, (3) it permits the magnitudes of the measurement errors to depend on the measurements, and (4) other "symmetric" lines such as the bisector and the orthogonal regression can be constructed. We refer to this method as BCES estimators (for bivariate correlated errors and intrinsic scatter). The second method is a weighted least squares (WLS) estimator, which applies only for the case in which the "independent" variable is measured without error and the magnitudes of the measurement errors on the "dependent" variable are independent of the measurements. Several applications are made to extragalactic astronomy: the BCES method, when applied to data describing the color-luminosity relations for field galaxies, yields significantly different slopes than OLS and other estimators used in the literature. Simulations with artificial data sets are used to evaluate the small sample performance of the estimators. Not surprisingly, the least-biased results are obtained when color is treated as the dependent variable. The Tully-Fisher relation is another example for which the BCES method should be used because errors in luminosity and velocity are correlated because of inclination corrections. We also find, via simulations, that the WLS method is by far the best method for the Tolman surface-brightness test, producing the smallest variance in slope by an order of magnitude. Moreover, with WLS it is not necessary to "reduce" galaxies to a fiducial surface-brightness, since this model incorporates intrinsic scatter.

AB - Two new methods are proposed for linear regression analysis for data with measurement errors. Both methods are designed to accommodate intrinsic scatter in addition to measurement errors. The first method is a direct extension of the ordinary least squares (OLS) estimator to allow for measurement errors. It is quite general in that (1) it allows for measurement errors on both variables (2) it permits the measurement errors for the two variables to be dependent, (3) it permits the magnitudes of the measurement errors to depend on the measurements, and (4) other "symmetric" lines such as the bisector and the orthogonal regression can be constructed. We refer to this method as BCES estimators (for bivariate correlated errors and intrinsic scatter). The second method is a weighted least squares (WLS) estimator, which applies only for the case in which the "independent" variable is measured without error and the magnitudes of the measurement errors on the "dependent" variable are independent of the measurements. Several applications are made to extragalactic astronomy: the BCES method, when applied to data describing the color-luminosity relations for field galaxies, yields significantly different slopes than OLS and other estimators used in the literature. Simulations with artificial data sets are used to evaluate the small sample performance of the estimators. Not surprisingly, the least-biased results are obtained when color is treated as the dependent variable. The Tully-Fisher relation is another example for which the BCES method should be used because errors in luminosity and velocity are correlated because of inclination corrections. We also find, via simulations, that the WLS method is by far the best method for the Tolman surface-brightness test, producing the smallest variance in slope by an order of magnitude. Moreover, with WLS it is not necessary to "reduce" galaxies to a fiducial surface-brightness, since this model incorporates intrinsic scatter.

UR - http://www.scopus.com/inward/record.url?scp=21444440377&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=21444440377&partnerID=8YFLogxK

U2 - 10.1086/177901

DO - 10.1086/177901

M3 - Article

VL - 470

SP - 706

EP - 714

JO - Astrophysical Journal

JF - Astrophysical Journal

SN - 0004-637X

IS - 2 PART I

ER -