On second-order optimality of the observed fisher information

Bruce G. Lindsay, Bing Li

Research output: Contribution to journalArticle

20 Citations (Scopus)

Abstract

The realized error of an estimate is determined not only by the efficiency of the estimator, but also by chance. For example, suppose that we have observed a bivariate normal vector whose expectation is known to be on a circle. Then, intuitively, the longer that vector happens to be, the more accurately its angle is likely to be estimated. Yet this chance, though its information is contained in the data, cannot be accounted for by the variance of the estimate. One way to capture it is by the direct estimation of the realized error. In this paper, we will demonstrate that the squared error of the maximum likelihood estimate, to the extent to which it can be estimated, can be most accurately estimated by the inverse of the observed Fisher information. In relation to this optimality, we will also study the properties of several other estimators, including the inverse of the expected Fisher information, the sandwich estimators, the jackknife and the bootstrap estimators. Unlike the observed Fisher information, these estimators are not optimal.

Original languageEnglish (US)
Pages (from-to)2172-2199
Number of pages28
JournalAnnals of Statistics
Volume25
Issue number5
DOIs
StatePublished - Jan 1 1997

Fingerprint

Observed Information
Fisher Information
Optimality
Estimator
Sandwich Estimator
Bivariate Normal
Jackknife
Normal vector
Maximum Likelihood Estimate
Estimate
Bootstrap
Circle
Likely
Angle
Fisher information
Demonstrate

All Science Journal Classification (ASJC) codes

  • Statistics and Probability
  • Statistics, Probability and Uncertainty

Cite this

Lindsay, Bruce G. ; Li, Bing. / On second-order optimality of the observed fisher information. In: Annals of Statistics. 1997 ; Vol. 25, No. 5. pp. 2172-2199.
@article{4ed7fe9ad80841c39c31e310ee071b6d,
title = "On second-order optimality of the observed fisher information",
abstract = "The realized error of an estimate is determined not only by the efficiency of the estimator, but also by chance. For example, suppose that we have observed a bivariate normal vector whose expectation is known to be on a circle. Then, intuitively, the longer that vector happens to be, the more accurately its angle is likely to be estimated. Yet this chance, though its information is contained in the data, cannot be accounted for by the variance of the estimate. One way to capture it is by the direct estimation of the realized error. In this paper, we will demonstrate that the squared error of the maximum likelihood estimate, to the extent to which it can be estimated, can be most accurately estimated by the inverse of the observed Fisher information. In relation to this optimality, we will also study the properties of several other estimators, including the inverse of the expected Fisher information, the sandwich estimators, the jackknife and the bootstrap estimators. Unlike the observed Fisher information, these estimators are not optimal.",
author = "Lindsay, {Bruce G.} and Bing Li",
year = "1997",
month = "1",
day = "1",
doi = "10.1214/aos/1069362393",
language = "English (US)",
volume = "25",
pages = "2172--2199",
journal = "Annals of Statistics",
issn = "0090-5364",
publisher = "Institute of Mathematical Statistics",
number = "5",

}

On second-order optimality of the observed fisher information. / Lindsay, Bruce G.; Li, Bing.

In: Annals of Statistics, Vol. 25, No. 5, 01.01.1997, p. 2172-2199.

Research output: Contribution to journalArticle

TY - JOUR

T1 - On second-order optimality of the observed fisher information

AU - Lindsay, Bruce G.

AU - Li, Bing

PY - 1997/1/1

Y1 - 1997/1/1

N2 - The realized error of an estimate is determined not only by the efficiency of the estimator, but also by chance. For example, suppose that we have observed a bivariate normal vector whose expectation is known to be on a circle. Then, intuitively, the longer that vector happens to be, the more accurately its angle is likely to be estimated. Yet this chance, though its information is contained in the data, cannot be accounted for by the variance of the estimate. One way to capture it is by the direct estimation of the realized error. In this paper, we will demonstrate that the squared error of the maximum likelihood estimate, to the extent to which it can be estimated, can be most accurately estimated by the inverse of the observed Fisher information. In relation to this optimality, we will also study the properties of several other estimators, including the inverse of the expected Fisher information, the sandwich estimators, the jackknife and the bootstrap estimators. Unlike the observed Fisher information, these estimators are not optimal.

AB - The realized error of an estimate is determined not only by the efficiency of the estimator, but also by chance. For example, suppose that we have observed a bivariate normal vector whose expectation is known to be on a circle. Then, intuitively, the longer that vector happens to be, the more accurately its angle is likely to be estimated. Yet this chance, though its information is contained in the data, cannot be accounted for by the variance of the estimate. One way to capture it is by the direct estimation of the realized error. In this paper, we will demonstrate that the squared error of the maximum likelihood estimate, to the extent to which it can be estimated, can be most accurately estimated by the inverse of the observed Fisher information. In relation to this optimality, we will also study the properties of several other estimators, including the inverse of the expected Fisher information, the sandwich estimators, the jackknife and the bootstrap estimators. Unlike the observed Fisher information, these estimators are not optimal.

UR - http://www.scopus.com/inward/record.url?scp=0031495021&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0031495021&partnerID=8YFLogxK

U2 - 10.1214/aos/1069362393

DO - 10.1214/aos/1069362393

M3 - Article

AN - SCOPUS:0031495021

VL - 25

SP - 2172

EP - 2199

JO - Annals of Statistics

JF - Annals of Statistics

SN - 0090-5364

IS - 5

ER -