Single-pass low-storage arbitrary quantile estimation for massive datasets

John C. Liechty, Dennis K.J. Lin, James P. McDermott

Research output: Contribution to journalArticle

7 Citations (Scopus)

Abstract

We present a single-pass, low-storage, sequential method for estimating an arbitrary quantile of an unknown distribution. The proposed method performs very well when compared to existing methods for estimating the median as well as arbitrary quantiles for a wide range of densities. In addition to explaining the method and presenting the results of the simulation study, we discuss intuition behind the method and demonstrate empirically, for certain densities, that the proposed estimator converges to the sample quantile.

Original languageEnglish (US)
Pages (from-to)91-100
Number of pages10
JournalStatistics and Computing
Volume13
Issue number2
DOIs
StatePublished - Apr 1 2003

Fingerprint

Quantile Estimation
Arbitrary
Quantile
Sample Quantiles
Sequential Methods
Simulation Study
Converge
Estimator
Unknown
Quantile estimation
Range of data
Demonstrate

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computational Theory and Mathematics
  • Statistics and Probability

Cite this

@article{d7911e6d5f084cce8b5f39e92419db49,
title = "Single-pass low-storage arbitrary quantile estimation for massive datasets",
abstract = "We present a single-pass, low-storage, sequential method for estimating an arbitrary quantile of an unknown distribution. The proposed method performs very well when compared to existing methods for estimating the median as well as arbitrary quantiles for a wide range of densities. In addition to explaining the method and presenting the results of the simulation study, we discuss intuition behind the method and demonstrate empirically, for certain densities, that the proposed estimator converges to the sample quantile.",
author = "Liechty, {John C.} and Lin, {Dennis K.J.} and McDermott, {James P.}",
year = "2003",
month = "4",
day = "1",
doi = "10.1023/A:1023296123228",
language = "English (US)",
volume = "13",
pages = "91--100",
journal = "Statistics and Computing",
issn = "0960-3174",
publisher = "Springer Netherlands",
number = "2",

}

Single-pass low-storage arbitrary quantile estimation for massive datasets. / Liechty, John C.; Lin, Dennis K.J.; McDermott, James P.

In: Statistics and Computing, Vol. 13, No. 2, 01.04.2003, p. 91-100.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Single-pass low-storage arbitrary quantile estimation for massive datasets

AU - Liechty, John C.

AU - Lin, Dennis K.J.

AU - McDermott, James P.

PY - 2003/4/1

Y1 - 2003/4/1

N2 - We present a single-pass, low-storage, sequential method for estimating an arbitrary quantile of an unknown distribution. The proposed method performs very well when compared to existing methods for estimating the median as well as arbitrary quantiles for a wide range of densities. In addition to explaining the method and presenting the results of the simulation study, we discuss intuition behind the method and demonstrate empirically, for certain densities, that the proposed estimator converges to the sample quantile.

AB - We present a single-pass, low-storage, sequential method for estimating an arbitrary quantile of an unknown distribution. The proposed method performs very well when compared to existing methods for estimating the median as well as arbitrary quantiles for a wide range of densities. In addition to explaining the method and presenting the results of the simulation study, we discuss intuition behind the method and demonstrate empirically, for certain densities, that the proposed estimator converges to the sample quantile.

UR - http://www.scopus.com/inward/record.url?scp=0346463260&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0346463260&partnerID=8YFLogxK

U2 - 10.1023/A:1023296123228

DO - 10.1023/A:1023296123228

M3 - Article

VL - 13

SP - 91

EP - 100

JO - Statistics and Computing

JF - Statistics and Computing

SN - 0960-3174

IS - 2

ER -