TWStream: Finding correlated data streams under time warping

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Consider the problem of monitoring multiple data streams and finding all correlated pairs in real time. Such correlations are of special interest for many applications, e.g., the price of two stocks may demonstrate quite similar rise/fall patterns, which provides the market trader with an opportunity of arbitrage. However, the correlated patterns may occur on any unknown scale, with arbitrary lag or even out of phase, which blinds most traditional methods. In this paper, we propose TWStream, a method that can detect pairs of streams, of which subsequences are correlated with elastic shift and arbitrary lag in the time axis. Specifically, (1) to accommodate varying scale and arbitrary lag, we propose to use the geometric time frame in conjunction with a piecewise smoothing approach; (2) to detect unsynchronized correlation, we extend the cross correlation to support time warping, which is proved much more robust than Euclidian based metrics. Our method has a sound theoretical foundation, and is efficient in terms of both time and space complexity. Experiments on both synthetic and real data are done to show its effectiveness and efficiency.

Original languageEnglish (US)
Title of host publicationFrontiers of WWW Research and Development - APWeb 2006 - 8th Asia-Pacific Web Conference, Proceedings
PublisherSpringer Verlag
Pages213-225
Number of pages13
ISBN (Print)3540311424, 9783540311423
DOIs
StatePublished - Jan 1 2006
Event8th Asia-Pacific Web Conference, APWeb 2006: Frontiers of WWW Research and Development - Harbin, China
Duration: Jan 16 2006Jan 18 2006

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume3841 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference8th Asia-Pacific Web Conference, APWeb 2006: Frontiers of WWW Research and Development
CountryChina
CityHarbin
Period1/16/061/18/06

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Fingerprint Dive into the research topics of 'TWStream: Finding correlated data streams under time warping'. Together they form a unique fingerprint.

  • Cite this

    Wang, T. (2006). TWStream: Finding correlated data streams under time warping. In Frontiers of WWW Research and Development - APWeb 2006 - 8th Asia-Pacific Web Conference, Proceedings (pp. 213-225). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 3841 LNCS). Springer Verlag. https://doi.org/10.1007/11610113_20