TWStream: Finding correlated data streams under time warping

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Consider the problem of monitoring multiple data streams and finding all correlated pairs in real time. Such correlations are of special interest for many applications, e.g., the price of two stocks may demonstrate quite similar rise/fall patterns, which provides the market trader with an opportunity of arbitrage. However, the correlated patterns may occur on any unknown scale, with arbitrary lag or even out of phase, which blinds most traditional methods. In this paper, we propose TWStream, a method that can detect pairs of streams, of which subsequences are correlated with elastic shift and arbitrary lag in the time axis. Specifically, (1) to accommodate varying scale and arbitrary lag, we propose to use the geometric time frame in conjunction with a piecewise smoothing approach; (2) to detect unsynchronized correlation, we extend the cross correlation to support time warping, which is proved much more robust than Euclidian based metrics. Our method has a sound theoretical foundation, and is efficient in terms of both time and space complexity. Experiments on both synthetic and real data are done to show its effectiveness and efficiency.

Original languageEnglish (US)
Title of host publicationFrontiers of WWW Research and Development - APWeb 2006 - 8th Asia-Pacific Web Conference, Proceedings
Pages213-225
Number of pages13
StatePublished - Jul 6 2006
Event8th Asia-Pacific Web Conference, APWeb 2006: Frontiers of WWW Research and Development - Harbin, China
Duration: Jan 16 2006Jan 18 2006

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume3841 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference8th Asia-Pacific Web Conference, APWeb 2006: Frontiers of WWW Research and Development
CountryChina
CityHarbin
Period1/16/061/18/06

Fingerprint

Time Warping
Correlated Data
Data Streams
Acoustic waves
Monitoring
Arbitrary
Arbitrage
Experiments
Space Complexity
Cross-correlation
Subsequence
Time Complexity
Smoothing
Metric
Unknown
Demonstrate
Experiment

All Science Journal Classification (ASJC) codes

  • Theoretical Computer Science
  • Computer Science(all)

Cite this

Wang, T. (2006). TWStream: Finding correlated data streams under time warping. In Frontiers of WWW Research and Development - APWeb 2006 - 8th Asia-Pacific Web Conference, Proceedings (pp. 213-225). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 3841 LNCS).
Wang, Ting. / TWStream : Finding correlated data streams under time warping. Frontiers of WWW Research and Development - APWeb 2006 - 8th Asia-Pacific Web Conference, Proceedings. 2006. pp. 213-225 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).
@inproceedings{dbf63252549747359d5ff84d6e6573da,
title = "TWStream: Finding correlated data streams under time warping",
abstract = "Consider the problem of monitoring multiple data streams and finding all correlated pairs in real time. Such correlations are of special interest for many applications, e.g., the price of two stocks may demonstrate quite similar rise/fall patterns, which provides the market trader with an opportunity of arbitrage. However, the correlated patterns may occur on any unknown scale, with arbitrary lag or even out of phase, which blinds most traditional methods. In this paper, we propose TWStream, a method that can detect pairs of streams, of which subsequences are correlated with elastic shift and arbitrary lag in the time axis. Specifically, (1) to accommodate varying scale and arbitrary lag, we propose to use the geometric time frame in conjunction with a piecewise smoothing approach; (2) to detect unsynchronized correlation, we extend the cross correlation to support time warping, which is proved much more robust than Euclidian based metrics. Our method has a sound theoretical foundation, and is efficient in terms of both time and space complexity. Experiments on both synthetic and real data are done to show its effectiveness and efficiency.",
author = "Ting Wang",
year = "2006",
month = "7",
day = "6",
language = "English (US)",
isbn = "3540311424",
series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",
pages = "213--225",
booktitle = "Frontiers of WWW Research and Development - APWeb 2006 - 8th Asia-Pacific Web Conference, Proceedings",

}

Wang, T 2006, TWStream: Finding correlated data streams under time warping. in Frontiers of WWW Research and Development - APWeb 2006 - 8th Asia-Pacific Web Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 3841 LNCS, pp. 213-225, 8th Asia-Pacific Web Conference, APWeb 2006: Frontiers of WWW Research and Development, Harbin, China, 1/16/06.

TWStream : Finding correlated data streams under time warping. / Wang, Ting.

Frontiers of WWW Research and Development - APWeb 2006 - 8th Asia-Pacific Web Conference, Proceedings. 2006. p. 213-225 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 3841 LNCS).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - TWStream

T2 - Finding correlated data streams under time warping

AU - Wang, Ting

PY - 2006/7/6

Y1 - 2006/7/6

N2 - Consider the problem of monitoring multiple data streams and finding all correlated pairs in real time. Such correlations are of special interest for many applications, e.g., the price of two stocks may demonstrate quite similar rise/fall patterns, which provides the market trader with an opportunity of arbitrage. However, the correlated patterns may occur on any unknown scale, with arbitrary lag or even out of phase, which blinds most traditional methods. In this paper, we propose TWStream, a method that can detect pairs of streams, of which subsequences are correlated with elastic shift and arbitrary lag in the time axis. Specifically, (1) to accommodate varying scale and arbitrary lag, we propose to use the geometric time frame in conjunction with a piecewise smoothing approach; (2) to detect unsynchronized correlation, we extend the cross correlation to support time warping, which is proved much more robust than Euclidian based metrics. Our method has a sound theoretical foundation, and is efficient in terms of both time and space complexity. Experiments on both synthetic and real data are done to show its effectiveness and efficiency.

AB - Consider the problem of monitoring multiple data streams and finding all correlated pairs in real time. Such correlations are of special interest for many applications, e.g., the price of two stocks may demonstrate quite similar rise/fall patterns, which provides the market trader with an opportunity of arbitrage. However, the correlated patterns may occur on any unknown scale, with arbitrary lag or even out of phase, which blinds most traditional methods. In this paper, we propose TWStream, a method that can detect pairs of streams, of which subsequences are correlated with elastic shift and arbitrary lag in the time axis. Specifically, (1) to accommodate varying scale and arbitrary lag, we propose to use the geometric time frame in conjunction with a piecewise smoothing approach; (2) to detect unsynchronized correlation, we extend the cross correlation to support time warping, which is proved much more robust than Euclidian based metrics. Our method has a sound theoretical foundation, and is efficient in terms of both time and space complexity. Experiments on both synthetic and real data are done to show its effectiveness and efficiency.

UR - http://www.scopus.com/inward/record.url?scp=33745669499&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=33745669499&partnerID=8YFLogxK

M3 - Conference contribution

AN - SCOPUS:33745669499

SN - 3540311424

SN - 9783540311423

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 213

EP - 225

BT - Frontiers of WWW Research and Development - APWeb 2006 - 8th Asia-Pacific Web Conference, Proceedings

ER -

Wang T. TWStream: Finding correlated data streams under time warping. In Frontiers of WWW Research and Development - APWeb 2006 - 8th Asia-Pacific Web Conference, Proceedings. 2006. p. 213-225. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).