Indexing earth mover's distance over network metrics

Ting Wang, Shicong Meng, Jiang Bian

Research output: Contribution to journalArticlepeer-review

1 Scopus citations

Abstract

The Earth Mover's Distance (EMD) is a well-known distance metric for data represented as probability distributions over a predefined feature space. Supporting EMD-based similarity search has attracted intensive research effort. Despite the plethora of literature, most existing solutions are optimized for L-p feature spaces (e.g., Euclidean space); while in a spectrum of applications, the relationships between features are better captured using networks. In this paper, we study the problem of answering k-nearest neighbor (k-NN) queries under network-based EMD metrics (NEMD). We propose Oasis, a new access method which leverages the network structure of feature space and enables efficient NEMD-based similarity search. Specifically, Oasis employs three novel techniques: (i) Range Oracle, a scalable model to estimate the range of k-th nearest neighbor under NEMD, (ii) Boundary Index, a structure that efficiently fetches candidates within given range, and (iii) Network Compression Hierarchy, an incremental filtering mechanism that effectively prunes false positive candidates to save unnecessary computation. Through extensive experiments using both synthetic and real data sets, we confirmed that Oasis significantly outperforms the state-of-the-art methods in query processing cost.

Original languageEnglish (US)
Article number6963483
Pages (from-to)1588-1601
Number of pages14
JournalIEEE Transactions on Knowledge and Data Engineering
Volume27
Issue number6
DOIs
StatePublished - Jun 1 2015

All Science Journal Classification (ASJC) codes

  • Information Systems
  • Computer Science Applications
  • Computational Theory and Mathematics

Fingerprint Dive into the research topics of 'Indexing earth mover's distance over network metrics'. Together they form a unique fingerprint.

Cite this