Distributed in-memory processing of All K Nearest Neighbor queries

Georgios Chatzimilioudis, Constantinos Costa, Demetrios Zeinalipour-Yazti, Wang-chien Lee, Evaggelia Pitoura

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)

Abstract

A wide spectrum of Internet-scale mobile applications, ranging from social networking, gaming and entertainment to emergency response and crisis management, all require efficient and scalable All k Nearest Neighbor (AkNN) computations over millions of moving objects every few seconds to be operational. In this paper we present Spitfire, a distributed algorithm that provides a scalable and high-performance AkNN processing framework to our award-winning geo-social network named Rayzit. The proposed algorithm deploys a fast load-balanced partitioning along with an efficient replication-set selection, to provide fast main-memory computations of the exact AkNN results in a batch-oriented manner. We evaluate, both analytically and experimentally, how the pruning efficiency of the Spitfire algorithm plays a pivotal role in reducing communication and response time up to an order of magnitude, compared to three state-of-the-art distributed AkNN algorithms executed in distributed main-memory.

Original languageEnglish (US)
Title of host publication2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages1490-1491
Number of pages2
ISBN (Electronic)9781509020195
DOIs
StatePublished - Jun 22 2016
Event32nd IEEE International Conference on Data Engineering, ICDE 2016 - Helsinki, Finland
Duration: May 16 2016May 20 2016

Other

Other32nd IEEE International Conference on Data Engineering, ICDE 2016
CountryFinland
CityHelsinki
Period5/16/165/20/16

Fingerprint

Data storage equipment
Processing
Parallel algorithms
Internet
Communication
Query
K-nearest neighbor
Pruning
Crisis management
Emergency response
Replication
Social networks
High performance
Partitioning
Entertainment
Response time
Social networking
World Wide Web
Mobile applications
Gaming

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Computational Theory and Mathematics
  • Computer Graphics and Computer-Aided Design
  • Computer Networks and Communications
  • Information Systems
  • Information Systems and Management

Cite this

Chatzimilioudis, G., Costa, C., Zeinalipour-Yazti, D., Lee, W., & Pitoura, E. (2016). Distributed in-memory processing of All K Nearest Neighbor queries. In 2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016 (pp. 1490-1491). [7498389] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICDE.2016.7498389
Chatzimilioudis, Georgios ; Costa, Constantinos ; Zeinalipour-Yazti, Demetrios ; Lee, Wang-chien ; Pitoura, Evaggelia. / Distributed in-memory processing of All K Nearest Neighbor queries. 2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016. Institute of Electrical and Electronics Engineers Inc., 2016. pp. 1490-1491
@inproceedings{be519ee12714457cbe36472e95ea3341,
title = "Distributed in-memory processing of All K Nearest Neighbor queries",
abstract = "A wide spectrum of Internet-scale mobile applications, ranging from social networking, gaming and entertainment to emergency response and crisis management, all require efficient and scalable All k Nearest Neighbor (AkNN) computations over millions of moving objects every few seconds to be operational. In this paper we present Spitfire, a distributed algorithm that provides a scalable and high-performance AkNN processing framework to our award-winning geo-social network named Rayzit. The proposed algorithm deploys a fast load-balanced partitioning along with an efficient replication-set selection, to provide fast main-memory computations of the exact AkNN results in a batch-oriented manner. We evaluate, both analytically and experimentally, how the pruning efficiency of the Spitfire algorithm plays a pivotal role in reducing communication and response time up to an order of magnitude, compared to three state-of-the-art distributed AkNN algorithms executed in distributed main-memory.",
author = "Georgios Chatzimilioudis and Constantinos Costa and Demetrios Zeinalipour-Yazti and Wang-chien Lee and Evaggelia Pitoura",
year = "2016",
month = "6",
day = "22",
doi = "10.1109/ICDE.2016.7498389",
language = "English (US)",
pages = "1490--1491",
booktitle = "2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
address = "United States",

}

Chatzimilioudis, G, Costa, C, Zeinalipour-Yazti, D, Lee, W & Pitoura, E 2016, Distributed in-memory processing of All K Nearest Neighbor queries. in 2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016., 7498389, Institute of Electrical and Electronics Engineers Inc., pp. 1490-1491, 32nd IEEE International Conference on Data Engineering, ICDE 2016, Helsinki, Finland, 5/16/16. https://doi.org/10.1109/ICDE.2016.7498389

Distributed in-memory processing of All K Nearest Neighbor queries. / Chatzimilioudis, Georgios; Costa, Constantinos; Zeinalipour-Yazti, Demetrios; Lee, Wang-chien; Pitoura, Evaggelia.

2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016. Institute of Electrical and Electronics Engineers Inc., 2016. p. 1490-1491 7498389.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Distributed in-memory processing of All K Nearest Neighbor queries

AU - Chatzimilioudis, Georgios

AU - Costa, Constantinos

AU - Zeinalipour-Yazti, Demetrios

AU - Lee, Wang-chien

AU - Pitoura, Evaggelia

PY - 2016/6/22

Y1 - 2016/6/22

N2 - A wide spectrum of Internet-scale mobile applications, ranging from social networking, gaming and entertainment to emergency response and crisis management, all require efficient and scalable All k Nearest Neighbor (AkNN) computations over millions of moving objects every few seconds to be operational. In this paper we present Spitfire, a distributed algorithm that provides a scalable and high-performance AkNN processing framework to our award-winning geo-social network named Rayzit. The proposed algorithm deploys a fast load-balanced partitioning along with an efficient replication-set selection, to provide fast main-memory computations of the exact AkNN results in a batch-oriented manner. We evaluate, both analytically and experimentally, how the pruning efficiency of the Spitfire algorithm plays a pivotal role in reducing communication and response time up to an order of magnitude, compared to three state-of-the-art distributed AkNN algorithms executed in distributed main-memory.

AB - A wide spectrum of Internet-scale mobile applications, ranging from social networking, gaming and entertainment to emergency response and crisis management, all require efficient and scalable All k Nearest Neighbor (AkNN) computations over millions of moving objects every few seconds to be operational. In this paper we present Spitfire, a distributed algorithm that provides a scalable and high-performance AkNN processing framework to our award-winning geo-social network named Rayzit. The proposed algorithm deploys a fast load-balanced partitioning along with an efficient replication-set selection, to provide fast main-memory computations of the exact AkNN results in a batch-oriented manner. We evaluate, both analytically and experimentally, how the pruning efficiency of the Spitfire algorithm plays a pivotal role in reducing communication and response time up to an order of magnitude, compared to three state-of-the-art distributed AkNN algorithms executed in distributed main-memory.

UR - http://www.scopus.com/inward/record.url?scp=84980385801&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84980385801&partnerID=8YFLogxK

U2 - 10.1109/ICDE.2016.7498389

DO - 10.1109/ICDE.2016.7498389

M3 - Conference contribution

AN - SCOPUS:84980385801

SP - 1490

EP - 1491

BT - 2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Chatzimilioudis G, Costa C, Zeinalipour-Yazti D, Lee W, Pitoura E. Distributed in-memory processing of All K Nearest Neighbor queries. In 2016 IEEE 32nd International Conference on Data Engineering, ICDE 2016. Institute of Electrical and Electronics Engineers Inc. 2016. p. 1490-1491. 7498389 https://doi.org/10.1109/ICDE.2016.7498389