Efficient joins with compressed bitmap indexes

Kamesh Madduri, Kesheng Wu

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Citations (Scopus)

Abstract

We present a new class of adaptive algorithms that use compressed bitmap indexes to speed up evaluation of the range join query in relational databases. We determine the best strategy to process a join query based on a fast sub-linear time computation of the join selectivity (the ratio of the number of tuples in the result to the total number of possible tuples). In addition, we use compressed bitmaps to represent the join output compactly: the space requirement for storing the tuples representing the join of two relations is asymptotically bounded by min(h; n.cb), where h is the number of tuple pairs in the result relation, n is the number of tuples in the smaller of the two relations, and cb is the cardinality of the larger column being joined. We present a theoretical analysis of our algorithms, as well as experimental results on large-scale synthetic and real data sets. Our implementations are efficient, and consistently outperform well-known approaches for a range of join selectivity factors. For instance, our count-only algorithm is up to three orders of magnitude faster than the sort-merge approach, and our best bitmap index-based algorithm is 1.2x-80x faster than the sort-merge algorithm, for various query instances. We achieve these speedups by exploiting several inherent performance advantages of compressed bitmap indexes for join processing: an implicit partitioning of the attributes, space-efficiency, and tolerance of high-cardinality relations.

Original languageEnglish (US)
Title of host publicationACM 18th International Conference on Information and Knowledge Management, CIKM 2009
Pages1017-1026
Number of pages10
DOIs
StatePublished - Dec 1 2009
EventACM 18th International Conference on Information and Knowledge Management, CIKM 2009 - Hong Kong, China
Duration: Nov 2 2009Nov 6 2009

Publication series

NameInternational Conference on Information and Knowledge Management, Proceedings

Other

OtherACM 18th International Conference on Information and Knowledge Management, CIKM 2009
CountryChina
CityHong Kong
Period11/2/0911/6/09

Fingerprint

Join
Query
Selectivity
Partitioning
Evaluation
Tolerance
Relational database
Theoretical analysis
Factors

All Science Journal Classification (ASJC) codes

  • Decision Sciences(all)
  • Business, Management and Accounting(all)

Cite this

Madduri, K., & Wu, K. (2009). Efficient joins with compressed bitmap indexes. In ACM 18th International Conference on Information and Knowledge Management, CIKM 2009 (pp. 1017-1026). (International Conference on Information and Knowledge Management, Proceedings). https://doi.org/10.1145/1645953.1646083
Madduri, Kamesh ; Wu, Kesheng. / Efficient joins with compressed bitmap indexes. ACM 18th International Conference on Information and Knowledge Management, CIKM 2009. 2009. pp. 1017-1026 (International Conference on Information and Knowledge Management, Proceedings).
@inproceedings{7efc3eece9004b5798adfb03250f28b7,
title = "Efficient joins with compressed bitmap indexes",
abstract = "We present a new class of adaptive algorithms that use compressed bitmap indexes to speed up evaluation of the range join query in relational databases. We determine the best strategy to process a join query based on a fast sub-linear time computation of the join selectivity (the ratio of the number of tuples in the result to the total number of possible tuples). In addition, we use compressed bitmaps to represent the join output compactly: the space requirement for storing the tuples representing the join of two relations is asymptotically bounded by min(h; n.cb), where h is the number of tuple pairs in the result relation, n is the number of tuples in the smaller of the two relations, and cb is the cardinality of the larger column being joined. We present a theoretical analysis of our algorithms, as well as experimental results on large-scale synthetic and real data sets. Our implementations are efficient, and consistently outperform well-known approaches for a range of join selectivity factors. For instance, our count-only algorithm is up to three orders of magnitude faster than the sort-merge approach, and our best bitmap index-based algorithm is 1.2x-80x faster than the sort-merge algorithm, for various query instances. We achieve these speedups by exploiting several inherent performance advantages of compressed bitmap indexes for join processing: an implicit partitioning of the attributes, space-efficiency, and tolerance of high-cardinality relations.",
author = "Kamesh Madduri and Kesheng Wu",
year = "2009",
month = "12",
day = "1",
doi = "10.1145/1645953.1646083",
language = "English (US)",
isbn = "9781605585123",
series = "International Conference on Information and Knowledge Management, Proceedings",
pages = "1017--1026",
booktitle = "ACM 18th International Conference on Information and Knowledge Management, CIKM 2009",

}

Madduri, K & Wu, K 2009, Efficient joins with compressed bitmap indexes. in ACM 18th International Conference on Information and Knowledge Management, CIKM 2009. International Conference on Information and Knowledge Management, Proceedings, pp. 1017-1026, ACM 18th International Conference on Information and Knowledge Management, CIKM 2009, Hong Kong, China, 11/2/09. https://doi.org/10.1145/1645953.1646083

Efficient joins with compressed bitmap indexes. / Madduri, Kamesh; Wu, Kesheng.

ACM 18th International Conference on Information and Knowledge Management, CIKM 2009. 2009. p. 1017-1026 (International Conference on Information and Knowledge Management, Proceedings).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Efficient joins with compressed bitmap indexes

AU - Madduri, Kamesh

AU - Wu, Kesheng

PY - 2009/12/1

Y1 - 2009/12/1

N2 - We present a new class of adaptive algorithms that use compressed bitmap indexes to speed up evaluation of the range join query in relational databases. We determine the best strategy to process a join query based on a fast sub-linear time computation of the join selectivity (the ratio of the number of tuples in the result to the total number of possible tuples). In addition, we use compressed bitmaps to represent the join output compactly: the space requirement for storing the tuples representing the join of two relations is asymptotically bounded by min(h; n.cb), where h is the number of tuple pairs in the result relation, n is the number of tuples in the smaller of the two relations, and cb is the cardinality of the larger column being joined. We present a theoretical analysis of our algorithms, as well as experimental results on large-scale synthetic and real data sets. Our implementations are efficient, and consistently outperform well-known approaches for a range of join selectivity factors. For instance, our count-only algorithm is up to three orders of magnitude faster than the sort-merge approach, and our best bitmap index-based algorithm is 1.2x-80x faster than the sort-merge algorithm, for various query instances. We achieve these speedups by exploiting several inherent performance advantages of compressed bitmap indexes for join processing: an implicit partitioning of the attributes, space-efficiency, and tolerance of high-cardinality relations.

AB - We present a new class of adaptive algorithms that use compressed bitmap indexes to speed up evaluation of the range join query in relational databases. We determine the best strategy to process a join query based on a fast sub-linear time computation of the join selectivity (the ratio of the number of tuples in the result to the total number of possible tuples). In addition, we use compressed bitmaps to represent the join output compactly: the space requirement for storing the tuples representing the join of two relations is asymptotically bounded by min(h; n.cb), where h is the number of tuple pairs in the result relation, n is the number of tuples in the smaller of the two relations, and cb is the cardinality of the larger column being joined. We present a theoretical analysis of our algorithms, as well as experimental results on large-scale synthetic and real data sets. Our implementations are efficient, and consistently outperform well-known approaches for a range of join selectivity factors. For instance, our count-only algorithm is up to three orders of magnitude faster than the sort-merge approach, and our best bitmap index-based algorithm is 1.2x-80x faster than the sort-merge algorithm, for various query instances. We achieve these speedups by exploiting several inherent performance advantages of compressed bitmap indexes for join processing: an implicit partitioning of the attributes, space-efficiency, and tolerance of high-cardinality relations.

UR - http://www.scopus.com/inward/record.url?scp=74549186511&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=74549186511&partnerID=8YFLogxK

U2 - 10.1145/1645953.1646083

DO - 10.1145/1645953.1646083

M3 - Conference contribution

SN - 9781605585123

T3 - International Conference on Information and Knowledge Management, Proceedings

SP - 1017

EP - 1026

BT - ACM 18th International Conference on Information and Knowledge Management, CIKM 2009

ER -

Madduri K, Wu K. Efficient joins with compressed bitmap indexes. In ACM 18th International Conference on Information and Knowledge Management, CIKM 2009. 2009. p. 1017-1026. (International Conference on Information and Knowledge Management, Proceedings). https://doi.org/10.1145/1645953.1646083