HDBTracker: Monitoring the aggregates on dynamic hidden web databases

Weimo Liu, Saad Bin Suhaim, Saravanan Thirumuruganathan, Nan Zhang, Gautam Das, Ali Jaoua

Research output: Contribution to journalArticle

1 Scopus citations


Numerous web databases, e.g., amazon.com, eBay.com, are "hidden" behind (i.e., accessible only through) their restrictive search and browsing interfaces. This demonstration showcases HDBTracker, a web-based system that reveals and tracks (the changes of) userspecified aggregate queries over such hidden web databases, especially those that are frequently updated, by issuing a small number of search queries through the public web interfaces of these databases. The ability to track and monitor aggregates has applications over a wide variety of domains - e.g., government agencies can track COUNT of openings at online job hunting websites to understand key economic indicators, while businesses can track the AVG price of a product over a basket of e-commerce websites to understand the competitive landscape and/or material costs. A key technique used in HDBTracker is RS-ESTIMATOR, the first algorithm that can efficiently monitor changes to aggregate query answers over a hidden web database.

Original languageEnglish (US)
Pages (from-to)1569-1572
Number of pages4
JournalProceedings of the VLDB Endowment
Issue number13
StatePublished - 2014

All Science Journal Classification (ASJC) codes

  • Computer Science (miscellaneous)
  • Computer Science(all)

Fingerprint Dive into the research topics of 'HDBTracker: Monitoring the aggregates on dynamic hidden web databases'. Together they form a unique fingerprint.

  • Cite this

    Liu, W., Suhaim, S. B., Thirumuruganathan, S., Zhang, N., Das, G., & Jaoua, A. (2014). HDBTracker: Monitoring the aggregates on dynamic hidden web databases. Proceedings of the VLDB Endowment, 7(13), 1569-1572. https://doi.org/10.14778/2733004.2733032