LAST-HDFS: Location-aware storage technique for hadoop distributed file system

Cong Liao, Anna Squicciarini, Dan Lin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Scopus citations

Abstract

Enabled by the state-of-the-art cloud computing technologies, cloud storage has gained increasing popularity in recent years. Despite of the benefit of flexible and reliable data access offered by such services, users have to bear with the fact of not actually knowing the whereabouts of their data. The lack of knowledge and control of the physical locations of data could raise legal and regulatory issues, especially for certain sensitive data that are governed by laws to remain within certain geographic boundaries and borders. In this paper, we study the problem of data placement control within distributed file systems supporting cloud storage. Particularly, we consider the open source Hadoop file system (HDFS) as the underlying architecture, and propose a location-aware cloud storage system, named LAST-HDFS, to support and enforce location-aware storage in HDFS-based clusters. In addition, it also includes a monitoring system deployed at individual hosts to oversee and detect potential data placement violations due to the existence of malicious datanodes. We carried out an extensive experimental evaluation in a real cloud environment that demonstrates the effectiveness and efficiency of our proposed system.

Original languageEnglish (US)
Title of host publicationProceedings - 2016 IEEE 9th International Conference on Cloud Computing, CLOUD 2016
EditorsIan Foster, Nimish Radia, Ian Foster
PublisherIEEE Computer Society
Pages662-669
Number of pages8
ISBN (Electronic)9781509026197
DOIs
StatePublished - Jan 17 2017
Event9th International Conference on Cloud Computing, CLOUD 2016 - San Francisco, United States
Duration: Jun 27 2016Jul 2 2016

Publication series

NameIEEE International Conference on Cloud Computing, CLOUD
ISSN (Print)2159-6182
ISSN (Electronic)2159-6190

Other

Other9th International Conference on Cloud Computing, CLOUD 2016
CountryUnited States
CitySan Francisco
Period6/27/167/2/16

All Science Journal Classification (ASJC) codes

  • Artificial Intelligence
  • Information Systems
  • Software

Fingerprint Dive into the research topics of 'LAST-HDFS: Location-aware storage technique for hadoop distributed file system'. Together they form a unique fingerprint.

  • Cite this

    Liao, C., Squicciarini, A., & Lin, D. (2017). LAST-HDFS: Location-aware storage technique for hadoop distributed file system. In I. Foster, N. Radia, & I. Foster (Eds.), Proceedings - 2016 IEEE 9th International Conference on Cloud Computing, CLOUD 2016 (pp. 662-669). [7820330] (IEEE International Conference on Cloud Computing, CLOUD). IEEE Computer Society. https://doi.org/10.1109/CLOUD.2016.91