Big data analytics concepts and management techniques

Tarek Elarabi, Bhanu Sharma, Karan Pahwa, Vishal Deep

Research output: Chapter in Book/Report/Conference proceedingConference contribution

3 Citations (Scopus)

Abstract

Emergence of big data sources at a rapid rate extends the necessity of old data organization due to the large volume, velocity, variety, value, and veracity of this data. Performing timely analysis on huge datasets is the central promise of big data analytics. The frameworks used to compose analytics jobs into a Directed Acyclic Graphs of small tasks, and then aggregate the intermediate results from the tasks to obtain the final result, does so with the help of a scheduler and a reliable storage layer that distributes the datasets on different machines. This paper presents the above two aspects, scheduling and storage, describe their key principles, and how these principles are realized in widely-deployed systems. The aim of this research project is to design a novel memory management for in-memory databases. Special designed hardware architectures can support the memory management of the host processor. This paper also explores how special designed hardware architectures can upkeep and fast-track data attainment, data straining and data investigation.

Original languageEnglish (US)
Title of host publicationProceedings of the International Conference on Inventive Computation Technologies, ICICT 2016
PublisherInstitute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)9781509012855
DOIs
StatePublished - Jul 2 2016
Event2016 International Conference on Inventive Computation Technologies, ICICT 2016 - Coimbatore, India
Duration: Aug 26 2016Aug 27 2016

Publication series

NameProceedings of the International Conference on Inventive Computation Technologies, ICICT 2016
Volume2

Other

Other2016 International Conference on Inventive Computation Technologies, ICICT 2016
CountryIndia
CityCoimbatore
Period8/26/168/27/16

Fingerprint

Data storage equipment
Information Storage and Retrieval
Computer hardware
Scheduling
Databases
Hardware
Research
Big data
Datasets

All Science Journal Classification (ASJC) codes

  • Computer Networks and Communications
  • Computer Science Applications
  • Computer Vision and Pattern Recognition
  • Health Informatics
  • Artificial Intelligence
  • Computer Graphics and Computer-Aided Design

Cite this

Elarabi, T., Sharma, B., Pahwa, K., & Deep, V. (2016). Big data analytics concepts and management techniques. In Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016 [7824813] (Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016; Vol. 2). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/INVENTIVE.2016.7824813
Elarabi, Tarek ; Sharma, Bhanu ; Pahwa, Karan ; Deep, Vishal. / Big data analytics concepts and management techniques. Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016. Institute of Electrical and Electronics Engineers Inc., 2016. (Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016).
@inproceedings{20f838d6cfe5419c866f4840dcceb2c3,
title = "Big data analytics concepts and management techniques",
abstract = "Emergence of big data sources at a rapid rate extends the necessity of old data organization due to the large volume, velocity, variety, value, and veracity of this data. Performing timely analysis on huge datasets is the central promise of big data analytics. The frameworks used to compose analytics jobs into a Directed Acyclic Graphs of small tasks, and then aggregate the intermediate results from the tasks to obtain the final result, does so with the help of a scheduler and a reliable storage layer that distributes the datasets on different machines. This paper presents the above two aspects, scheduling and storage, describe their key principles, and how these principles are realized in widely-deployed systems. The aim of this research project is to design a novel memory management for in-memory databases. Special designed hardware architectures can support the memory management of the host processor. This paper also explores how special designed hardware architectures can upkeep and fast-track data attainment, data straining and data investigation.",
author = "Tarek Elarabi and Bhanu Sharma and Karan Pahwa and Vishal Deep",
year = "2016",
month = "7",
day = "2",
doi = "10.1109/INVENTIVE.2016.7824813",
language = "English (US)",
series = "Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
booktitle = "Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016",
address = "United States",

}

Elarabi, T, Sharma, B, Pahwa, K & Deep, V 2016, Big data analytics concepts and management techniques. in Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016., 7824813, Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016, vol. 2, Institute of Electrical and Electronics Engineers Inc., 2016 International Conference on Inventive Computation Technologies, ICICT 2016, Coimbatore, India, 8/26/16. https://doi.org/10.1109/INVENTIVE.2016.7824813

Big data analytics concepts and management techniques. / Elarabi, Tarek; Sharma, Bhanu; Pahwa, Karan; Deep, Vishal.

Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016. Institute of Electrical and Electronics Engineers Inc., 2016. 7824813 (Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016; Vol. 2).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Big data analytics concepts and management techniques

AU - Elarabi, Tarek

AU - Sharma, Bhanu

AU - Pahwa, Karan

AU - Deep, Vishal

PY - 2016/7/2

Y1 - 2016/7/2

N2 - Emergence of big data sources at a rapid rate extends the necessity of old data organization due to the large volume, velocity, variety, value, and veracity of this data. Performing timely analysis on huge datasets is the central promise of big data analytics. The frameworks used to compose analytics jobs into a Directed Acyclic Graphs of small tasks, and then aggregate the intermediate results from the tasks to obtain the final result, does so with the help of a scheduler and a reliable storage layer that distributes the datasets on different machines. This paper presents the above two aspects, scheduling and storage, describe their key principles, and how these principles are realized in widely-deployed systems. The aim of this research project is to design a novel memory management for in-memory databases. Special designed hardware architectures can support the memory management of the host processor. This paper also explores how special designed hardware architectures can upkeep and fast-track data attainment, data straining and data investigation.

AB - Emergence of big data sources at a rapid rate extends the necessity of old data organization due to the large volume, velocity, variety, value, and veracity of this data. Performing timely analysis on huge datasets is the central promise of big data analytics. The frameworks used to compose analytics jobs into a Directed Acyclic Graphs of small tasks, and then aggregate the intermediate results from the tasks to obtain the final result, does so with the help of a scheduler and a reliable storage layer that distributes the datasets on different machines. This paper presents the above two aspects, scheduling and storage, describe their key principles, and how these principles are realized in widely-deployed systems. The aim of this research project is to design a novel memory management for in-memory databases. Special designed hardware architectures can support the memory management of the host processor. This paper also explores how special designed hardware architectures can upkeep and fast-track data attainment, data straining and data investigation.

UR - http://www.scopus.com/inward/record.url?scp=85011031769&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85011031769&partnerID=8YFLogxK

U2 - 10.1109/INVENTIVE.2016.7824813

DO - 10.1109/INVENTIVE.2016.7824813

M3 - Conference contribution

AN - SCOPUS:85011031769

T3 - Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016

BT - Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016

PB - Institute of Electrical and Electronics Engineers Inc.

ER -

Elarabi T, Sharma B, Pahwa K, Deep V. Big data analytics concepts and management techniques. In Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016. Institute of Electrical and Electronics Engineers Inc. 2016. 7824813. (Proceedings of the International Conference on Inventive Computation Technologies, ICICT 2016). https://doi.org/10.1109/INVENTIVE.2016.7824813