Distributed Computing Engines for Big Data Analytics
Bh. Prashanthi Griet1, G. Sowjanya Griet2, D. Krishna Madhuri Griet3
1Bh. Prashanthi, Griet, Assistant Professor, Dept of CSE, GRIET.
2G. Sowjanya, Griet, Assistant Professor, Dept of CSE, GRIET.
3D. Krishna Madhuri, Griet, Assistant Professor, Dept of CSE, GRIET.
Manuscript received on 13 March 2019 | Revised Manuscript received on 18 March 2019 | Manuscript published on 30 July 2019 | PP: 5841-5845 | Volume-8 Issue-2, July 2019 | Retrieval Number: B3771078219/19©BEIESP | DOI: 10.35940/ijrte.B3771.078219
Open Access | Ethics and Policies | Cite | Mendeley | Indexing and Abstracting
© The Authors. Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC-BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: Technologies like cloud computing paved way for dealing with massive amounts of data. Prior to cloud, it was not possible unless you invest large amounts for computing resources. Now there is ecosystem which is conducive to storing and processing voluminous data that cannot be handled by local computing resources. With such ecosystem, big data technology came into existence. Big data is the data characterized by volume, velocity, veracity and variety. This has enabled enterprises to give more value to every piece of data. This in turn led to the increased usage of cloud for both storage and processing. For processing big data efficient technologies are required. New programming paradigm like MapReduce with Hadoop distributed programming framework is widely used. However, there are other emerging frameworks like Apache Spark and Apache Flink to handle big data more efficiently. In this paper, empirical study is made on the three frameworks like Hadoop, Apache Spark and Apache Flink with different parameters like type of network, block size of HDFS, input data size and other configuration changes. The experimental results revealed that Apache Spark and Apache Flink outperform Hadoop. This is evaluated with different benchmark big data workloads.
Index Terms: Big Data, Big Data Analytics, Hadoop, Apache Spark, Apache Flink, Distributed Programming Frameworks
Scope of the Article: Big Data Analytics