Dynamic Load Aware Scheduler of Map Reduce Tasks for Cloud Environments
Adepu Sree Lakshmi1, N. Subhash Chandra2, M. Balraju3
1Adepu Sree Lakshmi, Associate Professor, Geethanjali College of Engineering & Technology, Hyderabad (Telangana), India.
2Dr. N. Subhash Chandra, Professor, CVR College of Engineering College, (Telangana), India.
3Dr. M. Balraju, Professor, Swamy Vivekananda Institute of Technology, (T.N), India.
Manuscript received on 11 October 2019 | Revised Manuscript received on 20 October 2019 | Manuscript Published on 02 November 2019 | PP: 510-517 | Volume-8 Issue-2S11 September 2019 | Retrieval Number: B10790982S1119/2019©BEIESP | DOI: 10.35940/ijrte.B1079.0982S1119
Open Access | Editorial and Publishing Policies | Cite | Mendeley | Indexing and Abstracting
© The Authors. Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC-BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: Most of the current day applications are data and compute intensive which led to invention of technologies like Hadoop. Hadoop uses Map Reduce framework for parallel processing of big data applications using the computing resources of multiple nodes. Hadoop is designed for cluster environments and has few limitations when executed in cloud environments. Hadoop on cloud has become a common choice due to its easy establishment of infrastructure and pay as you use model. Hadoop performance on cloud infrastructures is affected by the virtualization overhead of cloud environment. The execution times of Hadoop on cloud can be improved if the virtual resources are effectively used to schedule the tasks by studying the resource usage characteristics of the tasks and resource availability of the nodes. The proposed work is to build a dynamic scheduler for Hadoop framework which can make scheduling decision dynamically based on job resource usage and node load. The results of the proposed work indicate an improvement of up to 23% in execution time of the Hadoop Map Reduce applications.
Keywords: Hadoop, Map Reduce Scheduler, Capacity Scheduler, Load Aware, Big Data.
Scope of the Article: Cloud Computing