An Effective Resource Management in Hadoop Cluster using Optimized Algorithm
bs Vidhyasagar1, J Rajapaulperinbam2, m Krishnamurthy3, j Arunnehru4
1Vidhya Sagar Bs. Assistant Professor in the department of Computer Science and Engineering at SRM Institute of Science and Technology, Vadapalani Campus, Chennai, India.
2DR. J. Raja Paul Perinbam, Professor in Department of Electronics and Communication Engineering. Kings Engineering College, Chennai, India,
3Dr. M Krishnamurthy, Professor in Department of Computer Science and Engineering, KCG College of Technology. Chennai India.
4Dr. J Arunnehru , Assistant Professor in the Department of Computer Science and Engineering at SRM Institute of Science and Technology, Vadapalani Campus, Chennai, India.
Manuscript received on 20 April 2019 | Revised Manuscript received on 26 May 2019 | Manuscript published on 30 May 2019 | PP: 803-809 | Volume-8 Issue-1, May 2019 | Retrieval Number: A9191058119/19©BEIESP
Open Access | Ethics and Policies | Cite | Mendeley | Indexing and Abstracting
© The Authors. Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC-BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: Hadoop advances in executing the massive resources required by applications in a parallel and distributed computing environment, which uses the map-reduce framework to process the large dataset. In Hadoop we use two types of schedulers with YARN capabilities to run the application in big data environment namely Fair Scheduler and Capacitive Scheduler. Each scheduler has it is own queues and resource manager to allocate the resources to run the particular application. In this paper, introduction of PSO based centralized job queuing scheduler is used to manage and monitor the resources that will tune up the existing schedulers which gives the optimized resource utilization, speeds-up the execution and provides more active and dynamic execution of jobs in the big data environment.
Keywords: Hadoop, PSO, Central Queuing, Parallel and Distributed Computing, Slots and YARN.
Scope of the Article: Simulation Optimization and Risk Management