Development of Kafka Messaging System and its Performance Test Framework using Prometheus
Tejas V1, Kiran V2
1Tejas V*, department of Electronics and Communication Engineering, R V College of Engineering. Bengaluru, India.
2Dr. Kiran V, department of Electronics and Communication Engineering, R V College of Engineering. Bengaluru, India.
Manuscript received on April 30, 2020. | Revised Manuscript received on May 06, 2020. | Manuscript published on May 30, 2020. | PP: 1622-1626 | Volume-9 Issue-1, May 2020. | Retrieval Number: A2516059120/2020©BEIESP | DOI: 10.35940/ijrte.A2516.059120
Open Access | Ethics and Policies | Cite | Mendeley
© The Authors. Blue Eyes Intelligence Engineering and Sciences Publication (BEIESP). This is an open access article under the CC-BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)
Abstract: In today’s era of internet, communication plays a vital role in every aspect of daily activity, as the number of internet users increases an enormous volume of data is generated. This motivates to develop and use a messaging system that collects and analyzes large volume of generated data. Kafka is a distributed messaging system based on publish-subscribe model with high throughput and fault-tolerant mechanism. Around 40% of fortune 500 companies use Kafka messaging system for processing data. Kafka is used to stream real-time data and it is also used to process the data received from IOT sensors. This paper discusses about the procedure for developing and deploying the Kafka messaging system into the working environment based on container technology. The secure communication is enabled by configuring the Kafka with two security mechanisms. The performance of the developed system is evaluated considering QoS parameters for different test scenarios using Prometheus tool.
Keywords: Communication, data, Internet of Things, Kafka, QoS parameter.
Scope of the Article: Internet of Things