kafka, promethues, kabana

 https://www.statuspal.io/blog/2023-2-23-practical-introduction-to-prometheus-monitoring-in-2023

https://www.spiceworks.com/tech/data-management/articles/what-is-kafka/

Apache Kafaka is similar to a msg queue like RabbitMQ :

https://www.spiceworks.com/tech/data-management/articles/what-is-kafka/

Kafka API architecture

Producer, Consumer, Streams, and Connector are the four primary APIs of the Apache Kafka Architecture. Let’s go over each one individually:

Producer API: An application may submit a data stream to one or many Kafka topics using the Producer API. 

Consumer API: This one allows applications to manage the stream of data sent to them and subscribe to one or more topics.

Streams API: An application gets the ability to perform the duties of a stream processor thanks to the Streams API. The application receives an input stream from either one or some of the topics, processes it, and then sends the completed stream to the output topics. 

Connector API: The Connector API enables the creation and operation of reusable producers and consumers that link Kafka topics to apps or information systems. For example, links to relational databases may preserve a record of every modification made to tables. 

---------------------------------------------------------------------------

Promethus:

Prometheus is a powerful open-source monitoring system that can collect metrics from various sources and store them in a time-series database. It is widely used in the industry to monitor and alert the health of applications, servers, and other infrastructure components.

Grafana Integration

Prometheus integrates well with Grafana, a popular open-source dashboarding and visualization platform that allows you to create custom dashboards and visualize the collected metrics. To use Grafana with Prometheus, you need to set up a data source that points to your Prometheus server and create a dashboard that displays the desired metrics.

--------------------------------------------------------------------------

 monitor your kafka deployment health.

Kafka integration with Promethus

https://medium.com/@rramiz.rraza/kafka-metrics-integration-with-prometheus-and-grafana-14fe318fbb8b