IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

« Jolokia jmx metricset Kafka consumergroup metricset »

› ›

Kafka module

edit

Kafka module

edit

This is the Kafka module.

The default metricsets are consumergroup and partition.

If authorization is configured in the Kafka cluster, the following ACLs are required for the Metricbeat user:

READ Topic, for the topics to be monitored
DESCRIBE Group, for the groups to be monitored

For example, if the stats user is being used for Metricbeat, to monitor all topics and all consumer groups, ACLS can be granted with the following commands:

kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:stats --operation Read --topic '*'
kafka-acls --authorizer-properties zookeeper.connect=localhost:2181 --add --allow-principal User:stats --operation Describe --group '*'

Compability

edit

This module is tested with Kafka 0.10.2.1, 1.1.0 and 2.1.1.

Example configuration

edit

The Kafka module supports the standard configuration options that are described in Specify which modules to run. Here is an example configuration:

metricbeat.modules:
- module: kafka
  metricsets: ["consumergroup", "partition"]
  period: 10s
  hosts: ["localhost:9092"]
  enabled: true

  #client_id: metricbeat
  #retries: 3
  #backoff: 250ms

  # List of Topics to query metadata for. If empty, all topics will be queried.
  #topics: []

  # Optional SSL. By default is off.
  # List of root certificates for HTTPS server verifications
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

  # Certificate for SSL client authentication
  #ssl.certificate: "/etc/pki/client/cert.pem"

  # Client Certificate Key
  #ssl.key: "/etc/pki/client/cert.key"

  # SASL authentication
  #username: ""
  #password: ""

Metricsets

edit

The following metricsets are available:

« Jolokia jmx metricset Kafka consumergroup metricset »