Hints based autodiscover

Metricbeat supports autodiscover based on hints from the provider. The hints system looks for hints in Kubernetes Pod annotations or Docker labels which have the prefix co.elastic.metrics. As soon as the container starts, Metricbeat will check if it contains any hints and launch the proper config for it. Hints tell Metricbeat how to get metrics for the given container. This is the full list of supported hints:

Metricbeat module to use to fetch metrics. See Modules for the list of supported modules.

Hosts setting to use with the given module. Hosts can include ${data.host} and ${data.port} values from the autodiscover event, ie: ${data.host}:80.

List of metricsets to use, comma separated. If no metricsets are provided, default metricsets for the module are used.

The path to retrieve the metrics from (/metrics by default) for Prometheus module.

The time interval for metrics retrieval, ie: 10s

Metrics retrieval timeout, default: 3s

The username to use for authentication

The password to use for authentication. It is recommended to retrieve this sensitive information from an ENV variable and avoid placing passwords in plain text. Unlike static autodiscover configuration, hints based autodiscover has no access to the keystore of Metricbeat since it could be a potential security issue.

SSL parameters, as seen in SSL.

When an entire module configuration needs to be completely set the raw hint can be used. You can provide a stringified JSON of the input configuration. raw overrides every other hint and can be used to create both a single or a list of configurations.

co.elastic.metrics/raw: "[{\"enabled\":true,\"metricsets\":[\"default\"],\"module\":\"mockmoduledefaults\",\"period\":\"1m\",\"timeout\":\"3s\"}]"

Define a processor to be added to the Metricbeat module configuration. See Processors for the list of supported processors.

In order to provide ordering of the processor definition, numbers can be provided. If not, the hints builder will do arbitrary ordering:

co.elastic.logs/processors.1.add_locale.abbrevation: "MST"
co.elastic.logs/processors.add_locale.abbrevation: "PST"

In the above sample the processor definition tagged with 1 would be executed first.

Kubernetes autodiscover provider supports hints in Pod annotations. To enable it just set hints.enabled:

metricbeat.autodiscover:
  providers:
    - type: kubernetes
      hints.enabled: true

This configuration enables the hints autodiscover for Kubernetes. The hints system looks for hints in Kubernetes annotations or Docker labels which have the prefix co.elastic.metrics.

You can annotate Kubernetes Pods with useful info to spin up Metricbeat modules:

annotations:
  co.elastic.metrics/module: prometheus
  co.elastic.metrics/metricsets: collector
  co.elastic.metrics/hosts: '${data.host}:9090'
  co.elastic.metrics/period: 1m

The above annotations are used to spin up a Prometheus collector metricset and it polls the parent container on port 9090 at a 1 minute interval.

When a Pod has multiple containers, these settings are shared. To set hints specific to a container in the pod you can put the container name in the hint.

When a pod has multiple containers, the settings are shared unless you put the container name in the hint. For example, these hints configure a common behavior for all containers in the Pod, and sets a specific hosts hint for the container called sidecar.

annotations:
  co.elastic.metrics/module: apache
  co.elastic.metrics/hosts: '${data.host}:80'
  co.elastic.metrics.sidecar/hosts: '${data.host}:8080'

Hints can be configured on the Namespace’s annotations as defaults to use when Pod level annotations are missing. The resultant hints are a combination of Pod annotations and Namespace annotations with the Pod’s taking precedence. To enable Namespace defaults configure the add_resource_metadata for Namespace objects as follows:

metricbeat.autodiscover:
  providers:
    - type: kubernetes
      hints.enabled: true
      add_resource_metadata:
        namespace:
          enabled: true

Docker autodiscover provider supports hints in labels. To enable it just set hints.enabled:

metricbeat.autodiscover:
  providers:
    - type: docker
      hints.enabled: true

You can label Docker containers with useful info to spin up Metricbeat modules, for example:

  co.elastic.metrics/module: nginx
  co.elastic.metrics/metricsets: stubstatus
  co.elastic.metrics/hosts: '${data.host}:80'
  co.elastic.metrics/period: 10s

The above labels would allow Metricbeat to configure a Prometheus collector to poll port 9090 of the Docker container every 1 minute.