Hints annotations based autodiscover

Standalone Elastic Agent supports autodiscover based on hints from the provider. The hints mechanism looks for hints in Kubernetes Pod annotations that have the prefix co.elastic.hints. As soon as the container starts, Elastic Agent checks it for hints and launches the proper configuration for the container. Hints tell Elastic Agent how to monitor the container by using the proper integration. This is the full list of supported hints:

The package to use for monitoring.

The host to use for metrics retrieval.

The list of data streams to enable. If not specified, the integration’s default data streams are used. To find the defaults, refer to the Elastic integrations documentation.

If data streams are specified, additional hints can be defined per data stream. For example, co.elastic.hints/info.period: 5m if the data stream specified is info for the Redis module.

apiVersion: v1
kind: Pod
metadata:
  name: redis
  annotations:
    co.elastic.hints/package: redis
    co.elastic.hints/data_streams: info
    co.elastic.hints/info.period: 5m

If data stream hints are not specified, the top level hints will be used in its configuration.

The path to retrieve the metrics from.

The time interval for metrics retrieval, for example, 10s.

Metrics retrieval timeout, for example, 3s.

The username to use for authentication.

The password to use for authentication. It is recommended to retrieve this sensitive information from an ENV variable and avoid placing passwords in plain text.

The stream to use for logs collection, for example, stdout/stderr.

If the specified package has no logs support, a generic container’s logs input will be used as a fallback.

The available packages that are supported through hints can be found here.

To enable hints, you must add hints.enabled: true to the provider’s configuration:

providers:
  kubernetes:
    hints.enabled: true

Then ensure that the proper volumes and volumeMounts are specified by uncommenting the appropriate sections in the Elastic Agent manifest:

volumeMounts:
- name: external-inputs
  mountPath: /etc/elastic-agent/inputs.d
...
volumes:
- name: external-inputs
  emptyDir: {}
...

An init container is also required to download the hints templates. The init container is already defined, so uncomment the respective section:

initContainers:
- name: k8s-templates-downloader
  image: busybox:1.28
  command: ['sh']
  args:
    - -c
    - >-
      mkdir -p /etc/elastic-agent/inputs.d &&
      wget -O - https://github.com/elastic/elastic-agent/archive/8.6.tar.gz | tar xz -C /etc/elastic-agent/inputs.d --strip=5 "elastic-agent-8.6/deploy/kubernetes/elastic-agent-standalone/templates.d"
  volumeMounts:
    - name: external-inputs
      mountPath: /etc/elastic-agent/inputs.d

Enabling hints allows users deploying Pods on the cluster to automatically turn on Elastic monitoring at Pod deployment time. For example, to deploy a Redis Pod on the cluster and automatically enable Elastic monitoring, add the proper hints as annotations on the Pod manifest file:

...
apiVersion: v1
kind: Pod
metadata:
  name: redis
  annotations:
    co.elastic.hints/package: redis
    co.elastic.hints/data_streams: info
    co.elastic.hints/host: '${kubernetes.pod.ip}:6379'
    co.elastic.hints/info.period: 5s
  labels:
    k8s-app: redis
    app: redis
...

After deploying this Pod, the data will start flowing in automatically. You can find it on the index metrics-redis.info-default.

When things do not work as expected, you may need to troubleshoot your setup. Here we provide some directions to speed up your investigation: