Configuration

Upgrade the Elastic Agent specification

You can upgrade the Elastic Agent version or change settings by editing the YAML specification. ECK applies the changes by performing a rolling restart of the Agent’s Pods. Depending on the settings that you used, ECK will set the outputs part of the configuration, or restart Elastic Agent on certificate rollover.

Customize the Elastic Agent configuration

edit

The Elastic Agent configuration is defined in the config element:

apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
  name: quickstart
spec:
  version: 8.16.1
  elasticsearchRefs:
  - name: quickstart
  daemonSet:
    podTemplate:
      spec:
        securityContext:
          runAsUser: 0 
  config:
    inputs:
      - name: system-1
        revision: 1
        type: system/metrics
        use_output: default
        meta:
          package:
            name: system
            version: 0.9.1
        data_stream:
          namespace: default
        streams:
          - id: system/metrics-system.cpu
            data_stream:
              dataset: system.cpu
              type: metrics
            metricsets:
              - cpu
            cpu.metrics:
              - percentages
              - normalized_percentages
            period: 10s

The root user is required to persist state in a hostPath volume. See Storing local state in host path volume on how to disable this feature.

Alternatively, it can be provided through a Secret specified in the configRef element. The Secret must have an agent.yml entry with this configuration:

apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
  name: quickstart
spec:
  version: 8.16.1
  elasticsearchRefs:
  - name: quickstart
  daemonSet:
    podTemplate:
      spec:
        securityContext:
          runAsUser: 0
  configRef:
    secretName: system-cpu-config
---
apiVersion: v1
kind: Secret
metadata:
  name: system-cpu-config
stringData:
  agent.yml: |-
    inputs:
      - name: system-1
        revision: 1
        type: system/metrics
        use_output: default
        meta:
          package:
            name: system
            version: 0.9.1
        data_stream:
          namespace: default
        streams:
          - id: system/metrics-system.cpu
            data_stream:
              dataset: system.cpu
              type: metrics
            metricsets:
              - cpu
            cpu.metrics:
              - percentages
              - normalized_percentages
            period: 10s

You can use the Fleet application in Kibana to generate the configuration for Elastic Agent, even when running in standalone mode. Check the Elastic Agent standalone documentation. Adding the corresponding integration package to Kibana also adds the related dashboards and visualizations.

Use multiple Elastic Agent outputs

edit

Elastic Agent supports the use of multiple outputs. Therefore, the elasticsearchRefs element accepts multiple references to Elasticsearch clusters. ECK populates the outputs section of the Elastic Agent configuration based on those references. If you configure more than one output, you also have to specify a unique outputName attribute.

To send Elastic Agent’s internal monitoring and log data to a different Elasticsearch cluster called agent-monitoring in the elastic-monitoring namespace, and the harvested metrics to our quickstart cluster, you have to define two elasticsearchRefs as shown in the following example:

apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
  name: quickstart
spec:
  version: 8.16.1
  daemonSet:
    podTemplate:
      spec:
        securityContext:
          runAsUser: 0
  elasticsearchRefs:
  - name: quickstart
    outputName: default
  - name: agent-monitoring
    namespace: elastic-monitoring
    outputName: monitoring
  config:
    agent:
      monitoring:
        enabled: true
        use_output: monitoring
        logs: true
        metrics: true
    inputs:
      - name: system-1
        revision: 1
        type: system/metrics
        use_output: default
...

Customize the connection to an Elasticsearch cluster

edit

The elasticsearchRefs element allows ECK to automatically configure Elastic Agent to establish a secured connection to one or more managed Elasticsearch clusters. By default, it targets all nodes in your cluster. If you want to direct traffic to specific nodes of your Elasticsearch cluster, refer to Traffic Splitting for more information and examples.

Set manually Elastic Agent outputs

edit

If the elasticsearchRefs element is specified, ECK populates the outputs section of the Elastic Agent configuration. ECK creates a user with appropriate roles and permissions and uses its credentials. If required, it also mounts the CA certificate in all Agent Pods, and recreates Pods when this certificate changes.

The outputs can also be set manually. To do that, remove the elasticsearchRefs element from the specification and include an appropriate output configuration in the config, or indirectly through the configRef mechanism.

apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
  name: quickstart
spec:
  version: 8.16.1
  daemonSet:
    podTemplate:
      spec:
        securityContext:
          runAsUser: 0
  config:
    outputs:
      default:
        type: elasticsearch
        hosts:
          - "https://my-custom-elasticsearch-cluster.cloud.elastic.co:9243"
        password: ES_PASSWORD
        username: ES_USER
...

Choose the deployment model

edit

Depending on the use case, Elastic Agent may need to be deployed as a Deployment or a DaemonSet. Provide a podTemplate element under either the deployment or the daemonSet element in the specification to choose how your Elastic Agents should be deployed. When choosing the deployment option you can additionally specify the strategy used to replace old Pods with new ones.

Similarly, you can set the update strategy when deploying as a DaemonSet. This allows you to control the rollout speed for new configuration by modifying the maxUnavailable setting:

apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
  name: quickstart
spec:
  version: 8.16.1
  daemonSet:
    podTemplate:
      spec:
        securityContext:
          runAsUser: 0
    strategy:
      type: RollingUpdate
      rollingUpdate:
        maxUnavailable: 3
...

Check Set compute resources for Beats and Elastic Agent for more information on how to use the Pod template to adjust the resources given to Elastic Agent.

Role Based Access Control for Elastic Agent

edit

Some Elastic Agent features, such as the Kubernetes integration, require that Agent Pods interact with Kubernetes APIs. This functionality requires specific permissions. The standard Kubernetes RBAC rules apply. For example, to allow API interactions:

apiVersion: agent.k8s.elastic.co/v1alpha1
kind: Agent
metadata:
  name: elastic-agent
spec:
  version: 8.16.1
  elasticsearchRefs:
  - name: elasticsearch
  daemonSet:
    podTemplate:
      spec:
        automountServiceAccountToken: true
        serviceAccountName: elastic-agent
        securityContext:
          runAsUser: 0
...
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: elastic-agent
rules:
- apiGroups: [""] # "" indicates the core API group
  resources:
  - namespaces
  - pods
  - nodes
  - nodes/metrics
  - nodes/proxy
  - nodes/stats
  - events
  verbs:
  - get
  - watch
  - list
- nonResourceURLs:
  - /metrics
  verbs:
  - get
  - watch
  - list
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: elastic-agent
  namespace: default
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: elastic-agent
subjects:
- kind: ServiceAccount
  name: elastic-agent
  namespace: default
roleRef:
  kind: ClusterRole
  name: elastic-agent
  apiGroup: rbac.authorization.k8s.io

Deploying Elastic Agent in secured clusters

edit

To deploy Elastic Agent in clusters with the Pod Security Policy admission controller enabled, or in OpenShift clusters, you might need to grant additional permissions to the Service Account used by the Elastic Agent Pods. Those Service Accounts must be bound to a Role or ClusterRole that has use permission for the required Pod Security Policy or Security Context Constraints. Different Elastic Agent integrations might require different settings set in their PSP/SCC.

« Quickstart Configuration examples »