IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

« Install Elastic Agents in a containerized environment Run Elastic Agent on Kubernetes managed by Fleet »

› › ›

Run Elastic Agent in a container

edit

Run Elastic Agent in a container

edit

You can run Elastic Agent inside of a container — either with Fleet Server or standalone. Docker images for all versions of Elastic Agent are available from the Elastic Docker registry. If you are running in Kubernetes, refer to run Elastic Agent on ECK.

Considerations:

When Elastic Agent runs inside of a container, it cannot be upgraded through Fleet as it expects that the container itself is upgraded.
Enrolling and running an Elastic Agent is usually a two-step process. However, this doesn’t work in a container, so a special subcommand, container, is called. This command allows environment variables to configure all properties, and runs the enroll and run commands as a single command.

Pull the image

edit

There are two images for Elastic Agent, elastic-agent and elastic-agent-complete. The elastic-agent image contains all the binaries for running Beats, while the elastic-agent-complete image contains these binaries plus additional dependencies to run browser monitors through Elastic Synthetics. Refer to Synthetic monitoring via Elastic Agent and Fleet for more information.

Run the docker pull command against the Elastic Docker registry:

docker pull docker.elastic.co/beats/elastic-agent:{version}

If want to run synthetics tests, run the docker pull command to fetch the elastic-agent-complete image:

docker pull docker.elastic.co/beats/elastic-agent-complete:{version}

Elastic Agent container command

edit

The Elastic Agent container command offers a wide variety of options. To see the full list, run:

elastic-agent container -h

Elastic Cloud example

edit

The easiest way to get started is by using an Elastic cluster running on Elastic Cloud.

In Kibana, select Fleet > Fleet Settings, and copy the Fleet Server host URL.
Close the flyout panel and select Enrollment tokens. Find the Agent policy you want to enroll Elastic Agent into, and display and copy the secret token.
Enroll an Elastic Agent running in a container with the following command:

docker run \
  --env FLEET_ENROLL=1 \
  --env FLEET_URL={fleet-server-host-url} \
  --env FLEET_ENROLLMENT_TOKEN={enrollment-token} \
  --rm docker.elastic.co/beats/elastic-agent:{version}

Replace docker.elastic.co/beats/elastic-agent with docker.elastic.co/beats/elastic-agent-complete and use the elastic-agent user instead of root to run Synthetics Browser tests. Synthetics tests cannot run under the root user. Refer to Synthetics Fleet Quickstart for more information.

Self-managed example

edit

If you’re running a self-managed cluster and want to run your own Fleet Server, run the following command, which will spin up Elastic Agent and Fleet Server in a container:

docker run \
  --env FLEET_SERVER_ENABLE=true \
  --env FLEET_SERVER_ELASTICSEARCH_HOST={elasticsearch-host} \ 
  --env FLEET_SERVER_SERVICE_TOKEN={service-token} \ 
  --env FLEET_SERVER_POLICY_ID=fleet-server-policy \ 
  --rm docker.elastic.co/beats/elastic-agent:{version}

	Your cluster’s Elasticsearch host URL
	The Fleet service token — generate one in the Fleet UI if you don’t have one already
	ID of the Fleet Server policy. To learn how to create a policy, refer Create an agent policy without using the UI.

We recommend only having one fleet-server policy.

Docker compose example

edit

Elastic Agent can be run in docker-compose. The example below shows how to enroll an Elastic Agent:

version: "3"
services:
  elastic-agent:
    image: docker.elastic.co/beats/elastic-agent:{version}
    container_name: elastic-agent
    restart: always
    user: root # note, synthetic browser monitors require this set to `elastic-agent`
    environment:
      - FLEET_ENROLLMENT_TOKEN={enrollment-token}
      - FLEET_ENROLL=1
      - FLEET_URL={fleet-server-url}

Need to run Fleet Server as well? Adjust the docker-compose file above by adding these environment variables:

      - FLEET_SERVER_ENABLE=true
      - FLEET_SERVER_ELASTICSEARCH_HOST={elasticsearch-host}
      - FLEET_SERVER_SERVICE_TOKEN={service-token}

Logs

edit

As a container supports only a single version of Elastic Agent, logs and state are stored a bit differently than when running an Elastic Agent outside of a container. The logs can be found under: /usr/share/elastic-agent/state/data/logs/*.

It’s important to note that only the logs from the Elastic Agent process itself are logged to stdout; Subprocess logs are not. Each subprocess writes its own logs to the default directory inside the logs directory:

/usr/share/elastic-agent/state/data/logs/default/*

Running into errors with Fleet Server? Check the fleet-server subprocess logs for more information.

Debugging

edit

A monitoring endpoint can be enabled to expose resource usage and event processing data. The endpoint is compatible with Elastic Agents running in both Fleet mode and Standalone mode.

Enable the monitoring endpoint in elastic-agent.yml on the host where the Elastic Agent is installed. A sample configuration looks like this:

agent.monitoring:
  enabled: true 
  logs: true 
  metrics: true 
  http:
      enabled: true 
      host: localhost 
      port: 6791

	Enable monitoring of running processes.
	Enable log monitoring.
	Enable metrics monitoring.
	Expose Elastic Agent metrics over HTTP. By default, sockets and named pipes are used.
	The hostname, IP address, unix socket, or named pipe that the HTTP endpoint will bind to. When using IP addresses, we recommend only using `localhost`.
	The port that the HTTP endpoint will bind to.

The above configuration exposes a monitoring endpoint at http://localhost:6791/processes.

http://localhost:6791/processes output

{
   "processes":[
      {
         "id":"metricbeat-default",
         "pid":"36923",
         "binary":"metricbeat",
         "source":{
            "kind":"configured",
            "outputs":[
               "default"
            ]
         }
      },
      {
         "id":"filebeat-default-monitoring",
         "pid":"36924",
         "binary":"filebeat",
         "source":{
            "kind":"internal",
            "outputs":[
               "default"
            ]
         }
      },
      {
         "id":"metricbeat-default-monitoring",
         "pid":"36925",
         "binary":"metricbeat",
         "source":{
            "kind":"internal",
            "outputs":[
               "default"
            ]
         }
      }
   ]
}

Each process ID in the /processes output can be accessed for more details.

http://localhost:6791/processes/{process-name} output

{
   "beat":{
      "cpu":{
         "system":{
            "ticks":537,
            "time":{
               "ms":537
            }
         },
         "total":{
            "ticks":795,
            "time":{
               "ms":796
            },
            "value":795
         },
         "user":{
            "ticks":258,
            "time":{
               "ms":259
            }
         }
      },
      "info":{
         "ephemeral_id":"eb7e8025-7496-403f-9f9a-42b20439c737",
         "uptime":{
            "ms":75332
         },
         "version":"7.14.0"
      },
      "memstats":{
         "gc_next":23920624,
         "memory_alloc":20046048,
         "memory_sys":76104712,
         "memory_total":60823368,
         "rss":83165184
      },
      "runtime":{
         "goroutines":58
      }
   },
   "libbeat":{
      "config":{
         "module":{
            "running":4,
            "starts":4,
            "stops":0
         },
         "reloads":1,
         "scans":1
      },
      "output":{
         "events":{
            "acked":0,
            "active":0,
            "batches":0,
            "dropped":0,
            "duplicates":0,
            "failed":0,
            "toomany":0,
            "total":0
         },
         "read":{
            "bytes":0,
            "errors":0
         },
         "type":"elasticsearch",
         "write":{
            "bytes":0,
            "errors":0
         }
      },
      "pipeline":{
         "clients":4,
         "events":{
            "active":231,
            "dropped":0,
            "failed":0,
            "filtered":0,
            "published":231,
            "retry":112,
            "total":231
         },
         "queue":{
            "acked":0,
            "max_events":4096
         }
      }
   },
   "metricbeat":{
      "system":{
         "cpu":{
            "events":8,
            "failures":0,
            "success":8
         },
         "filesystem":{
            "events":80,
            "failures":0,
            "success":80
         },
         "memory":{
            "events":8,
            "failures":0,
            "success":8
         },
         "network":{
            "events":135,
            "failures":0,
            "success":135
         }
      }
   },
   "system":{
      "cpu":{
         "cores":8
      },
      "load":{
         "1":2.5957,
         "15":5.415,
         "5":3.5815,
         "norm":{
            "1":0.3245,
            "15":0.6769,
            "5":0.4477
         }
      }
   }
}

« Install Elastic Agents in a containerized environment Run Elastic Agent on Kubernetes managed by Fleet »