Fleet and Elastic Agent 8.8.0

Details
Elastic Agent upgrades can sometimes stall without returning an error message, and without the agent upgrade process restarting automatically.

Impact
In this situation the agent returns from Updating to a Healthy state, but without the new version having been installed. To address this, you can trigger a new upgrade manually.

This issue is specific to version 8.8.0 and is resolved in version 8.8.1.

Details
When an internally generated file path exceeds this length it is truncated using a hash, and the newly constructed path might not be accessible to the agent.

To identify the problem, check the output of elastic-agent status --output=yaml or the state.yaml file in a diagnostics bundle for output like the following:

- id: kubernetes/metrics-60f88f50-c873-11ed-9baf-09fb5640c56a
  state:
    state: 4
    message: 'Failed: pid ''3770789'' exited with code ''1'''
    units:
      ? unittype: 1
        unitid: kubernetes/metrics-60f88f50-c873-11ed-9baf-09fb5640c56a
      : state: 4
        message: 'Failed: pid ''3770789'' exited with code ''1'''
      ? unittype: 0
        unitid: kubernetes/metrics-60f88f50-c873-11ed-9baf-09fb5640c56a-kubernetes/metrics-kubelet-0d1f291d-9b2e-4f44-a0dc-82ebee865799
      : state: 4
        message: 'Failed: pid ''3770789'' exited with code ''1'''
      ? unittype: 0
        unitid: kubernetes/metrics-60f88f50-c873-11ed-9baf-09fb5640c56a-kubernetes/metrics-kube-proxy-0d1f291d-9b2e-4f44-a0dc-82ebee865799
      : state: 4
        message: 'Failed: pid ''3770789'' exited with code ''1'''
    features_idx: 0
    version_info:
      name: ""
      version: ""

This is accompanied by an error message in the logs:

logs/elastic-agent-20230530-23.ndjson:{"log.level":"error","@timestamp":"2023-05-30T11:42:46.776Z","message":"Exiting: could not start the HTTP server for the API: listen unix /tmp/elastic-agent/6dd26cab2bb93d6254d75a9ef22c5fb5d3c5ffbd8866f26288d86d2f672d2ae6.sock: bind: no such file or directory","component":{"binary":"metricbeat","dataset":"elastic_agent.metricbeat","id":"kubernetes/metrics-60f88f50-c873-11ed-9baf-08ec5473d24b","type":"kubernetes/metrics"},"log":{"source":"kubernetes/metrics-60e22e52-d872-12dc-4adf-09fb5242c26b"},"log.origin":{"file.line":1142,"file.name":"instance/beat.go"},"service.name":"metricbeat","ecs.version":"1.6.0","ecs.version":"1.6.0"}

Impact

This issue is being investigated. Until it’s resolved, as a workaround you can reduce the length of the agent output name until the problem stops occurring.

Details

When the output server (Elasticsearch or Logstash) is unreachable, versions 8.8.0 & 8.8.1 of Elastic Defend (or Elastic Endpoint) and Elastic Agent may enter a state where they repeatedly communicate with each other indefinitely. This manifests as both processes consuming dramatically more CPU, constantly.

Versions 8.8.0 & 8.8.1 are affected on all operating systems. Elastic Agent does not manifest the behavior unless the Elastic Defend integration is enabled.

Impact

This issue was resolved in version 8.8.2. If you are using Elastic Agent with the Elastic Defend integration, please update to 8.8.2 or later.