IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Monitoring in a production environment Collecting Elasticsearch monitoring data with Metricbeat »
Elastic Docs ›Elasticsearch Guide [8.11] ›Monitor a cluster

Collecting Elasticsearch monitoring data with Elastic Agent

edit

Collecting Elasticsearch monitoring data with Elastic Agent

edit

In 8.5 and later, you can use Elastic Agent to collect data about Elasticsearch and ship it to the monitoring cluster, rather than using Metricbeat or routing it through exporters as described in Legacy collection methods.

Prerequisites

edit

Add Elasticsearch monitoring data

edit

To collect Elasticsearch monitoring data, add an Elasticsearch integration to an Elastic Agent and deploy it to the host where Elasticsearch is running.

  1. Go to the Kibana home page and click Add integrations.
  2. In the query bar, search for and select the Elasticsearch integration for Elastic Agent.
  3. Read the overview to make sure you understand integration requirements and other considerations.

  4. Click Add Elasticsearch.

    If you’re installing an integration for the first time, you may be prompted to install Elastic Agent. Click Add integration only (skip agent installation).

  5. Configure the integration name and optionally add a description. Make sure you configure all required settings:

    1. Under Collect Elasticsearch logs, modify the log paths to match your Elasticsearch environment.
    2. Under Collect Elasticsearch metrics, make sure the hosts setting points to your Elasticsearch host URLs. By default, the integration collects Elasticsearch monitoring metrics from localhost:9200. If that host and port number are not correct, update the hosts setting. If you configured Elasticsearch to use encrypted communications, you must access it via HTTPS. For example, use a hosts setting like https://localhost:9200.
    3. Expand Advanced options. If the Elastic security features are enabled, enter the username and password of a user that has the remote_monitoring_collector role.

    4. Specify the scope:

      • Specify cluster if each entry in the hosts list indicates a single endpoint for a distinct Elasticsearch cluster (for example, a load-balancing proxy fronting the cluster that directs requests to the master-ineligible nodes in the cluster).
      • Otherwise, accept the default scope, node. If this scope is set, you will need to install Elastic Agent on each Elasticsearch node to collect all metrics. Elastic Agent will collect most of the metrics from the elected master of the cluster, so you must scale up all your master-eligible nodes to account for this extra load. Do not use this node if you have dedicated master nodes.
  6. Choose where to add the integration policy. Click New hosts to add it to new agent policy or Existing hosts to add it to an existing agent policy.
  7. Click Save and continue. This step takes a minute or two to complete. When it’s done, you’ll have an agent policy that contains an integration for collecting monitoring data from Elasticsearch.

  8. If an Elastic Agent is already assigned to the policy and deployed to the host where Elasticsearch is running, you’re done. Otherwise, you need to deploy an Elastic Agent. To deploy an Elastic Agent:

    1. Go to Fleet → Agents, then click Add agent.
    2. Follow the steps in the Add agent flyout to download, install, and enroll the Elastic Agent. Make sure you choose the agent policy you created earlier.
  9. Wait a minute or two until incoming data is confirmed.
  10. View the monitoring data in Kibana.
« Monitoring in a production environment Collecting Elasticsearch monitoring data with Metricbeat »