This documentation contains work-in-progress information for future Elastic Stack and Cloud releases. Use the version selector to view supported release docs. It also contains some Elastic Cloud serverless information. Check out our serverless docs for more details.
« Monitor APM Server Monitor the APM Server binary »
Elastic Docs ›Elastic Observability [master] ›Application and service monitoring ›Application performance monitoring (APM) ›Monitor APM Server

Monitor a Fleet-managed APM Server

edit

Monitor a Fleet-managed APM Server

edit

This guide assumes you are already ingesting APM data into the Elastic Stack.

In 8.0 and later, you can use Metricbeat to collect data about APM Server and ship it to a monitoring cluster. To collect and ship monitoring data:

  1. Configure Elastic Agent to send monitoring data
  2. Install and configure Metricbeat to collect monitoring data
Configure Elastic Agent to send monitoring data
edit

Before you can monitor APM, you must have monitoring data for the Elasticsearch production cluster. To learn how, see Collect Elasticsearch monitoring data with Metricbeat. Alternatively, open the Stack Monitoring app in Kibana and follow the in-product guide.

  1. Enable monitoring of Elastic Agent by adding the following settings to your elastic-agent.yml configuration file:

    agent.monitoring:
  http:
    enabled: true 
    host: localhost 
    port: 6791

    Enable monitoring

    The host to expose logs/metrics on

    The port to expose logs/metrics on

  2. Enroll Elastic Agent

    After editing elastic-agent.yml, you must re-enroll Elastic Agent for the changes to take effect.

    To enroll the Elastic Agent in Fleet:

    elastic-agent enroll --url <string>
                     --enrollment-token <string>
                     [--ca-sha256 <string>]
                     [--certificate-authorities <string>]
                     [--daemon-timeout <duration>]
                     [--delay-enroll]
                     [--elastic-agent-cert <string>]
                     [--elastic-agent-cert-key <string>]
                     [--elastic-agent-cert-key-passphrase <string>]
                     [--force]
                     [--header <strings>]
                     [--help]
                     [--insecure ]
                     [--proxy-disabled]
                     [--proxy-header <strings>]
                     [--proxy-url <string>]
                     [--staging <string>]
                     [--tag <string>]
                     [global-flags]

See the Elastic Agent command reference for more information on the enroll command.

Install and configure Metricbeat to collect monitoring data
edit
  1. Install Metricbeat on the same server as Elastic Agent. To learn how, see Get started with Metricbeat. If you already have Metricbeat installed, skip this step.

  2. Enable the beat-xpack module in Metricbeat.

    For example, to enable the default configuration in the modules.d directory, run the following command, using the correct command syntax for your OS:

    metricbeat modules enable beat-xpack

    For more information, see Configure modules and beat module.

  3. Configure the beat-xpack module in Metricbeat.

    When complete, your modules.d/beat-xpack.yml file should look similar to this:

    - module: beat
  xpack.enabled: true
  period: 10s
  hosts: ["http://localhost:6791"]
  basepath: "/processes/apm-server-default"
  username: remote_monitoring_user
  password: your_password
    1. Do not change the module name or xpack.enabled boolean; these are required for stack monitoring. We recommend accepting the default period for now.

    2. Set the hosts to match the host:port configured in your elastic-agent.yml file. In this example, that’s http://localhost:6791.

      To monitor multiple APM Server instances running in multiple Elastic Agents, specify a list of hosts, for example:

      hosts: ["http://localhost:5066","http://localhost:5067","http://localhost:5068"]

      If you configured Elastic Agent to use encrypted communications, you must access it via HTTPS. For example, use a hosts setting like https://localhost:5066.

    3. APM Server metrics are exposed at /processes/apm-server-default. Add this location as the basepath.

    4. Set the username and password settings as required by your environment. If Elastic security features are enabled, you must provide a username and password so that Metricbeat can collect metrics successfully:

      1. Create a user on the Elasticsearch cluster that has the remote_monitoring_collector built-in role. Alternatively, if it’s available in your environment, use the remote_monitoring_user built-in user.
      2. Add the username and password settings to the beat module configuration file.

  4. Optional: Disable the system module in the Metricbeat.

    By default, the system module is enabled. The information it collects, however, is not shown on the Stack Monitoring page in Kibana. Unless you want to use that information for other purposes, run the following command:

    metricbeat modules disable system

  5. Identify where to send the monitoring data.

    In production environments, you should send your deployment logs and metrics to a dedicated monitoring deployment (referred to as the monitoring cluster). Monitoring indexes logs and metrics into Elasticsearch and these indexes consume storage, memory, and CPU cycles like any other index. By using a separate monitoring deployment, you avoid affecting your other production deployments and can view the logs and metrics even when a production deployment is unavailable.

    For example, specify the Elasticsearch output information in the Metricbeat configuration file (metricbeat.yml):

    output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["http://es-mon-1:9200", "http://es-mon2:9200"] 

  # Optional protocol and basic auth credentials.
  #protocol: "https"
  #api_key:  "id:api_key" 
  #username: "elastic"
  #password: "changeme"

    In this example, the data is stored on a monitoring cluster with nodes es-mon-1 and es-mon-2.

    Specify one of api_key or username/password.

    If you configured the monitoring cluster to use encrypted communications, you must access it via HTTPS. For example, use a hosts setting like https://es-mon-1:9200.

    The Elasticsearch monitoring features use ingest pipelines, therefore the cluster that stores the monitoring data must have at least one ingest node.

    If the Elasticsearch security features are enabled on the monitoring cluster, you must provide a valid user ID and password so that Metricbeat can send metrics successfully:

    1. Create a user on the monitoring cluster that has the remote_monitoring_agent built-in role. Alternatively, if it’s available in your environment, use the remote_monitoring_user built-in user.
    2. Add the username and password settings to the Elasticsearch output information in the Metricbeat configuration file.

    For more information about these configuration options, see Configure the Elasticsearch output.

  6. Start Metricbeat to begin collecting APM monitoring data.
  7. View the monitoring data in Kibana.
« Monitor APM Server Monitor the APM Server binary »