Fleet-managed APM Server

edit

This guide will explain how to set up and configure a Fleet-managed APM Server.

Prerequisites
edit

You need Elasticsearch for storing and searching your data, and Kibana for visualizing and managing it. When setting these components up, you need:

  • Elasticsearch cluster and Kibana (version 8.17) with a basic license or higher. Learn how to install the Elastic Stack on your own hardware.
  • Secure, encrypted connection between Kibana and Elasticsearch. For more information, see Start the Elastic Stack with security enabled.
  • Internet connection for Kibana to download integration packages from the Elastic Package Registry. Make sure the Kibana server can connect to https://epr.elastic.co on port 443. If your environment has network traffic restrictions, there are ways to work around this requirement. See Air-gapped environments for more information.
  • Kibana user with All privileges on Fleet and Integrations. Since many Integrations assets are shared across spaces, users need the Kibana privileges in all spaces.
  • In the Elasticsearch configuration, the built-in API key service must be enabled. (xpack.security.authc.api_key.enabled: true)
  • In the Kibana configuration, the saved objects encryption key must be set. Fleet requires this setting in order to save API keys and encrypt them in Kibana. You can either set xpack.encryptedSavedObjects.encryptionKey to an alphanumeric value of at least 32 characters, or run the kibana-encryption-keys command to generate the key.

Example security settings

For testing purposes, you can use the following settings to get started quickly, but make sure you properly secure the Elastic Stack before sending real data.

elasticsearch.yml example:

xpack.security.enabled: true
xpack.security.authc.api_key.enabled: true

kibana.yml example:

elasticsearch.username: "kibana_system" 
xpack.encryptedSavedObjects.encryptionKey: "something_at_least_32_characters"

The password should be stored in the Kibana keystore as described in the Elasticsearch security documentation.

Step 1: Set up Fleet
edit

Use Fleet in Kibana to get APM data into the Elastic Stack. The first time you use Fleet, you’ll need to set it up and add a Fleet Server:

To deploy a self-managed Fleet Server, you install an Elastic Agent and enroll it in an agent policy containing the Fleet Server integration.

You can install only a single Elastic Agent per host, which means you cannot run Fleet Server and another Elastic Agent on the same host unless you deploy a containerized Fleet Server.

  1. In Fleet, open the Settings tab. For more information about these settings, see Fleet settings.
  2. Under Fleet Server hosts, click Edit hosts and specify one or more host URLs your Elastic Agents will use to connect to Fleet Server. For example, https://192.0.2.1:8220, where 192.0.2.1 is the host IP where you will install Fleet Server. Save and apply your settings.

    If the Edit hosts option is grayed out, Fleet Server hosts are configured outside of Fleet. For more information, refer to Fleet settings in Kibana.

  3. In the Elasticsearch hosts field, specify the Elasticsearch URLs where Elastic Agents will send data. For example, https://192.0.2.0:9200. Skip this step if you’ve started the Elastic Stack with security enabled (you cannot change this setting because it’s managed outside of Fleet).
  4. Save and apply the settings.
  5. Click the Agents tab and follow the in-product instructions to add a Fleet server:

    In-product instructions for adding a Fleet Server

Notes:

  • Choose Quick Start if you want Fleet to generate a Fleet Server policy and enrollment token for you. The Fleet Server policy will include a Fleet Server integration plus a system integration for monitoring Elastic Agent. This option generates self-signed certificates and is not recommended for production use cases.
  • Choose Advanced if you want to either:

  • It’s recommended you generate a unique service token for each Fleet Server. For other ways to generate service tokens, see elasticsearch-service-tokens.
  • If you are providing your own certificates:

    • Before running the install command, make sure you replace the values in angle brackets.
    • Note that the URL specified by --url must match the DNS name used to generate the certificate specified by --fleet-server-cert.
  • The install command installs the Elastic Agent as a managed service and enrolls it in a Fleet Server policy. For more Fleet Server commands, see Elastic Agent command reference.

If installation is successful, you’ll see confirmation that Fleet Server connected. Click Continue enrolling Elastic Agent to begin enrolling your agents in Fleet Server.

If you’re unable to add a Fleet-managed agent, click the Agents tab and confirm that the agent running Fleet Server is healthy.

For more information, refer to Fleet Server.

Step 2: Add and configure the APM integration
edit
  1. In Kibana, select Add integrations > Elastic APM.

    Fleet showing APM integration
  2. Click Add Elastic APM.

    Fleet showing APM integration overview
  3. On the Add Elastic APM integration page, define the host and port where APM Server will listen. Make a note of this value—​you’ll need it later.

    Using Docker or Kubernetes? Set the host to 0.0.0.0 to bind to all interfaces.

  4. Under Agent authorization, set a Secret token. This will be used to authorize requests from APM agents to the APM Server. Make a note of this value—​you’ll need it later.
  5. Click Save and continue. This step takes a minute or two to complete. When it’s done, you’ll have an agent policy that contains an APM integration policy for the configuration you just specified.
  6. To view the new policy, click Agent policy 1.

    Fleet showing apm policy

    Any Elastic Agents assigned to this policy will collect APM data from your instrumented services.

Step 3: Install APM agents
edit

APM agents are written in the same language as your service. To monitor a new service, you must install the agent and configure it with a service name, APM Server host, and Secret token.

  • Service name: The APM integration maps an instrumented service’s name–defined in each APM agent’s configuration– to the index that its data is stored in Elasticsearch. Service names are case-insensitive and must be unique. For example, you cannot have a service named Foo and another named foo. Special characters will be removed from service names and replaced with underscores (_).
  • APM Server URL: The host and port that APM Server listens for events on. This should match the host and port defined when setting up the APM integration.
  • Secret token: Authentication method for APM agent and APM Server communication. This should match the secret token defined when setting up the APM integration.

You can edit your APM integration settings if you need to change the APM Server URL or secret token to match your APM agents.

Manually set up and configure the agent with the -javaagent JVM option. No application code change is required, but this requires an application restart. See below for more information on this setup method.

1. Download the APM agent

The first step in getting started with the Elastic APM Java agent is to retrieve a copy of the agent JAR. Java agent releases are published to Maven central. In order to get a copy you can either:

  • download the latest agent or previous releases from Maven central.
  • download with curl:

    curl -o 'elastic-apm-agent.jar' -L 'https://oss.sonatype.org/service/local/artifact/maven/redirect?r=releases&g=co.elastic.apm&a=elastic-apm-agent&v=LATEST'

2. Add -javaagent flag

When starting your application, add the JVM flag -javaagent:/path/to/elastic-apm-agent-<version>.jar

3. Configure

Different application servers have different ways of setting the -javaagent flag and system properties. Start your application (for example a Spring Boot application or other embedded servers) and add the -javaagent JVM flag. Use the -D prefix to configure the agent using system properties:

java -javaagent:/path/to/elastic-apm-agent-<version>.jar -Delastic.apm.service_name=my-cool-service -Delastic.apm.application_packages=org.example,org.another.example -Delastic.apm.server_url=http://127.0.0.1:8200 -jar my-application.jar

Refer to Manual setup with -javaagent flag to learn more.

Alternate setup methods

  • Automatic setup with apm-agent-attach-cli.jar
    Automatically set up the agent without needing to alter the configuration of your JVM or application server. This method requires no changes to application code or JVM options, and allows attaching to a running JVM. Refer to the Java agent documentation for more information on this setup method.
  • Programmatic API setup to self-attach
    Set up the agent with a one-line code change and an extra apm-agent-attach dependency. This method requires no changes to JVM options, and the agent artifact is embedded within the packaged application binary. Refer to the Java agent documentation for more information on this setup method.
Step 4: View your data
edit

Back in Kibana, under Observability, select APM. You should see application performance monitoring data flowing into the Elastic Stack!

Applications UI with data