IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

« Monitor the APM Server binary Use the select metrics emitted directly to your monitoring cluster »

› › › ›

Use internal collection to send monitoring data

edit

Use internal collection to send monitoring data

edit

Use internal collectors to send Beats monitoring data directly to your monitoring cluster. Or as an alternative to internal collection, use Use Metricbeat collection. The benefit of using internal collection instead of Metricbeat is that you have fewer pieces of software to install and maintain.

Create an API key or user that has appropriate authority to send system-level monitoring data to Elasticsearch. For example, you can use the built-in apm_system user or assign the built-in apm_system role to another user. For more information on the required privileges, see Create a monitoring user. For more information on how to use API keys, see Grant access using API keys.

Add the monitoring settings in the APM Server configuration file. If you configured the Elasticsearch output and want to send APM Server monitoring events to the same Elasticsearch cluster, specify the following minimal configuration:

monitoring:
  enabled: true
  elasticsearch:
    api_key:  id:api_key 
    username: apm_system
    password: somepassword

Specify one of api_key or username/password.

If you want to send monitoring events to an Elastic Cloud monitoring cluster, you can use two simpler settings. When defined, these settings overwrite settings from other parts in the configuration. For example:

monitoring:
  enabled: true
  cloud.id: 'staging:dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRjZWM2ZjI2MWE3NGJmMjRjZTMzYmI4ODExYjg0Mjk0ZiRjNmMyY2E2ZDA0MjI0OWFmMGNjN2Q3YTllOTYyNTc0Mw=='
  cloud.auth: 'elastic:{pwd}'

If you configured a different output, such as Logstash or you want to send APM Server monitoring events to a separate Elasticsearch cluster (referred to as the monitoring cluster), you must specify additional configuration options. For example:

monitoring:
  enabled: true
  cluster_uuid: PRODUCTION_ES_CLUSTER_UUID 
  elasticsearch:
    hosts: ["https://example.com:9200", "https://example2.com:9200"] 
    api_key:  id:api_key 
    username: apm_system
    password: somepassword

	This setting identifies the Elasticsearch cluster under which the monitoring data for this APM Server instance will appear in the Stack Monitoring UI. To get a cluster’s `cluster_uuid`, call the `GET /` API against that cluster.
	This setting identifies the hosts and port numbers of Elasticsearch nodes that are part of the monitoring cluster.
	Specify one of `api_key` or `username`/`password`.

If you want to use PKI authentication to send monitoring events to Elasticsearch, you must specify a different set of configuration options. For example:

monitoring:
  enabled: true
  cluster_uuid: PRODUCTION_ES_CLUSTER_UUID
  elasticsearch:
    hosts: ["https://example.com:9200", "https://example2.com:9200"]
    username: ""
    ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
    ssl.certificate: "/etc/pki/client/cert.pem"
    ssl.key: "/etc/pki/client/cert.key"

You must specify the username as "" explicitly so that the username from the client certificate (CN) is used. See SSL/TLS output settings for more information about SSL settings.

Start APM Server.
View the monitoring data in Kibana.

Settings for internal collection

edit

Use the following settings to configure internal collection when you are not using Metricbeat to collect monitoring data.

You specify these settings in the X-Pack monitoring section of the apm-server.yml config file:

monitoring.enablededit

The monitoring.enabled config is a boolean setting to enable or disable X-Pack monitoring. If set to true, monitoring is enabled.

The default value is false.

monitoring.elasticsearchedit

The Elasticsearch instances that you want to ship your APM Server metrics to. This configuration option contains the following fields:

api_keyedit

The detail of the API key to be used to send monitoring information to Elasticsearch. See Grant access using API keys for more information.

bulk_max_sizeedit

The maximum number of metrics to bulk in a single Elasticsearch bulk API index request. The default is 50. For more information, see Elasticsearch.

backoff.initedit

The number of seconds to wait before trying to reconnect to Elasticsearch after a network error. After waiting backoff.init seconds, APM Server tries to reconnect. If the attempt fails, the backoff timer is increased exponentially up to backoff.max. After a successful connection, the backoff timer is reset. The default is 1s.

backoff.maxedit

The maximum number of seconds to wait before attempting to connect to Elasticsearch after a network error. The default is 60s.

compression_leveledit

The gzip compression level. Setting this value to 0 disables compression. The compression level must be in the range of 1 (best speed) to 9 (best compression). The default value is 0. Increasing the compression level reduces the network usage but increases the CPU usage.

headersedit

Custom HTTP headers to add to each request. For more information, see Elasticsearch.

hostsedit

The list of Elasticsearch nodes to connect to. Monitoring metrics are distributed to these nodes in round robin order. For more information, see Elasticsearch.

max_retriesedit

The number of times to retry sending the monitoring metrics after a failure. After the specified number of retries, the metrics are typically dropped. The default value is 3. For more information, see Elasticsearch.

parametersedit

Dictionary of HTTP parameters to pass within the URL with index operations.

passwordedit

The password that APM Server uses to authenticate with the Elasticsearch instances for shipping monitoring data.

metrics.periodedit

The time interval (in seconds) when metrics are sent to the Elasticsearch cluster. A new snapshot of APM Server metrics is generated and scheduled for publishing each period. The default value is 10 * time.Second.

state.periodedit

The time interval (in seconds) when state information are sent to the Elasticsearch cluster. A new snapshot of APM Server state is generated and scheduled for publishing each period. The default value is 60 * time.Second.

protocoledit

The name of the protocol to use when connecting to the Elasticsearch cluster. The options are: http or https. The default is http. If you specify a URL for hosts, however, the value of protocol is overridden by the scheme you specify in the URL.

proxy_urledit

The URL of the proxy to use when connecting to the Elasticsearch cluster. For more information, see Elasticsearch.

timeoutedit

The HTTP request timeout in seconds for the Elasticsearch request. The default is 90.

ssledit

Configuration options for Transport Layer Security (TLS) or Secure Sockets Layer (SSL) parameters like the certificate authority (CA) to use for HTTPS-based connections. If the ssl section is missing, the host CAs are used for HTTPS connections to Elasticsearch. For more information, see SSL/TLS output settings.

usernameedit

The user ID that APM Server uses to authenticate with the Elasticsearch instances for shipping monitoring data.

« Monitor the APM Server binary Use the select metrics emitted directly to your monitoring cluster »