Use internal collection to send monitoring data
editUse internal collection to send monitoring data
editUse internal collectors to send Beats monitoring data directly to your monitoring cluster. Or as an alternative to internal collection, use Use Metricbeat collection. The benefit of using internal collection instead of Metricbeat is that you have fewer pieces of software to install and maintain.
-
Create an API key or user that has appropriate authority to send system-level monitoring
data to Elasticsearch. For example, you can use the built-in
apm_system
user or assign the built-inapm_system
role to another user. For more information on the required privileges, see Create a monitoring role. For more information on how to use API keys, see Grant access using API keys. -
Add the
monitoring
settings in the APM Server configuration file. If you configured the Elasticsearch output and want to send APM Server monitoring events to the same Elasticsearch cluster, specify the following minimal configuration:monitoring: enabled: true elasticsearch: api_key: id:api_key username: apm_system password: somepassword
If you want to send monitoring events to an Elastic Cloud monitoring cluster, you can use two simpler settings. When defined, these settings overwrite settings from other parts in the configuration. For example:
monitoring: enabled: true cloud.id: 'staging:dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRjZWM2ZjI2MWE3NGJmMjRjZTMzYmI4ODExYjg0Mjk0ZiRjNmMyY2E2ZDA0MjI0OWFmMGNjN2Q3YTllOTYyNTc0Mw==' cloud.auth: 'elastic:{pwd}'
If you configured a different output, such as Logstash or you want to send APM Server monitoring events to a separate Elasticsearch cluster (referred to as the monitoring cluster), you must specify additional configuration options. For example:
monitoring: enabled: true cluster_uuid: PRODUCTION_ES_CLUSTER_UUID elasticsearch: hosts: ["https://example.com:9200", "https://example2.com:9200"] api_key: id:api_key username: apm_system password: somepassword
This setting identifies the Elasticsearch cluster under which the monitoring data for this APM Server instance will appear in the Stack Monitoring UI. To get a cluster’s
cluster_uuid
, call theGET /
API against that cluster.This setting identifies the hosts and port numbers of Elasticsearch nodes that are part of the monitoring cluster.
Specify one of
api_key
orusername
/password
.If you want to use PKI authentication to send monitoring events to Elasticsearch, you must specify a different set of configuration options. For example:
monitoring: enabled: true cluster_uuid: PRODUCTION_ES_CLUSTER_UUID elasticsearch: hosts: ["https://example.com:9200", "https://example2.com:9200"] username: "" ssl.certificate_authorities: ["/etc/pki/root/ca.pem"] ssl.certificate: "/etc/pki/client/cert.pem" ssl.key: "/etc/pki/client/cert.key"
You must specify the
username
as""
explicitly so that the username from the client certificate (CN
) is used. See SSL/TLS output settings for more information about SSL settings. - Start APM Server.
- View the monitoring data in Kibana.
Settings for internal collection
editUse the following settings to configure internal collection when you are not using Metricbeat to collect monitoring data.
You specify these settings in the X-Pack monitoring section of the
apm-server.yml
config file:
monitoring.enabled
The monitoring.enabled
config is a boolean setting to enable or disable X-Pack monitoring.
If set to true
, monitoring is enabled.
The default value is false
.
monitoring.elasticsearch
The Elasticsearch instances that you want to ship your APM Server metrics to. This configuration option contains the following fields:
api_key
The detail of the API key to be used to send monitoring information to Elasticsearch. See Grant access using API keys for more information.
bulk_max_size
The maximum number of metrics to bulk in a single Elasticsearch bulk API index request.
The default is 50
. For more information, see Elasticsearch.
backoff.init
The number of seconds to wait before trying to reconnect to Elasticsearch after
a network error. After waiting backoff.init
seconds, APM Server tries to
reconnect. If the attempt fails, the backoff timer is increased exponentially up
to backoff.max
. After a successful connection, the backoff timer is reset. The
default is 1s
.
backoff.max
The maximum number of seconds to wait before attempting to connect to
Elasticsearch after a network error. The default is 60s
.
compression_level
The gzip compression level. Setting this value to 0
disables compression. The
compression level must be in the range of 1
(best speed) to 9
(best
compression). The default value is 0
. Increasing the compression level
reduces the network usage but increases the CPU usage.
headers
Custom HTTP headers to add to each request. For more information, see Elasticsearch.
hosts
The list of Elasticsearch nodes to connect to. Monitoring metrics are distributed to these nodes in round robin order. For more information, see Elasticsearch.
max_retries
The number of times to retry sending the monitoring metrics after a failure.
After the specified number of retries, the metrics are typically dropped. The
default value is 3
. For more information, see Elasticsearch.
parameters
Dictionary of HTTP parameters to pass within the URL with index operations.
password
The password that APM Server uses to authenticate with the Elasticsearch instances for shipping monitoring data.
metrics.period
The time interval (in seconds) when metrics are sent to the Elasticsearch cluster. A new snapshot of APM Server metrics is generated and scheduled for publishing each period. The default value is 10 * time.Second.
state.period
The time interval (in seconds) when state information are sent to the Elasticsearch cluster. A new snapshot of APM Server state is generated and scheduled for publishing each period. The default value is 60 * time.Second.
protocol
The name of the protocol to use when connecting to the Elasticsearch cluster. The options
are: http
or https
. The default is http
. If you specify a URL for hosts
,
however, the value of protocol is overridden by the scheme you specify in the URL.
proxy_url
The URL of the proxy to use when connecting to the Elasticsearch cluster. For more information, see Elasticsearch.
timeout
The HTTP request timeout in seconds for the Elasticsearch request. The default is 90
.
ssl
Configuration options for Transport Layer Security (TLS) or Secure Sockets Layer
(SSL) parameters like the certificate authority (CA) to use for HTTPS-based
connections. If the ssl
section is missing, the host CAs are used for
HTTPS connections to Elasticsearch. For more information, see SSL/TLS output settings.
username
The user ID that APM Server uses to authenticate with the Elasticsearch instances for shipping monitoring data.