IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

« Monitoring in a production environment Pausing data collection »

› ›

Configuring monitoring in Elasticsearch

edit

Configuring monitoring in Elasticsearch

edit

If you enable the Elastic monitoring features in your cluster, you can optionally collect metrics about Elasticsearch. By default, monitoring is enabled but data collection is disabled.

Advanced monitoring settings enable you to control how frequently data is collected, configure timeouts, and set the retention period for locally-stored monitoring indices. You can also adjust how monitoring data is displayed.

To collect monitoring data about your Elasticsearch cluster:
1. Verify that the xpack.monitoring.enabled and xpack.monitoring.collection.enabled settings are true on each node in the cluster. By default, data collection is disabled. For more information, see Monitoring Settings.
2. Optional: Specify which indices you want to monitor.
  
  By default, the monitoring agent collects data from all Elasticsearch indices. To collect data from particular indices, configure the xpack.monitoring.collection.indices setting. You can specify multiple indices as a comma-separated list or use an index pattern to match multiple indices. For example:
```
xpack.monitoring.collection.indices: logstash-*, index1, test2
```
  You can prepend + or - to explicitly include or exclude index names or patterns. For example, to include all indices that start with test except test3, you could specify +test*,-test3.
3. Optional: Specify how often to collect monitoring data. The default value for the xpack.monitoring.collection.interval setting 10 seconds. See Monitoring Settings.
Optional: Configure your cluster to route monitoring data from sources such as Kibana, Beats, and Logstash to a monitoring cluster:
1. Verify that xpack.monitoring.collection.enabled settings are true on each node in the cluster.
2. Configure monitoring across the Elastic Stack. For example, see Monitoring in a production environment.
Identify where to store monitoring data.

By default, Elasticsearch monitoring features use a local exporter that indexes monitoring data on the same cluster. See Default exporters and Local Exporters.

Alternatively, you can use an http exporter to send data to a separate monitoring cluster. See HTTP exporters.

The Elasticsearch monitoring features use ingest pipelines, therefore the cluster that stores the monitoring data must have at least one ingest node.

For more information about typical monitoring architectures, see Overview.
If Elasticsearch security features are enabled and you are using an http exporter to send data to a dedicated monitoring cluster:
1. Create a user on the monitoring cluster that has the remote_monitoring_agent built-in role. For example, the following request creates a remote_monitor user that has the remote_monitoring_agent role:
```
POST /_xpack/security/user/remote_monitor
{
  "password" : "changeme",
  "roles" : [ "remote_monitoring_agent"],
  "full_name" : "Internal Agent For Remote Monitoring"
}
```
2. On each node in the cluster that is being monitored, configure the http exporter to use the appropriate credentials when data is shipped to the monitoring cluster.
  
  If SSL/TLS is enabled on the monitoring cluster, you must use the HTTPS protocol in the host setting. You must also include the CA certificate in each node’s trusted certificates in order to verify the identities of the nodes in the monitoring cluster.
  
  The following example specifies the location of the PEM encoded certificate with the certificate_authorities setting:
```
xpack.monitoring.exporters:
  id1:
    type: http
    host: ["https://es-mon1:9200", "https://es-mon2:9200"]
    auth:
      username: remote_monitor 
      password: changeme
    ssl:
      certificate_authorities: [ "/path/to/ca.crt" ]
  id2:
    type: local
```
  The username and password parameters provide the user credentials.
  
  Alternatively, you can configure trusted certificates using a truststore (a Java Keystore file that contains the certificates):
```
xpack.monitoring.exporters:
  id1:
    type: http
    host: ["https://es-mon1:9200", "https://es-mon2:9200"]
    auth:
      username: remote_monitor
      password: changeme
    ssl:
      truststore.path: /path/to/file
      truststore.password: password
  id2:
    type: local
```
If the Elasticsearch security features are enabled and you want to visualize monitoring data in Kibana, you must create users that have access to the Kibana indices and permission to read from the monitoring indices.

You set up Monitoring UI users on the cluster where the monitoring data is stored, that is to say the monitoring cluster. To grant all of the necessary permissions, assign users the monitoring_user and kibana_user roles. For more information, see Mapping users and groups to roles.
Optional: Configure the indices that store the monitoring data.

« Monitoring in a production environment Pausing data collection »