Configuring monitoring in Elasticsearch
editConfiguring monitoring in Elasticsearch
editIf you enable the Elastic monitoring features in your cluster, you can optionally collect metrics about Elasticsearch. By default, monitoring is enabled but data collection is disabled.
Advanced monitoring settings enable you to control how frequently data is collected, configure timeouts, and set the retention period for locally-stored monitoring indices. You can also adjust how monitoring data is displayed.
-
To collect monitoring data about your Elasticsearch cluster:
-
Verify that the
xpack.monitoring.enabled
andxpack.monitoring.collection.enabled
settings aretrue
on each node in the cluster. By default, data collection is disabled. For more information, see Monitoring Settings. -
Optional: Specify which indices you want to monitor.
By default, the monitoring agent collects data from all Elasticsearch indices. To collect data from particular indices, configure the
xpack.monitoring.collection.indices
setting. You can specify multiple indices as a comma-separated list or use an index pattern to match multiple indices. For example:xpack.monitoring.collection.indices: logstash-*, index1, test2
You can prepend
+
or-
to explicitly include or exclude index names or patterns. For example, to include all indices that start withtest
excepttest3
, you could specify+test*,-test3
. -
Optional: Specify how often to collect monitoring data. The default value for
the
xpack.monitoring.collection.interval
setting 10 seconds. See Monitoring Settings.
-
Verify that the
-
Optional: Configure your cluster to route monitoring data from sources such as Kibana, Beats, and Logstash to a monitoring cluster:
-
Verify that
xpack.monitoring.collection.enabled
settings aretrue
on each node in the cluster. - Configure monitoring across the Elastic Stack. For example, see Monitoring in a production environment.
-
Verify that
-
Identify where to store monitoring data.
By default, Elasticsearch monitoring features use a
local
exporter that indexes monitoring data on the same cluster. See Default exporters and Local Exporters.Alternatively, you can use an
http
exporter to send data to a separate monitoring cluster. See HTTP exporters.The Elasticsearch monitoring features use ingest pipelines, therefore the cluster that stores the monitoring data must have at least one ingest node.
For more information about typical monitoring architectures, see Overview.
-
If Elasticsearch security features are enabled and you are using an
http
exporter to send data to a dedicated monitoring cluster:-
Create a user on the monitoring cluster that has the
remote_monitoring_agent
built-in role. For example, the following request creates aremote_monitor
user that has theremote_monitoring_agent
role:POST /_xpack/security/user/remote_monitor { "password" : "changeme", "roles" : [ "remote_monitoring_agent"], "full_name" : "Internal Agent For Remote Monitoring" }
-
On each node in the cluster that is being monitored, configure the
http
exporter to use the appropriate credentials when data is shipped to the monitoring cluster.If SSL/TLS is enabled on the monitoring cluster, you must use the HTTPS protocol in the
host
setting. You must also include the CA certificate in each node’s trusted certificates in order to verify the identities of the nodes in the monitoring cluster.The following example specifies the location of the PEM encoded certificate with the
certificate_authorities
setting:xpack.monitoring.exporters: id1: type: http host: ["https://es-mon1:9200", "https://es-mon2:9200"] auth: username: remote_monitor password: changeme ssl: certificate_authorities: [ "/path/to/ca.crt" ] id2: type: local
Alternatively, you can configure trusted certificates using a truststore (a Java Keystore file that contains the certificates):
xpack.monitoring.exporters: id1: type: http host: ["https://es-mon1:9200", "https://es-mon2:9200"] auth: username: remote_monitor password: changeme ssl: truststore.path: /path/to/file truststore.password: password id2: type: local
-
-
If Elasticsearch security features are enabled and you want to visualize monitoring data in Kibana, you must create users that have access to the Kibana indices and permission to read from the monitoring indices.
You set up Monitoring UI users on the cluster where the monitoring data is stored, that is to say the monitoring cluster. To grant all of the necessary permissions, assign users the
monitoring_user
andkibana_user
roles. For more information, see Mapping users and groups to roles. - Optional: Configure the indices that store the monitoring data.