Monitoring settings in Elasticsearch
editMonitoring settings in Elasticsearch
editBy default, monitoring is enabled but data collection is disabled. To enable
data collection, use the xpack.monitoring.collection.enabled
setting.
You can configure these monitoring settings in the elasticsearch.yml
file. You
can also dynamically set some of these settings using the
cluster update settings API.
Cluster settings take precedence over settings in the elasticsearch.yml
file.
To adjust how monitoring data is displayed in the monitoring UI, configure
xpack.monitoring
settings in
kibana.yml
. To control how monitoring data is collected from
Logstash, configure
xpack.monitoring
settings
in logstash.yml
.
For more information, see Monitor a cluster.
General Monitoring Settings
edit-
xpack.monitoring.enabled
-
Set to
true
(default) to enable Elasticsearch X-Pack monitoring for Elasticsearch on the node.To enable data collection, you must also set
xpack.monitoring.collection.enabled
totrue
. Its default value isfalse
.
Monitoring Collection Settings
editThe xpack.monitoring.collection
settings control how data is collected from
your Elasticsearch nodes. You can dynamically change all monitoring collection
settings using the cluster update settings API.
-
xpack.monitoring.collection.enabled
(Dynamic) -
[6.3.0]
Added in 6.3.0.
Set to
true
to enable the collection of monitoring data. When this setting isfalse
(default), Elasticsearch monitoring data is not collected and all monitoring data from other sources such as Kibana, Beats, and Logstash is ignored. -
xpack.monitoring.collection.interval
(Dynamic) -
Setting to
-1
to disable data collection is no longer supported beginning with 7.0.0. [6.3.0] Deprecated in 6.3.0. Usexpack.monitoring.collection.enabled
set tofalse
instead.Controls how often data samples are collected. Defaults to
10s
. If you modify the collection interval, set thexpack.monitoring.min_interval_seconds
option inkibana.yml
to the same value. -
xpack.monitoring.elasticsearch.collection.enabled
(Dynamic) -
Controls whether statistics about your Elasticsearch cluster should be collected. Defaults to
true
. This is different from xpack.monitoring.collection.enabled, which allows you to enable or disable all monitoring collection. However, this setting simply disables the collection of Elasticsearch data while still allowing other data (e.g., Kibana, Logstash, Beats, or APM Server monitoring data) to pass through this cluster. -
xpack.monitoring.collection.cluster.stats.timeout
(Dynamic) -
(time value) Timeout for collecting the cluster statistics. Defaults to
10s
. -
xpack.monitoring.collection.node.stats.timeout
(Dynamic) -
(time value) Timeout for collecting the node statistics. Defaults to
10s
. -
xpack.monitoring.collection.indices
(Dynamic) -
Controls which indices Monitoring collects data from. Defaults to all indices. Specify the index names
as a comma-separated list, for example
test1,test2,test3
. Names can include wildcards, for exampletest*
. You can explicitly exclude indices by prepending-
. For exampletest*,-test3
will monitor all indexes that start withtest
except fortest3
. System indices like .security* or .kibana* always start with a.
, and generally should be monitored. Consider adding.*
to the list of indices ensure monitoring of system indices. For example.*,test*,-test3
-
xpack.monitoring.collection.index.stats.timeout
(Dynamic) -
(time value) Timeout for collecting index statistics. Defaults to
10s
. -
xpack.monitoring.collection.index.recovery.active_only
(Dynamic) -
Controls whether or not all recoveries are collected. Set to
true
to collect only active recoveries. Defaults tofalse
. -
xpack.monitoring.collection.index.recovery.timeout
(Dynamic) -
(time value) Timeout for collecting the recovery information. Defaults to
10s
. -
xpack.monitoring.history.duration
(Dynamic) -
(time value) Retention duration beyond which the indices created by a Monitoring exporter are automatically deleted. Defaults to
7d
(7 days).This setting has a minimum value of
1d
(1 day) to ensure that something is being monitored, and it cannot be disabled.This setting currently only impacts
local
-type exporters. Indices created using thehttp
exporter will not be deleted automatically. -
xpack.monitoring.exporters
- Configures where the agent stores monitoring data. By default, the agent uses a local exporter that indexes monitoring data on the cluster where it is installed. Use an HTTP exporter to send data to a separate monitoring cluster. For more information, see Local exporter settings, HTTP exporter settings, and How it works.
Local Exporter Settings
editThe local
exporter is the default exporter used by Monitoring. As the name is
meant to imply, it exports data to the local cluster, which means that there
is not much needed to be configured.
If you do not supply any exporters, then Monitoring will automatically create one for you. If any exporter is provided, then no default is added.
xpack.monitoring.exporters.my_local: type: local
-
type
-
The value for a Local exporter must always be
local
and it is required. -
use_ingest
-
Whether to supply a placeholder pipeline to the cluster and a pipeline processor with
every bulk request. The default value is
true
. If disabled, then it means that it will not use pipelines, which means that a future release cannot automatically upgrade bulk requests to future-proof them. -
cluster_alerts.management.enabled
-
Whether to create cluster alerts for this cluster. The default value is
true
. To use this feature, Watcher must be enabled. If you have a basic license, cluster alerts are not displayed.
HTTP Exporter Settings
editThe following lists settings that can be supplied with the http
exporter.
All settings are shown as what follows the name you select for your exporter:
xpack.monitoring.exporters.my_remote: type: http host: ["host:port", ...]
-
type
-
The value for an HTTP exporter must always be
http
and it is required. -
host
-
Host supports multiple formats, both as an array or as a single value. Supported formats include
hostname
,hostname:port
,http://hostname
http://hostname:port
,https://hostname
, andhttps://hostname:port
. Hosts cannot be assumed. The default scheme is alwayshttp
and the default port is always9200
if not supplied as part of thehost
string.xpack.monitoring.exporters: example1: type: http host: "10.1.2.3" example2: type: http host: ["http://10.1.2.4"] example3: type: http host: ["10.1.2.5", "10.1.2.6"] example4: type: http host: ["https://10.1.2.3:9200"]
-
auth.username
-
The username is required if a
auth.password
is supplied. -
auth.password
-
The password for the
auth.username
. -
connection.timeout
-
(time value) Amount of time that the HTTP connection is supposed to wait for a socket to open for the
request. The default value is
6s
. -
connection.read_timeout
-
(time value) Amount of time that the HTTP connection is supposed to wait for a socket to
send back a response. The default value is
10 * connection.timeout
(60s
if neither are set). -
ssl
- Each HTTP exporter can define its own TLS / SSL settings or inherit them. See the TLS / SSL section below.
-
proxy.base_path
-
The base path to prefix any outgoing request, such as
/base/path
(e.g., bulk requests would then be sent as/base/path/_bulk
). There is no default value. -
headers
-
Optional headers that are added to every request, which can assist with routing requests through proxies.
xpack.monitoring.exporters.my_remote: headers: X-My-Array: [abc, def, xyz] X-My-Header: abc123
Array-based headers are sent
n
times wheren
is the size of the array.Content-Type
andContent-Length
cannot be set. Any headers created by the Monitoring agent will override anything defined here. -
index.name.time_format
-
A mechanism for changing the default date suffix for the, by default, daily Monitoring indices.
The default value is
yyyy.MM.dd
, which is why the indices are created daily. -
use_ingest
-
Whether to supply a placeholder pipeline to the monitoring cluster and a pipeline processor with
every bulk request. The default value is
true
. If disabled, then it means that it will not use pipelines, which means that a future release cannot automatically upgrade bulk requests to future-proof them. -
cluster_alerts.management.enabled
-
Whether to create cluster alerts for this cluster. The default value is
true
. To use this feature, Watcher must be enabled. If you have a basic license, cluster alerts are not displayed. -
cluster_alerts.management.blacklist
-
Prevents the creation of specific cluster alerts. It also removes any applicable watches that already exist in the current cluster.
You can add any of the following watch identifiers to the blacklist:
-
elasticsearch_cluster_status
-
elasticsearch_version_mismatch
-
elasticsearch_nodes
-
kibana_version_mismatch
-
logstash_version_mismatch
-
xpack_license_expiration
For example:
["elasticsearch_version_mismatch","xpack_license_expiration"]
. -
X-Pack monitoring TLS/SSL settings
editYou can configure the following TLS/SSL settings.
-
xpack.monitoring.exporters.$NAME.ssl.supported_protocols
-
Supported protocols with versions. Valid protocols:
SSLv2Hello
,SSLv3
,TLSv1
,TLSv1.1
,TLSv1.2
,TLSv1.3
. If the JVM’s SSL provider supports TLSv1.3, the default isTLSv1.3,TLSv1.2,TLSv1.1
. Otherwise, the default isTLSv1.2,TLSv1.1
.If
xpack.security.fips_mode.enabled
istrue
, you cannot useSSLv2Hello
orSSLv3
. See FIPS 140-2. -
xpack.monitoring.exporters.$NAME.ssl.verification_mode
-
Controls the verification of certificates. Valid values are:
-
full
, which verifies that the provided certificate is signed by a trusted authority (CA) and also verifies that the server’s hostname (or IP address) matches the names identified within the certificate. -
certificate
, which verifies that the provided certificate is signed by a trusted authority (CA), but does not perform any hostname verification. -
none
, which performs no verification of the server’s certificate. This mode disables many of the security benefits of SSL/TLS and should only be used after very careful consideration. It is primarily intended as a temporary diagnostic mechanism when attempting to resolve TLS errors; its use on production clusters is strongly discouraged.The default value is
full
.
-
-
xpack.monitoring.exporters.$NAME.ssl.cipher_suites
-
Supported cipher suites vary depending on which version of Java you use. For example, for version 11 the default value is
TLS_AES_256_GCM_SHA384
,TLS_AES_128_GCM_SHA256
,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA
,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
,TLS_RSA_WITH_AES_256_GCM_SHA384
,TLS_RSA_WITH_AES_128_GCM_SHA256
,TLS_RSA_WITH_AES_256_CBC_SHA256
,TLS_RSA_WITH_AES_128_CBC_SHA256
,TLS_RSA_WITH_AES_256_CBC_SHA
,TLS_RSA_WITH_AES_128_CBC_SHA
.The default cipher suites list above includes TLSv1.3 ciphers and ciphers that require the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files for 256-bit AES encryption. If TLSv1.3 is not available, the TLSv1.3 ciphers
TLS_AES_256_GCM_SHA384
andTLS_AES_128_GCM_SHA256
are not included in the default list. If 256-bit AES is unavailable, ciphers withAES_256
in their names are not included in the default list. Finally, AES GCM has known performance issues in Java versions prior to 11 and is included in the default list only when using Java 11 or above.For more information, see Oracle’s Java Cryptography Architecture documentation.
X-Pack monitoring TLS/SSL key and trusted certificate settings
editThe following settings are used to specify a private key, certificate, and the trusted certificates that should be used when communicating over an SSL/TLS connection. A private key and certificate are optional and would be used if the server requires client authentication for PKI authentication.
PEM encoded files
editWhen using PEM encoded files, use the following settings:
-
xpack.monitoring.exporters.$NAME.ssl.key
- Path to a PEM encoded file containing the private key.
-
xpack.monitoring.exporters.$NAME.ssl.key_passphrase
- The passphrase that is used to decrypt the private key. Since the key might not be encrypted, this value is optional.
-
xpack.monitoring.exporters.$NAME.ssl.secure_key_passphrase
(Secure) - The passphrase that is used to decrypt the private key. Since the key might not be encrypted, this value is optional.
-
xpack.monitoring.exporters.$NAME.ssl.certificate
- Specifies the path for the PEM encoded certificate (or certificate chain) that is associated with the key.
-
xpack.monitoring.exporters.$NAME.ssl.certificate_authorities
- List of paths to PEM encoded certificate files that should be trusted.
Java keystore files
editWhen using Java keystore files (JKS), which contain the private key, certificate and certificates that should be trusted, use the following settings:
-
xpack.monitoring.exporters.$NAME.ssl.keystore.path
- The path for the keystore file that contains a private key and certificate.
-
xpack.monitoring.exporters.$NAME.ssl.keystore.password
- The password for the keystore.
-
xpack.monitoring.exporters.$NAME.ssl.keystore.secure_password
(Secure) - The password for the keystore.
-
xpack.monitoring.exporters.$NAME.ssl.keystore.key_password
- The password for the key in the keystore. The default is the keystore password.
-
xpack.monitoring.exporters.$NAME.ssl.keystore.secure_key_password
(Secure) - The password for the key in the keystore. The default is the keystore password.
-
xpack.monitoring.exporters.$NAME.ssl.truststore.path
- The path for the keystore that contains the certificates to trust. It must be either a Java keystore (jks) or a PKCS#12 file.
-
xpack.monitoring.exporters.$NAME.ssl.truststore.password
- The password for the truststore.
-
xpack.monitoring.exporters.$NAME.ssl.truststore.secure_password
(Secure) - Password for the truststore.
PKCS#12 files
editElasticsearch can be configured to use PKCS#12 container files (.p12
or .pfx
files)
that contain the private key, certificate and certificates that should be trusted.
PKCS#12 files are configured in the same way as Java keystore files:
-
xpack.monitoring.exporters.$NAME.ssl.keystore.path
- The path for the keystore file that contains a private key and certificate.
-
xpack.monitoring.exporters.$NAME.ssl.keystore.type
-
The format of the keystore file. It must be either
jks
orPKCS12
. If the keystore path ends in ".p12", ".pfx", or ".pkcs12", this setting defaults toPKCS12
. Otherwise, it defaults tojks
. -
xpack.monitoring.exporters.$NAME.ssl.keystore.password
- The password for the keystore.
-
xpack.monitoring.exporters.$NAME.ssl.keystore.secure_password
(Secure) - The password for the keystore.
-
xpack.monitoring.exporters.$NAME.ssl.keystore.key_password
- The password for the key in the keystore. The default is the keystore password.
-
xpack.monitoring.exporters.$NAME.ssl.keystore.secure_key_password
(Secure) - The password for the key in the keystore. The default is the keystore password.
-
xpack.monitoring.exporters.$NAME.ssl.truststore.path
- The path for the keystore that contains the certificates to trust. It must be either a Java keystore (jks) or a PKCS#12 file.
-
xpack.monitoring.exporters.$NAME.ssl.truststore.type
-
Set this to
PKCS12
to indicate that the truststore is a PKCS#12 file. -
xpack.monitoring.exporters.$NAME.ssl.truststore.password
- The password for the truststore.
-
xpack.monitoring.exporters.$NAME.ssl.truststore.secure_password
(Secure) - Password for the truststore.
PKCS#11 tokens
editElasticsearch can be configured to use a PKCS#11 token that contains the private key, certificate and certificates that should be trusted.
PKCS#11 token require additional configuration on the JVM level and can be enabled via the following settings:
-
xpack.monitoring.exporters.$NAME.keystore.type
-
Set this to
PKCS11
to indicate that the PKCS#11 token should be used as a keystore. -
xpack.monitoring.exporters.$NAME.truststore.type
-
The format of the truststore file. For the Java keystore format, use
jks
. For PKCS#12 files, usePKCS12
. For a PKCS#11 token, usePKCS11
. The default isjks
.
When configuring the PKCS#11 token that your JVM is configured to use as
a keystore or a truststore for Elasticsearch, the PIN for the token can be
configured by setting the appropriate value to ssl.truststore.password
or ssl.truststore.secure_password
in the context that you are configuring.
Since there can only be one PKCS#11 token configured, only one keystore and
truststore will be usable for configuration in Elasticsearch. This in turn means
that only one certificate can be used for TLS both in the transport and the
http layer.