- Elasticsearch Guide: other versions:
- Getting Started
- Set up Elasticsearch
- Installing Elasticsearch
- Configuring Elasticsearch
- Important Elasticsearch configuration
- Important System Configuration
- Bootstrap Checks
- Heap size check
- File descriptor check
- Memory lock check
- Maximum number of threads check
- Maximum size virtual memory check
- Max file size check
- Maximum map count check
- Client JVM check
- Use serial collector check
- System call filter check
- OnError and OnOutOfMemoryError checks
- Early-access check
- G1GC check
- Stopping Elasticsearch
- Upgrade Elasticsearch
- Set up X-Pack
- Breaking changes
- Breaking changes in 6.0
- Aggregations changes
- Analysis changes
- Cat API changes
- Clients changes
- Cluster changes
- Document API changes
- Indices changes
- Ingest changes
- Java API changes
- Mapping changes
- Packaging changes
- Percolator changes
- Plugins changes
- Reindex changes
- REST changes
- Scripting changes
- Search and Query DSL changes
- Settings changes
- Stats and info changes
- Breaking changes in 6.1
- Breaking changes in 6.0
- X-Pack Breaking Changes
- API Conventions
- Document APIs
- Search APIs
- Aggregations
- Metrics Aggregations
- Avg Aggregation
- Cardinality Aggregation
- Extended Stats Aggregation
- Geo Bounds Aggregation
- Geo Centroid Aggregation
- Max Aggregation
- Min Aggregation
- Percentiles Aggregation
- Percentile Ranks Aggregation
- Scripted Metric Aggregation
- Stats Aggregation
- Sum Aggregation
- Top Hits Aggregation
- Value Count Aggregation
- Bucket Aggregations
- Adjacency Matrix Aggregation
- Children Aggregation
- Composite Aggregation
- Date Histogram Aggregation
- Date Range Aggregation
- Diversified Sampler Aggregation
- Filter Aggregation
- Filters Aggregation
- Geo Distance Aggregation
- GeoHash grid Aggregation
- Global Aggregation
- Histogram Aggregation
- IP Range Aggregation
- Missing Aggregation
- Nested Aggregation
- Range Aggregation
- Reverse nested Aggregation
- Sampler Aggregation
- Significant Terms Aggregation
- Significant Text Aggregation
- Terms Aggregation
- Pipeline Aggregations
- Avg Bucket Aggregation
- Derivative Aggregation
- Max Bucket Aggregation
- Min Bucket Aggregation
- Sum Bucket Aggregation
- Stats Bucket Aggregation
- Extended Stats Bucket Aggregation
- Percentiles Bucket Aggregation
- Moving Average Aggregation
- Cumulative Sum Aggregation
- Bucket Script Aggregation
- Bucket Selector Aggregation
- Bucket Sort Aggregation
- Serial Differencing Aggregation
- Matrix Aggregations
- Caching heavy aggregations
- Returning only aggregation results
- Aggregation Metadata
- Returning the type of the aggregation
- Metrics Aggregations
- Indices APIs
- Create Index
- Delete Index
- Get Index
- Indices Exists
- Open / Close Index API
- Shrink Index
- Split Index
- Rollover Index
- Put Mapping
- Get Mapping
- Get Field Mapping
- Types Exists
- Index Aliases
- Update Indices Settings
- Get Settings
- Analyze
- Index Templates
- Indices Stats
- Indices Segments
- Indices Recovery
- Indices Shard Stores
- Clear Cache
- Flush
- Refresh
- Force Merge
- cat APIs
- Cluster APIs
- Query DSL
- Mapping
- Analysis
- Anatomy of an analyzer
- Testing analyzers
- Analyzers
- Normalizers
- Tokenizers
- Token Filters
- Standard Token Filter
- ASCII Folding Token Filter
- Flatten Graph Token Filter
- Length Token Filter
- Lowercase Token Filter
- Uppercase Token Filter
- NGram Token Filter
- Edge NGram Token Filter
- Porter Stem Token Filter
- Shingle Token Filter
- Stop Token Filter
- Word Delimiter Token Filter
- Word Delimiter Graph Token Filter
- Stemmer Token Filter
- Stemmer Override Token Filter
- Keyword Marker Token Filter
- Keyword Repeat Token Filter
- KStem Token Filter
- Snowball Token Filter
- Phonetic Token Filter
- Synonym Token Filter
- Synonym Graph Token Filter
- Compound Word Token Filters
- Reverse Token Filter
- Elision Token Filter
- Truncate Token Filter
- Unique Token Filter
- Pattern Capture Token Filter
- Pattern Replace Token Filter
- Trim Token Filter
- Limit Token Count Token Filter
- Hunspell Token Filter
- Common Grams Token Filter
- Normalization Token Filter
- CJK Width Token Filter
- CJK Bigram Token Filter
- Delimited Payload Token Filter
- Keep Words Token Filter
- Keep Types Token Filter
- Classic Token Filter
- Apostrophe Token Filter
- Decimal Digit Token Filter
- Fingerprint Token Filter
- Minhash Token Filter
- Character Filters
- Modules
- Index Modules
- Ingest Node
- Pipeline Definition
- Ingest APIs
- Accessing Data in Pipelines
- Handling Failures in Pipelines
- Processors
- Append Processor
- Convert Processor
- Date Processor
- Date Index Name Processor
- Fail Processor
- Foreach Processor
- Grok Processor
- Gsub Processor
- Join Processor
- JSON Processor
- KV Processor
- Lowercase Processor
- Remove Processor
- Rename Processor
- Script Processor
- Set Processor
- Split Processor
- Sort Processor
- Trim Processor
- Uppercase Processor
- Dot Expander Processor
- URL Decode Processor
- Monitoring Elasticsearch
- X-Pack APIs
- Info API
- Explore API
- Machine Learning APIs
- Close Jobs
- Create Datafeeds
- Create Jobs
- Delete Datafeeds
- Delete Jobs
- Delete Model Snapshots
- Flush Jobs
- Forecast Jobs
- Get Buckets
- Get Overall Buckets
- Get Categories
- Get Datafeeds
- Get Datafeed Statistics
- Get Influencers
- Get Jobs
- Get Job Statistics
- Get Model Snapshots
- Get Records
- Open Jobs
- Post Data to Jobs
- Preview Datafeeds
- Revert Model Snapshots
- Start Datafeeds
- Stop Datafeeds
- Update Datafeeds
- Update Jobs
- Update Model Snapshots
- Security APIs
- Watcher APIs
- Migration APIs
- Deprecation Info APIs
- Definitions
- X-Pack Commands
- How To
- Testing
- Glossary of terms
- Release Notes
- 6.1.4 Release Notes
- 6.1.3 Release Notes
- 6.1.2 Release Notes
- 6.1.1 Release Notes
- 6.1.0 Release Notes
- 6.0.1 Release Notes
- 6.0.0 Release Notes
- 6.0.0-rc2 Release Notes
- 6.0.0-rc1 Release Notes
- 6.0.0-beta2 Release Notes
- 6.0.0-beta1 Release Notes
- 6.0.0-alpha2 Release Notes
- 6.0.0-alpha1 Release Notes
- 6.0.0-alpha1 Release Notes (Changes previously released in 5.x)
- X-Pack Release Notes
WARNING: Version 6.1 of Elasticsearch has passed its EOL date.
This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.
Monitoring is enabled by default when you install X-Pack. You can configure
these monitoring settings in the elasticsearch.yml
file.
To adjust how monitoring data is displayed in the monitoring UI, configure
xpack.monitoring
settings in
kibana.yml
. To control how monitoring data is collected from
Logstash, configure
xpack.monitoring
settings
in logstash.yml
.
For more information, see Monitoring the Elastic Stack.
-
xpack.monitoring.enabled
-
Set to
false
to disable Elasticsearch X-Pack monitoring for Elasticsearch on the node.
The xpack.monitoring.collection
settings control how data is collected from
your Elasticsearch nodes.
-
xpack.monitoring.collection.cluster.stats.timeout
-
Sets the timeout for collecting the cluster statistics. Defaults to
10s
. -
xpack.monitoring.collection.indices
-
Controls which indices Monitoring collects data from. Defaults to all indices. Specify the index names as a comma-separated list, for example
test1,test2,test3
. Names can include wildcards, for exampletest*
. You can explicitly include or exclude indices by prepending+
to include the index, or-
to exclude the index. For example, to include all indices that start withtest
excepttest3
, you could specify+test*,-test3
.You can update this setting through the Cluster Update Settings API.
-
xpack.monitoring.collection.index.stats.timeout
-
Sets the timeout for collecting index statistics. Defaults to
10s
. -
xpack.monitoring.collection.indices.stats.timeout
-
Sets the timeout for collecting total indices statistics. Defaults to
10s
. -
xpack.monitoring.collection.index.recovery.active_only
-
Controls whether or not all recoveries are collected. Set to
true
to collect only active recoveries. Defaults tofalse
. -
xpack.monitoring.collection.index.recovery.timeout
-
Sets the timeout for collecting the recovery information. Defaults to
10s
. -
xpack.monitoring.collection.interval
-
Controls how often data samples are collected. Defaults to
10s
. If you modify the collection interval, set thexpack.monitoring.min_interval_seconds
option inkibana.yml
to the same value. Set to-1
to temporarily disable data collection. You can update this setting through the Cluster Update Settings API. -
xpack.monitoring.history.duration
-
Sets the retention duration beyond which the indices created by a Monitoring exporter are automatically deleted. Defaults to
7d
(7 days).This setting has a minimum value of
1d
(1 day) to ensure that something is being monitored, and it cannot be disabled.This setting currently only impacts
local
-type exporters. Indices created using thehttp
exporter will not be deleted automatically.If both X-Pack monitoring and Watcher are enabled, you can use this setting to affect the Watcher cleaner service too. For more information, see the
xpack.watcher.history.cleaner_service.enabled
setting in the Watcher Settings. -
xpack.monitoring.exporters
- Configures where the agent stores monitoring data. By default, the agent uses a local exporter that indexes monitoring data on the cluster where it is installed. Use an HTTP exporter to send data to a separate monitoring cluster. For more information, see Local Exporter Settings, HTTP Exporter Settings, and How Monitoring Works.
The local
exporter is the default exporter used by Monitoring. As the name is
meant to imply, it exports data to the local cluster, which means that there
is not much needed to be configured.
If you do not supply any exporters, then Monitoring will automatically create one for you. If any exporter is provided, then no default is added.
xpack.monitoring.exporters.my_local: type: local
-
type
-
The value for a Local exporter must always be
local
and it is required. -
use_ingest
-
Whether to supply a placeholder pipeline to the cluster and a pipeline processor with
every bulk request. The default value is
true
. If disabled, then it means that it will not use pipelines, which means that a future release cannot automatically upgrade bulk requests to future-proof them. -
cluster_alerts.management.enabled
-
Whether to create cluster alerts for this cluster. The default value is
true
. To use this feature, Watcher must be enabled. If you have a basic license, cluster alerts are not displayed.
The following lists settings that can be supplied with the http
exporter.
All settings are shown as what follows the name you select for your exporter:
xpack.monitoring.exporters.my_remote: type: http host: ["host:port", ...]
-
type
-
The value for an HTTP exporter must always be
http
and it is required. -
host
-
Host supports multiple formats, both as an array or as a single value. Supported formats include
hostname
,hostname:port
,http://hostname
http://hostname:port
,https://hostname
, andhttps://hostname:port
. Hosts cannot be assumed. The default scheme is alwayshttp
and the default port is always9200
if not supplied as part of thehost
string.xpack.monitoring.exporters: example1: type: http host: "10.1.2.3" example2: type: http host: ["http://10.1.2.4"] example3: type: http host: ["10.1.2.5", "10.1.2.6"] example4: type: http host: ["https://10.1.2.3:9200"]
-
auth.username
-
The username is required if a
auth.password
is supplied. -
auth.password
-
The password for the
auth.username
. -
connection.timeout
-
The amount of time that the HTTP connection is supposed to wait for a socket to open for the
request. The default value is
6s
. -
connection.read_timeout
-
The amount of time that the HTTP connection is supposed to wait for a socket to
send back a response. The default value is
10 * connection.timeout
(60s
if neither are set). -
ssl
- Each HTTP exporter can define its own TLS / SSL settings or inherit them. See the TLS / SSL section below.
-
proxy.base_path
-
The base path to prefix any outgoing request, such as
/base/path
(e.g., bulk requests would then be sent as/base/path/_bulk
). There is no default value. -
headers
-
Optional headers that are added to every request, which can assist with routing requests through proxies.
xpack.monitoring.exporters.my_remote: headers: X-My-Array: [abc, def, xyz] X-My-Header: abc123
Array-based headers are sent
n
times wheren
is the size of the array.Content-Type
andContent-Length
cannot be set. Any headers created by the Monitoring agent will override anything defined here. -
index.name.time_format
-
A mechanism for changing the default date suffix for the, by default, daily Monitoring indices.
The default value is
YYYY.MM.DD
, which is why the indices are created daily. -
use_ingest
-
Whether to supply a placeholder pipeline to the monitoring cluster and a pipeline processor with
every bulk request. The default value is
true
. If disabled, then it means that it will not use pipelines, which means that a future release cannot automatically upgrade bulk requests to future-proof them. -
cluster_alerts.management.enabled
-
Whether to create cluster alerts for this cluster. The default value is
true
. To use this feature, Watcher must be enabled. If you have a basic license, cluster alerts are not displayed. -
cluster_alerts.management.blacklist
-
Prevents the creation of specific cluster alerts. It also removes any applicable watches that already exist in the current cluster.
You can add any of the following watch identifiers to the blacklist:
-
elasticsearch_cluster_status
-
elasticsearch_version_mismatch
-
kibana_version_mismatch
-
logstash_version_mismatch
-
xpack_license_expiration
For example:
["elasticsearch_version_mismatch","xpack_license_expiration"]
. -
You can configure the following TLS/SSL settings. If the settings are not configured, the Default TLS/SSL Settings are used.
-
xpack.monitoring.exporters.$NAME.ssl.supported_protocols
-
Supported protocols with versions. Valid protocols:
SSLv2Hello
,SSLv3
,TLSv1
,TLSv1.1
,TLSv1.2
. Defaults toTLSv1.2
,TLSv1.1
,TLSv1
. Defaults to the value ofxpack.ssl.supported_protocols
. -
xpack.monitoring.exporters.$NAME.ssl.verification_mode
-
Controls the verification of certificates. Valid values are
none
,certificate
, andfull
. Defaults to the value ofxpack.ssl.verification_mode
. -
xpack.monitoring.exporters.$NAME.ssl.cipher_suites
-
Supported cipher suites can be found in Oracle’s
Java Cryptography Architecture documentation. Defaults to the value of
xpack.ssl.cipher_suites
.
The following settings are used to specify a private key, certificate, and the trusted certificates that should be used when communicating over an SSL/TLS connection. If none of the settings below are specified, the Default TLS/SSL Settings are used. A private key and certificate are optional and would be used if the server requires client authentication for PKI authentication. If none of the settings below are specified, the Default TLS/SSL Settings are used.
When using PEM encoded files, use the following settings:
-
xpack.monitoring.exporters.$NAME.ssl.key
- Path to a PEM encoded file containing the private key.
-
xpack.monitoring.exporters.$NAME.ssl.key_passphrase
- The passphrase that is used to decrypt the private key. This value is optional as the key might not be encrypted.
-
xpack.monitoring.exporters.$NAME.ssl.secure_key_passphrase
(Secure) - The passphrase that is used to decrypt the private key. This value is optional as the key might not be encrypted.
-
xpack.monitoring.exporters.$NAME.ssl.certificate
- Path to a PEM encoded file containing the certificate (or certificate chain) that will be presented when requested.
-
xpack.monitoring.exporters.$NAME.ssl.certificate_authorities
- List of paths to the PEM encoded certificate files that should be trusted.
When using Java keystore files (JKS), which contain the private key, certificate and certificates that should be trusted, use the following settings:
-
xpack.monitoring.exporters.$NAME.ssl.keystore.path
- Path to the keystore that holds the private key and certificate.
-
xpack.monitoring.exporters.$NAME.ssl.keystore.password
- Password to the keystore.
- +xpack.monitoring.exporters.$NAME.ssl.keystore.secure_password` (Secure)
- Password to the keystore.
-
xpack.monitoring.exporters.$NAME.ssl.keystore.key_password
-
Password for the private key in the keystore. Defaults to the
same value as
xpack.monitoring.exporters.$NAME.ssl.keystore.password
. -
xpack.monitoring.exporters.$NAME.ssl.keystore.secure_key_password
(Secure) - Password for the private key in the keystore.
-
xpack.monitoring.exporters.$NAME.ssl.truststore.path
- Path to the truststore file.
-
xpack.monitoring.exporters.$NAME.ssl.truststore.password
- Password to the truststore.
-
xpack.monitoring.exporters.$NAME.ssl.truststore.secure_password
(Secure) - Password to the truststore.
X-Pack security can be configured to use PKCS#12 container files (.p12
or .pfx
files)
that contain the private key, certificate and certificates that should be trusted.
PKCS#12 files are configured in the same way as Java Keystore Files:
-
xpack.monitoring.exporters.$NAME.ssl.keystore.path
- Path to the PKCS#12 file that holds the private key and certificate.
-
xpack.monitoring.exporters.$NAME.ssl.keystore.type
-
Set this to
PKCS12
to indicate that the keystore is a PKCS#12 file. -
xpack.monitoring.exporters.$NAME.ssl.keystore.password
- Password to the PKCS#12 file.
-
xpack.monitoring.exporters.$NAME.ssl.keystore.secure_password
(Secure) - Password to the PKCS#12 file.
-
xpack.monitoring.exporters.$NAME.ssl.keystore.key_password
-
Password for the private key stored in the PKCS#12 file.
Defaults to the same value as
xpack.monitoring.exporters.$NAME.ssl.keystore.password
. -
xpack.monitoring.exporters.$NAME.ssl.keystore.secure_key_password
(Secure) - Password for the private key stored in the PKCS#12 file.
-
xpack.monitoring.exporters.$NAME.ssl.truststore.path
- Path to the PKCS#12 file that holds the certificates to be trusted.
-
xpack.monitoring.exporters.$NAME.ssl.truststore.type
-
Set this to
PKCS12
to indicate that the truststore is a PKCS#12 file. -
xpack.monitoring.exporters.$NAME.ssl.truststore.password
- Password to the PKCS#12 file.
-
xpack.monitoring.exporters.$NAME.ssl.truststore.secure_password
(Secure) - Password to the PKCS#12 file.
On this page