- Elasticsearch Guide: other versions:
- Elasticsearch introduction
- Getting started with Elasticsearch
- Set up Elasticsearch
- Installing Elasticsearch
- Configuring Elasticsearch
- Important Elasticsearch configuration
- Important System Configuration
- Bootstrap Checks
- Heap size check
- File descriptor check
- Memory lock check
- Maximum number of threads check
- Max file size check
- Maximum size virtual memory check
- Maximum map count check
- Client JVM check
- Use serial collector check
- System call filter check
- OnError and OnOutOfMemoryError checks
- Early-access check
- G1GC check
- All permission check
- Discovery configuration check
- Starting Elasticsearch
- Stopping Elasticsearch
- Adding nodes to your cluster
- Set up X-Pack
- Configuring X-Pack Java Clients
- Bootstrap Checks for X-Pack
- Upgrade Elasticsearch
- API conventions
- Document APIs
- Search APIs
- Aggregations
- Metrics Aggregations
- Avg Aggregation
- Weighted Avg Aggregation
- Cardinality Aggregation
- Extended Stats Aggregation
- Geo Bounds Aggregation
- Geo Centroid Aggregation
- Max Aggregation
- Min Aggregation
- Percentiles Aggregation
- Percentile Ranks Aggregation
- Scripted Metric Aggregation
- Stats Aggregation
- Sum Aggregation
- Top Hits Aggregation
- Value Count Aggregation
- Median Absolute Deviation Aggregation
- Bucket Aggregations
- Adjacency Matrix Aggregation
- Auto-interval Date Histogram Aggregation
- Children Aggregation
- Composite Aggregation
- Date Histogram Aggregation
- Date Range Aggregation
- Diversified Sampler Aggregation
- Filter Aggregation
- Filters Aggregation
- Geo Distance Aggregation
- GeoHash grid Aggregation
- GeoTile Grid Aggregation
- Global Aggregation
- Histogram Aggregation
- IP Range Aggregation
- Missing Aggregation
- Nested Aggregation
- Parent Aggregation
- Range Aggregation
- Reverse nested Aggregation
- Sampler Aggregation
- Significant Terms Aggregation
- Significant Text Aggregation
- Terms Aggregation
- Pipeline Aggregations
- Avg Bucket Aggregation
- Derivative Aggregation
- Max Bucket Aggregation
- Min Bucket Aggregation
- Sum Bucket Aggregation
- Stats Bucket Aggregation
- Extended Stats Bucket Aggregation
- Percentiles Bucket Aggregation
- Moving Average Aggregation
- Moving Function Aggregation
- Cumulative Sum Aggregation
- Bucket Script Aggregation
- Bucket Selector Aggregation
- Bucket Sort Aggregation
- Serial Differencing Aggregation
- Matrix Aggregations
- Caching heavy aggregations
- Returning only aggregation results
- Aggregation Metadata
- Returning the type of the aggregation
- Metrics Aggregations
- Indices APIs
- Create Index
- Delete Index
- Get Index
- Indices Exists
- Open / Close Index API
- Shrink Index
- Split Index
- Rollover Index
- Put Mapping
- Get Mapping
- Get Field Mapping
- Types Exists
- Index Aliases
- Update Indices Settings
- Get Settings
- Analyze
- Index Templates
- Indices Stats
- Indices Segments
- Indices Recovery
- Indices Shard Stores
- Clear Cache
- Flush
- Refresh
- Force Merge
- cat APIs
- Cluster APIs
- Query DSL
- Scripting
- Mapping
- Analysis
- Anatomy of an analyzer
- Testing analyzers
- Analyzers
- Normalizers
- Tokenizers
- Standard Tokenizer
- Letter Tokenizer
- Lowercase Tokenizer
- Whitespace Tokenizer
- UAX URL Email Tokenizer
- Classic Tokenizer
- Thai Tokenizer
- NGram Tokenizer
- Edge NGram Tokenizer
- Keyword Tokenizer
- Pattern Tokenizer
- Char Group Tokenizer
- Simple Pattern Tokenizer
- Simple Pattern Split Tokenizer
- Path Hierarchy Tokenizer
- Path Hierarchy Tokenizer Examples
- Token Filters
- ASCII Folding Token Filter
- Flatten Graph Token Filter
- Length Token Filter
- Lowercase Token Filter
- Uppercase Token Filter
- NGram Token Filter
- Edge NGram Token Filter
- Porter Stem Token Filter
- Shingle Token Filter
- Stop Token Filter
- Word Delimiter Token Filter
- Word Delimiter Graph Token Filter
- Multiplexer Token Filter
- Conditional Token Filter
- Predicate Token Filter Script
- Stemmer Token Filter
- Stemmer Override Token Filter
- Keyword Marker Token Filter
- Keyword Repeat Token Filter
- KStem Token Filter
- Snowball Token Filter
- Phonetic Token Filter
- Synonym Token Filter
- Parsing synonym files
- Synonym Graph Token Filter
- Compound Word Token Filters
- Reverse Token Filter
- Elision Token Filter
- Truncate Token Filter
- Unique Token Filter
- Pattern Capture Token Filter
- Pattern Replace Token Filter
- Trim Token Filter
- Limit Token Count Token Filter
- Hunspell Token Filter
- Common Grams Token Filter
- Normalization Token Filter
- CJK Width Token Filter
- CJK Bigram Token Filter
- Delimited Payload Token Filter
- Keep Words Token Filter
- Keep Types Token Filter
- Exclude mode settings example
- Classic Token Filter
- Apostrophe Token Filter
- Decimal Digit Token Filter
- Fingerprint Token Filter
- MinHash Token Filter
- Remove Duplicates Token Filter
- Character Filters
- Modules
- Index modules
- Ingest node
- Pipeline Definition
- Ingest APIs
- Accessing Data in Pipelines
- Conditional Execution in Pipelines
- Handling Failures in Pipelines
- Processors
- Append Processor
- Bytes Processor
- Convert Processor
- Date Processor
- Date Index Name Processor
- Dissect Processor
- Dot Expander Processor
- Drop Processor
- Fail Processor
- Foreach Processor
- GeoIP Processor
- Grok Processor
- Gsub Processor
- HTML Strip Processor
- Join Processor
- JSON Processor
- KV Processor
- Lowercase Processor
- Pipeline Processor
- Remove Processor
- Rename Processor
- Script Processor
- Set Processor
- Set Security User Processor
- Split Processor
- Sort Processor
- Trim Processor
- Uppercase Processor
- URL Decode Processor
- User Agent processor
- Managing the index lifecycle
- Getting started with index lifecycle management
- Policy phases and actions
- Set up index lifecycle management policy
- Using policies to manage index rollover
- Update policy
- Index lifecycle error handling
- Restoring snapshots of managed indices
- Start and stop index lifecycle management
- Using ILM with existing indices
- SQL access
- Overview
- Getting Started with SQL
- Conventions and Terminology
- Security
- SQL REST API
- SQL Translate API
- SQL CLI
- SQL JDBC
- SQL ODBC
- SQL Client Applications
- SQL Language
- Functions and Operators
- Comparison Operators
- Logical Operators
- Math Operators
- Cast Operators
- LIKE and RLIKE Operators
- Aggregate Functions
- Grouping Functions
- Date/Time and Interval Functions and Operators
- Full-Text Search Functions
- Mathematical Functions
- String Functions
- Type Conversion Functions
- Geo Functions
- Conditional Functions And Expressions
- System Functions
- Reserved keywords
- SQL Limitations
- Monitor a cluster
- Frozen indices
- Set up a cluster for high availability
- Roll up or transform your data
- X-Pack APIs
- Info API
- Cross-cluster replication APIs
- Explore API
- Freeze index
- Index lifecycle management API
- Licensing APIs
- Machine learning APIs
- Add events to calendar
- Add jobs to calendar
- Close jobs
- Create jobs
- Create calendar
- Create datafeeds
- Create filter
- Delete calendar
- Delete datafeeds
- Delete events from calendar
- Delete filter
- Delete forecast
- Delete jobs
- Delete jobs from calendar
- Delete model snapshots
- Delete expired data
- Find file structure
- Flush jobs
- Forecast jobs
- Get calendars
- Get buckets
- Get overall buckets
- Get categories
- Get datafeeds
- Get datafeed statistics
- Get influencers
- Get jobs
- Get job statistics
- Get machine learning info
- Get model snapshots
- Get scheduled events
- Get filters
- Get records
- Open jobs
- Post data to jobs
- Preview datafeeds
- Revert model snapshots
- Set upgrade mode
- Start datafeeds
- Stop datafeeds
- Update datafeeds
- Update filter
- Update jobs
- Update model snapshots
- Migration APIs
- Rollup APIs
- Security APIs
- Authenticate
- Change passwords
- Clear cache
- Clear roles cache
- Create API keys
- Create or update application privileges
- Create or update role mappings
- Create or update roles
- Create or update users
- Delete application privileges
- Delete role mappings
- Delete roles
- Delete users
- Disable users
- Enable users
- Get API key information
- Get application privileges
- Get role mappings
- Get roles
- Get token
- Get users
- Has privileges
- Invalidate API key
- Invalidate token
- OpenID Connect Prepare Authentication API
- OpenID Connect Authenticate API
- OpenID Connect Logout API
- SSL certificate
- Transform APIs
- Unfreeze index
- Watcher APIs
- Definitions
- Secure a cluster
- Overview
- Configuring security
- Encrypting communications in Elasticsearch
- Encrypting communications in an Elasticsearch Docker Container
- Enabling cipher suites for stronger encryption
- Separating node-to-node and client traffic
- Configuring an Active Directory realm
- Configuring a file realm
- Configuring an LDAP realm
- Configuring a native realm
- Configuring a PKI realm
- Configuring a SAML realm
- Configuring a Kerberos realm
- Security files
- FIPS 140-2
- How security works
- User authentication
- Built-in users
- Internal users
- Token-based authentication services
- Realms
- Realm chains
- Active Directory user authentication
- File-based user authentication
- LDAP user authentication
- Native user authentication
- OpenID Connect authentication
- PKI user authentication
- SAML authentication
- Kerberos authentication
- Integrating with other authentication systems
- Enabling anonymous access
- Controlling the user cache
- Configuring SAML single-sign-on on the Elastic Stack
- Configuring single sign-on to the Elastic Stack using OpenID Connect
- User authorization
- Auditing security events
- Encrypting communications
- Restricting connections with IP filtering
- Cross cluster search, clients, and integrations
- Tutorial: Getting started with security
- Tutorial: Encrypting communications
- Troubleshooting
- Some settings are not returned via the nodes settings API
- Authorization exceptions
- Users command fails due to extra arguments
- Users are frequently locked out of Active Directory
- Certificate verification fails for curl on Mac
- SSLHandshakeException causes connections to fail
- Common SSL/TLS exceptions
- Common Kerberos exceptions
- Common SAML issues
- Internal Server Error in Kibana
- Setup-passwords command fails due to connection failure
- Failures due to relocation of the configuration files
- Limitations
- Alerting on cluster and index events
- Command line tools
- How To
- Testing
- Glossary of terms
- Release highlights
- Breaking changes
- Release notes
- Elasticsearch version 7.2.1
- Elasticsearch version 7.2.0
- Elasticsearch version 7.1.1
- Elasticsearch version 7.1.0
- Elasticsearch version 7.0.0
- Elasticsearch version 7.0.0-rc2
- Elasticsearch version 7.0.0-rc1
- Elasticsearch version 7.0.0-beta1
- Elasticsearch version 7.0.0-alpha2
- Elasticsearch version 7.0.0-alpha1
Index modules
editIndex modules
editIndex Modules are modules created per index and control all aspects related to an index.
Index Settings
editIndex level settings can be set per-index. Settings may be:
- static
- They can only be set at index creation time or on a closed index.
- dynamic
- They can be changed on a live index using the update-index-settings API.
Changing static or dynamic index settings on a closed index could result in incorrect settings that are impossible to rectify without deleting and recreating the index.
Static index settings
editBelow is a list of all static index settings that are not associated with any specific index module:
-
index.number_of_shards
-
The number of primary shards that an index should have. Defaults to 1.
This setting can only be set at index creation time. It cannot be
changed on a closed index. Note: the number of shards are limited to
1024
per index. This limitation is a safety limit to prevent accidental creation of indices that can destabilize a cluster due to resource allocation. The limit can be modified by specifyingexport ES_JAVA_OPTS="-Des.index.max_number_of_shards=128"
system property on every node that is part of the cluster. -
index.shard.check_on_startup
-
Whether or not shards should be checked for corruption before opening. When corruption is detected, it will prevent the shard from being opened. Accepts:
-
false
- (default) Don’t check for corruption when opening a shard.
-
checksum
- Check for physical corruption.
-
true
-
Check for both physical and logical corruption. This is much more expensive in terms of CPU and memory usage.
Expert only. Checking shards may take a lot of time on large indices.
-
-
index.codec
-
The
default
value compresses stored data with LZ4 compression, but this can be set tobest_compression
which uses DEFLATE for a higher compression ratio, at the expense of slower stored fields performance. If you are updating the compression type, the new one will be applied after segments are merged. Segment merging can be forced using force merge. -
index.routing_partition_size
-
The number of shards a custom routing value can go to.
Defaults to 1 and can only be set at index creation time. This value must be less
than the
index.number_of_shards
unless theindex.number_of_shards
value is also 1. See Routing to an index partition for more details about how this setting is used. -
index.load_fixed_bitset_filters_eagerly
-
Indicates whether cached filters are pre-loaded for
nested queries. Possible values are
true
(default) andfalse
.
Dynamic index settings
editBelow is a list of all dynamic index settings that are not associated with any specific index module:
-
index.number_of_replicas
- The number of replicas each primary shard has. Defaults to 1.
-
index.auto_expand_replicas
-
Auto-expand the number of replicas based on the number of data nodes in the cluster.
Set to a dash delimited lower and upper bound (e.g.
0-5
) or useall
for the upper bound (e.g.0-all
). Defaults tofalse
(i.e. disabled). Note that the auto-expanded number of replicas does not take any other allocation rules into account, such as shard allocation awareness, filtering or total shards per node, and this can lead to the cluster health becomingYELLOW
if the applicable rules prevent all the replicas from being allocated. -
index.search.idle.after
-
How long a shard can not receive a search or get request until it’s considered
search idle. (default is
30s
) -
index.refresh_interval
-
How often to perform a refresh operation, which makes recent changes to the
index visible to search. Defaults to
1s
. Can be set to-1
to disable refresh. If this setting is not explicitly set, shards that haven’t seen search traffic for at leastindex.search.idle.after
seconds will not receive background refreshes until they receive a search request. Searches that hit an idle shard where a refresh is pending will wait for the next background refresh (within1s
). This behavior aims to automatically optimize bulk indexing in the default case when no searches are performed. In order to opt out of this behavior an explicit value of1s
should set as the refresh interval. -
index.max_result_window
-
The maximum value of
from + size
for searches to this index. Defaults to10000
. Search requests take heap memory and time proportional tofrom + size
and this limits that memory. See Scroll or Search After for a more efficient alternative to raising this. -
index.max_inner_result_window
-
The maximum value of
from + size
for inner hits definition and top hits aggregations to this index. Defaults to100
. Inner hits and top hits aggregation take heap memory and time proportional tofrom + size
and this limits that memory. -
index.max_rescore_window
-
The maximum value of
window_size
forrescore
requests in searches of this index. Defaults toindex.max_result_window
which defaults to10000
. Search requests take heap memory and time proportional tomax(window_size, from + size)
and this limits that memory. -
index.max_docvalue_fields_search
-
The maximum number of
docvalue_fields
that are allowed in a query. Defaults to100
. Doc-value fields are costly since they might incur a per-field per-document seek. -
index.max_script_fields
-
The maximum number of
script_fields
that are allowed in a query. Defaults to32
. -
index.max_ngram_diff
-
The maximum allowed difference between min_gram and max_gram for NGramTokenizer and NGramTokenFilter.
Defaults to
1
. -
index.max_shingle_diff
-
The maximum allowed difference between max_shingle_size and min_shingle_size for ShingleTokenFilter.
Defaults to
3
. -
index.blocks.read_only
-
Set to
true
to make the index and index metadata read only,false
to allow writes and metadata changes. -
index.blocks.read_only_allow_delete
-
Identical to
index.blocks.read_only
but allows deleting the index to free up resources. -
index.blocks.read
-
Set to
true
to disable read operations against the index. -
index.blocks.write
-
Set to
true
to disable data write operations against the index. Unlikeread_only
, this setting does not affect metadata. For instance, you can close an index with awrite
block, but not an index with aread_only
block. -
index.blocks.metadata
-
Set to
true
to disable index metadata reads and writes. -
index.max_refresh_listeners
-
Maximum number of refresh listeners available on each shard of the index.
These listeners are used to implement
refresh=wait_for
. -
index.analyze.max_token_count
-
The maximum number of tokens that can be produced using _analyze API.
Defaults to
10000
. -
index.highlight.max_analyzed_offset
-
The maximum number of characters that will be analyzed for a highlight request.
This setting is only applicable when highlighting is requested on a text that was indexed without offsets or term vectors.
Defaults to
1000000
.
-
index.max_terms_count
-
The maximum number of terms that can be used in Terms Query.
Defaults to
65536
.
-
index.max_regex_length
-
The maximum length of regex that can be used in Regexp Query.
Defaults to
1000
. -
index.routing.allocation.enable
-
Controls shard allocation for this index. It can be set to:
-
all
(default) - Allows shard allocation for all shards. -
primaries
- Allows shard allocation only for primary shards. -
new_primaries
- Allows shard allocation only for newly-created primary shards. -
none
- No shard allocation is allowed.
-
-
index.routing.rebalance.enable
-
Enables shard rebalancing for this index. It can be set to:
-
all
(default) - Allows shard rebalancing for all shards. -
primaries
- Allows shard rebalancing only for primary shards. -
replicas
- Allows shard rebalancing only for replica shards. -
none
- No shard rebalancing is allowed.
-
-
index.gc_deletes
-
The length of time that a deleted document’s version number remains available for further versioned operations.
Defaults to
60s
. -
index.default_pipeline
-
The default ingest node pipeline for this index. Index requests will fail
if the default pipeline is set and the pipeline does not exist. The default may be
overridden using the
pipeline
parameter. The special pipeline name_none
indicates no ingest pipeline should be run.
Settings in other index modules
editOther index settings are available in index modules:
- Analysis
- Settings to define analyzers, tokenizers, token filters and character filters.
- Index shard allocation
- Control over where, when, and how shards are allocated to nodes.
- Mapping
- Enable or disable dynamic mapping for an index.
- Merging
- Control over how shards are merged by the background merge process.
- Similarities
- Configure custom similarity settings to customize how search results are scored.
- Slowlog
- Control over how slow queries and fetch requests are logged.
- Store
- Configure the type of filesystem used to access shard data.
- Translog
- Control over the transaction log and background flush operations.
X-Pack index settings
edit- Index lifecycle management
- Specify the lifecycle policy and rollover alias for an index.
On this page