New

The executive guide to generative AI

Read more
About usPartnersSupport|Login
IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Reindex from a remote cluster Analysis »
Elastic Docs ›Elasticsearch Guide [7.12]

Index modules

edit

Index modules

edit

Index Modules are modules created per index and control all aspects related to an index.

Index Settings

edit

Index level settings can be set per-index. Settings may be:

static
They can only be set at index creation time or on a closed index.
dynamic
They can be changed on a live index using the update-index-settings API.

Changing static or dynamic index settings on a closed index could result in incorrect settings that are impossible to rectify without deleting and recreating the index.

Static index settings

edit

Below is a list of all static index settings that are not associated with any specific index module:

index.number_of_shards

The number of primary shards that an index should have. Defaults to 1. This setting can only be set at index creation time. It cannot be changed on a closed index.

The number of shards are limited to 1024 per index. This limitation is a safety limit to prevent accidental creation of indices that can destabilize a cluster due to resource allocation. The limit can be modified by specifying export ES_JAVA_OPTS="-Des.index.max_number_of_shards=128" system property on every node that is part of the cluster.

index.number_of_routing_shards

Number of routing shards used to split an index.

For example, a 5 shard index with number_of_routing_shards set to 30 (5 x 2 x 3) could be split by a factor of 2 or 3. In other words, it could be split as follows:

  • 5 → 10 → 30 (split by 2, then by 3)
  • 5 → 15 → 30 (split by 3, then by 2)
  • 5 → 30 (split by 6)

This setting’s default value depends on the number of primary shards in the index. The default is designed to allow you to split by factors of 2 up to a maximum of 1024 shards.

In Elasticsearch 7.0.0 and later versions, this setting affects how documents are distributed across shards. When reindexing an older index with custom routing, you must explicitly set index.number_of_routing_shards to maintain the same document distribution. See the related breaking change.

index.shard.check_on_startup

Whether or not shards should be checked for corruption before opening. When corruption is detected, it will prevent the shard from being opened. Accepts:

false
(default) Don’t check for corruption when opening a shard.
checksum
Check for physical corruption.
true

Check for both physical and logical corruption. This is much more expensive in terms of CPU and memory usage.

Expert only. Checking shards may take a lot of time on large indices.

index.codec
The default value compresses stored data with LZ4 compression, but this can be set to best_compression which uses DEFLATE for a higher compression ratio, at the expense of slower stored fields performance. If you are updating the compression type, the new one will be applied after segments are merged. Segment merging can be forced using force merge.
index.routing_partition_size
The number of shards a custom routing value can go to. Defaults to 1 and can only be set at index creation time. This value must be less than the index.number_of_shards unless the index.number_of_shards value is also 1. See Routing to an index partition for more details about how this setting is used.
index.soft_deletes.enabled
[7.6.0] Deprecated in 7.6.0. Creating indices with soft-deletes disabled is deprecated and will be removed in future Elasticsearch versions. Indicates whether soft deletes are enabled on the index. Soft deletes can only be configured at index creation and only on indices created on or after Elasticsearch 6.5.0. Defaults to true.
index.soft_deletes.retention_lease.period
The maximum period to retain a shard history retention lease before it is considered expired. Shard history retention leases ensure that soft deletes are retained during merges on the Lucene index. If a soft delete is merged away before it can be replicated to a follower the following process will fail due to incomplete history on the leader. Defaults to 12h.
index.load_fixed_bitset_filters_eagerly
Indicates whether cached filters are pre-loaded for nested queries. Possible values are true (default) and false.
index.hidden
Indicates whether the index should be hidden by default. Hidden indices are not returned by default when using a wildcard expression. This behavior is controlled per request through the use of the expand_wildcards parameter. Possible values are true and false (default).

Dynamic index settings

edit

Below is a list of all dynamic index settings that are not associated with any specific index module:

index.number_of_replicas
The number of replicas each primary shard has. Defaults to 1.
index.auto_expand_replicas
Auto-expand the number of replicas based on the number of data nodes in the cluster. Set to a dash delimited lower and upper bound (e.g. 0-5) or use all for the upper bound (e.g. 0-all). Defaults to false (i.e. disabled). Note that the auto-expanded number of replicas only takes allocation filtering rules into account, but ignores any other allocation rules such as shard allocation awareness and total shards per node, and this can lead to the cluster health becoming YELLOW if the applicable rules prevent all the replicas from being allocated.
index.search.idle.after
How long a shard can not receive a search or get request until it’s considered search idle. (default is 30s)
index.refresh_interval
How often to perform a refresh operation, which makes recent changes to the index visible to search. Defaults to 1s. Can be set to -1 to disable refresh. If this setting is not explicitly set, shards that haven’t seen search traffic for at least index.search.idle.after seconds will not receive background refreshes until they receive a search request. Searches that hit an idle shard where a refresh is pending will wait for the next background refresh (within 1s). This behavior aims to automatically optimize bulk indexing in the default case when no searches are performed. In order to opt out of this behavior an explicit value of 1s should set as the refresh interval.
index.max_result_window
The maximum value of from + size for searches to this index. Defaults to 10000. Search requests take heap memory and time proportional to from + size and this limits that memory. See Scroll or Search After for a more efficient alternative to raising this.
index.max_inner_result_window
The maximum value of from + size for inner hits definition and top hits aggregations to this index. Defaults to 100. Inner hits and top hits aggregation take heap memory and time proportional to from + size and this limits that memory.
index.max_rescore_window
The maximum value of window_size for rescore requests in searches of this index. Defaults to index.max_result_window which defaults to 10000. Search requests take heap memory and time proportional to max(window_size, from + size) and this limits that memory.
index.max_docvalue_fields_search
The maximum number of docvalue_fields that are allowed in a query. Defaults to 100. Doc-value fields are costly since they might incur a per-field per-document seek.
index.max_script_fields
The maximum number of script_fields that are allowed in a query. Defaults to 32.
index.max_ngram_diff
The maximum allowed difference between min_gram and max_gram for NGramTokenizer and NGramTokenFilter. Defaults to 1.
index.max_shingle_diff
The maximum allowed difference between max_shingle_size and min_shingle_size for the shingle token filter. Defaults to 3.
index.max_refresh_listeners
Maximum number of refresh listeners available on each shard of the index. These listeners are used to implement refresh=wait_for.
index.analyze.max_token_count
The maximum number of tokens that can be produced using _analyze API. Defaults to 10000.
index.highlight.max_analyzed_offset
The maximum number of characters that will be analyzed for a highlight request. This setting is only applicable when highlighting is requested on a text that was indexed without offsets or term vectors. Defaults to 1000000.
index.max_terms_count
The maximum number of terms that can be used in Terms Query. Defaults to 65536.
index.max_regex_length
The maximum length of regex that can be used in Regexp Query. Defaults to 1000.
index.query.default_field

(string or array of strings) Wildcard (*) patterns matching one or more fields. The following query types search these matching fields by default:

Defaults to *, which matches all fields eligible for term-level queries, excluding metadata fields.

index.routing.allocation.enable

Controls shard allocation for this index. It can be set to:

  • all (default) - Allows shard allocation for all shards.
  • primaries - Allows shard allocation only for primary shards.
  • new_primaries - Allows shard allocation only for newly-created primary shards.
  • none - No shard allocation is allowed.
index.routing.rebalance.enable

Enables shard rebalancing for this index. It can be set to:

  • all (default) - Allows shard rebalancing for all shards.
  • primaries - Allows shard rebalancing only for primary shards.
  • replicas - Allows shard rebalancing only for replica shards.
  • none - No shard rebalancing is allowed.
index.gc_deletes
The length of time that a deleted document’s version number remains available for further versioned operations. Defaults to 60s.
index.default_pipeline
The default ingest node pipeline for this index. Index requests will fail if the default pipeline is set and the pipeline does not exist. The default may be overridden using the pipeline parameter. The special pipeline name _none indicates no ingest pipeline should be run.
index.final_pipeline

The final ingest node pipeline for this index. Indexing requests will fail if the final pipeline is set and the pipeline does not exist. The final pipeline always runs after the request pipeline (if specified) and the default pipeline (if it exists). The special pipeline name _none indicates no ingest pipeline will run.

You can’t use a final pipelines to change the _index field. If the pipeline attempts to change the _index field, the indexing request will fail.

Settings in other index modules

edit

Other index settings are available in index modules:

Analysis
Settings to define analyzers, tokenizers, token filters and character filters.
Index shard allocation
Control over where, when, and how shards are allocated to nodes.
Mapping
Enable or disable dynamic mapping for an index.
Merging
Control over how shards are merged by the background merge process.
Similarities
Configure custom similarity settings to customize how search results are scored.
Slowlog
Control over how slow queries and fetch requests are logged.
Store
Configure the type of filesystem used to access shard data.
Translog
Control over the transaction log and background flush operations.
History retention
Control over the retention of a history of operations in the index.
Indexing pressure
Configure indexing back pressure limits.

X-Pack index settings

edit
Index lifecycle management
Specify the lifecycle policy and rollover alias for an index.
« Reindex from a remote cluster Analysis »