Settings changes

edit

From Elasticsearch 5.0 on all settings are validated before they are applied. Node level and default index level settings are validated on node startup, dynamic cluster and index setting are validated before they are updated/added to the cluster state.

Every setting must be a known setting. All settings must have been registered with the node or transport client they are used with. This implies that plugins that define custom settings must register all of their settings during plugin loading using the SettingsModule#registerSettings(Setting) method.

Index Level Settings

edit

In previous versions Elasticsearch allowed to specify index level setting as defaults on the node level, inside the elasticsearch.yaml file or even via command-line parameters. From Elasticsearch 5.0 on only selected settings like for instance index.codec can be set on the node level. All other settings must be set on each individual index. To set default values on every index, index templates should be used instead.

Node settings

edit

The name setting has been removed and is replaced by node.name. Usage of -Dname=some_node_name is not supported anymore.

The node.add_id_to_custom_path was renamed to add_lock_id_to_custom_path.

The default for the node.name settings is now the first 7 characters of the node id, which is in turn a randomly generated UUID.

The settings node.mode and node.local are removed. Local mode should be configured via transport.type: local. In order to disable HTTP please use http.enabled: false

Node attribute settings

edit

Node level attributes used for allocation filtering, forced awareness or other node identification / grouping must be prefixed with node.attr. In previous versions it was possible to specify node attributes with the node. prefix. All node attributes except of node.master, node.data and node.ingest must be moved to the new node.attr. namespace.

Node types settings

edit

The node.client setting has been removed. A node with such a setting set will not start up. Instead, each node role needs to be set separately using the existing node.master, node.data and node.ingest supported static settings.

Gateway settings

edit

The gateway.format setting for configuring global and index state serialization format has been removed. By default, smile is used as the format.

Transport Settings

edit

All settings with a netty infix have been replaced by their already existing transport synonyms. For instance transport.netty.bind_host is no longer supported and should be replaced by the superseding setting transport.bind_host.

Security manager settings

edit

The option to disable the security manager security.manager.enabled has been removed. In order to grant special permissions to elasticsearch users must edit the local Java Security Policy.

Network settings

edit

The _non_loopback_ value for settings like network.host would arbitrarily pick the first interface not marked as loopback. Instead, specify by address scope (e.g. _local_,_site_ for all loopback and private network addresses) or by explicit interface names, hostnames, or addresses.

The netty.epollBugWorkaround settings is removed. This settings allow people to enable a netty work around for a high CPU usage issue with early JVM versions. This bug was fixed in Java 7. Since Elasticsearch 5.0 requires Java 8 the settings is removed. Note that if the workaround needs to be reintroduced you can still set the org.jboss.netty.epollBugWorkaround system property to control Netty directly.

Forbid changing of thread pool types

edit

Previously, thread pool types could be dynamically adjusted. The thread pool type effectively controls the backing queue for the thread pool and modifying this is an expert setting with minimal practical benefits and high risk of being misused. The ability to change the thread pool type for any thread pool has been removed. It is still possible to adjust relevant thread pool parameters for each of the thread pools (e.g., depending on the thread pool type, keep_alive, queue_size, etc.).

Threadpool settings

edit

The suggest threadpool has been removed, now suggest requests use the search threadpool.

The prefix on all thread pool settings has been changed from threadpool to thread_pool.

The minimum size setting for a scaling thread pool has been changed from min to core.

The maximum size setting for a scaling thread pool has been changed from size to max.

The queue size setting for a fixed thread pool must be queue_size (all other variants that were previously supported are no longer supported).

Thread pool settings are now node-level settings. As such, it is not possible to update thread pool settings via the cluster settings API.

Analysis settings

edit

The index.analysis.analyzer.default_index analyzer is not supported anymore. If you wish to change the analyzer to use for indexing, change the index.analysis.analyzer.default analyzer instead.

Ping settings

edit

Previously, there were three settings for the ping timeout: discovery.zen.initial_ping_timeout, discovery.zen.ping.timeout and discovery.zen.ping_timeout. The former two have been removed and the only setting key for the ping timeout is now discovery.zen.ping_timeout. The default value for ping timeouts remains at three seconds.

discovery.zen.master_election.filter_client and discovery.zen.master_election.filter_data have been removed in favor of the new discovery.zen.master_election.ignore_non_master_pings. This setting control how ping responses are interpreted during master election and should be used with care and only in extreme cases. See documentation for details.

Recovery settings

edit

Recovery settings deprecated in 1.x have been removed:

  • index.shard.recovery.translog_size is superseded by indices.recovery.translog_size
  • index.shard.recovery.translog_ops is superseded by indices.recovery.translog_ops
  • index.shard.recovery.file_chunk_size is superseded by indices.recovery.file_chunk_size
  • indices.recovery.concurrent_streams is superseded by cluster.routing.allocation.node_concurrent_recoveries
  • index.shard.recovery.concurrent_small_file_streams is superseded by indices.recovery.concurrent_small_file_streams
  • indices.recovery.max_size_per_sec is superseded by indices.recovery.max_bytes_per_sec

If you are using any of these settings please take the time to review their purpose. All of the settings above are considered expert settings and should only be used if absolutely necessary. If you have set any of the above setting as persistent cluster settings please use the settings update API and set their superseded keys accordingly.

The following settings have been removed without replacement

  • indices.recovery.concurrent_small_file_streams - recoveries are now single threaded. The number of concurrent outgoing recoveries are throttled via allocation deciders
  • indices.recovery.concurrent_file_streams - recoveries are now single threaded. The number of concurrent outgoing recoveries are throttled via allocation deciders

Translog settings

edit

The index.translog.flush_threshold_ops setting is not supported anymore. In order to control flushes based on the transaction log growth use index.translog.flush_threshold_size instead.

Changing the translog type with index.translog.fs.type is not supported anymore, the buffered implementation is now the only available option and uses a fixed 8kb buffer.

The translog by default is fsynced after every index, create, update, delete, or bulk request. The ability to fsync on every operation is not necessary anymore. In fact, it can be a performance bottleneck and it’s trappy since it enabled by a special value set on index.translog.sync_interval. Now, index.translog.sync_interval doesn’t accept a value less than 100ms which prevents fsyncing too often if async durability is enabled. The special value 0 is no longer supported.

index.translog.interval has been removed.

Request Cache Settings

edit

The deprecated settings index.cache.query.enable and indices.cache.query.size have been removed and are replaced with index.requests.cache.enable and indices.requests.cache.size respectively.

indices.requests.cache.clean_interval has been replaced with indices.cache.clean_interval and is no longer supported.

Field Data Cache Settings

edit

The indices.fielddata.cache.clean_interval setting has been replaced with indices.cache.clean_interval.

Allocation settings

edit

The cluster.routing.allocation.concurrent_recoveries setting has been replaced with cluster.routing.allocation.node_concurrent_recoveries.

Similarity settings

edit

The default similarity has been renamed to classic.

Indexing settings

edit

The indices.memory.min_shard_index_buffer_size and indices.memory.max_shard_index_buffer_size have been removed as Elasticsearch now allows any one shard to use amount of heap as long as the total indexing buffer heap used across all shards is below the node’s indices.memory.index_buffer_size (defaults to 10% of the JVM heap).

Removed es.max-open-files

edit

Setting the system property es.max-open-files to true to get Elasticsearch to print the number of maximum open files for the Elasticsearch process has been removed. This same information can be obtained from the Nodes Info API, and a warning is logged on startup if it is set too low.

Removed es.netty.gathering

edit

Disabling Netty from using NIO gathering could be done via the escape hatch of setting the system property "es.netty.gathering" to "false". Time has proven enabling gathering by default is a non-issue and this non-documented setting has been removed.

Removed es.useLinkedTransferQueue

edit

The system property es.useLinkedTransferQueue could be used to control the queue implementation used in the cluster service and the handling of ping responses during discovery. This was an undocumented setting and has been removed.

Cache concurrency level settings removed

edit

Two cache concurrency level settings indices.requests.cache.concurrency_level and indices.fielddata.cache.concurrency_level because they no longer apply to the cache implementation used for the request cache and the field data cache.

Using system properties to configure Elasticsearch

edit

Elasticsearch can no longer be configured by setting system properties. This means that support for all of the following has been removed:

  • setting via command line arguments to elasticsearch as -Des.name.of.setting=value.of.setting
  • setting via the JAVA_OPTS environment variable JAVA_OPTS=$JAVA_OPTS -Des.name.of.setting=value.of.setting
  • setting via the ES_JAVA_OPTS environment variable ES_JAVA_OPTS=$ES_JAVA_OPTS -Des.name.of.setting=value.of.setting

Instead, use -Ename.of.setting=value.of.setting.

Removed using double-dashes to configure Elasticsearch

edit

Elasticsearch could previously be configured on the command line by setting settings via --name.of.setting value.of.setting. This feature has been removed. Instead, use -Ename.of.setting=value.of.setting.

Remove support for .properties config files

edit

The Elasticsearch configuration and logging configuration can no longer be stored in the Java properties file format (line-delimited key=value pairs with a .properties extension).

Discovery Settings

edit

The discovery.zen.minimum_master_nodes must be set for nodes that have network.host, network.bind_host, network.publish_host, transport.host, transport.bind_host, or transport.publish_host configuration options set. We see those nodes as in "production" mode and thus require the setting.

Realtime get setting

edit

The action.get.realtime setting has been removed. This setting was a fallback realtime setting for the get and mget APIs when realtime wasn’t specified. Now if the parameter isn’t specified we always default to true.

Memory lock settings

edit

The setting bootstrap.mlockall has been renamed to bootstrap.memory_lock.

Snapshot settings

edit

The default setting include_global_state for restoring snapshots has been changed from true to false. It has not been changed for taking snapshots and still defaults to true in that case.

Time value parsing

edit

The unit w representing weeks is no longer supported.

Fractional time values (e.g., 0.5s) are no longer supported. For example, this means when setting timeouts "0.5s" will be rejected and should instead be input as "500ms".

Node max local storage nodes

edit

Previous versions of Elasticsearch defaulted to allowing multiple nodes to share the same data directory (up to 50). This can be confusing where users accidentally startup multiple nodes and end up thinking that they’ve lost data because the second node will start with an empty data directory. While the default of allowing multiple nodes is friendly to playing with forming a small cluster on a laptop, and end-users do sometimes run multiple nodes on the same host, this tends to be the exception. Keeping with Elasticsearch’s continual movement towards safer out-of-the-box defaults, and optimizing for the norm instead of the exception, the default for node.max_local_storage_nodes is now one.

Script settings

edit

Indexed script settings

edit

Due to the fact that indexed script has been replaced by stored scripts the following settings have been replaced to:

  • script.indexed has been replaced by script.stored
  • script.engine.*.indexed.aggs has been replaced by script.engine.*.stored.aggs (where * represents the script language, like groovy, mustache, painless etc.)
  • script.engine.*.indexed.mapping has been replaced by script.engine.*.stored.mapping (where * represents the script language, like groovy, mustache, painless etc.)
  • script.engine.*.indexed.search has been replaced by script.engine.*.stored.search (where * represents the script language, like groovy, mustache, painless etc.)
  • script.engine.*.indexed.update has been replaced by script.engine.*.stored.update (where * represents the script language, like groovy, mustache, painless etc.)
  • script.engine.*.indexed.plugin has been replaced by script.engine.*.stored.plugin (where * represents the script language, like groovy, mustache, painless etc.)

Script mode settings

edit

Previously script mode settings (e.g., "script.inline: true", "script.engine.groovy.inline.aggs: false", etc.) accepted a wide range of "truthy" or "falsy" values. This is now much stricter and supports only the true and false options.

Script sandbox settings removed

edit

Prior to 5.0 a third option could be specified for the script.inline and script.stored settings ("sandbox"). This has been removed, you can now only set script.inline: true or script.stored: true.

Search settings

edit

The setting index.query.bool.max_clause_count has been removed. In order to set the maximum number of boolean clauses indices.query.bool.max_clause_count should be used instead.