Migrating to 8.0

Details
The default for the action.destructive_requires_name setting changes from false to true in Elasticsearch 8.0.0.

Previously, defaulting to false allowed users to use wildcard patterns to delete, close, or change index blocks on indices. To prevent the accidental deletion of indices that happen to match a wildcard pattern, we now default to requiring that destructive operations explicitly name the indices to be modified.

Impact
To use wildcard patterns for destructive actions, set action.destructive_requires_name to false using the https://www.elastic.co/guide/en/elasticsearch/reference/8.3/cluster-update-settings.html cluster settings API].

Details
You can no longer set xpack.searchable.snapshot.shared_cache.size on a node that doesn’t have the data_frozen node role. This setting reserves disk space for the shared cache of partially mounted indices. Elasticsearch only allocates partially mounted indices to nodes with the data_frozen role.

Impact
Remove xpack.searchable.snapshot.shared_cache.size from elasticsearch.yml for nodes that don’t have the data_frozen role. Specifying the setting on a non-frozen node will result in an error on startup.

Details
Elasticsearch will now dynamically set the maximum number of allowed clauses in a query, using a heuristic based on the size of the search thread pool and the size of the heap allocated to the JVM. This limit has a minimum value of 1024 and will in most cases be larger (for example, a node with 30Gb RAM and 48 CPUs will have a maximum clause count of around 27,000). Larger heaps lead to higher values, and larger thread pools result in lower values.

Impact
Queries with many clauses should be avoided whenever possible. If you previously bumped this setting to accommodate heavy queries, you might need to increase the amount of memory available to Elasticsearch, or to reduce the size of your search thread pool so that more memory is available to each concurrent search.

In previous versions of Lucene you could get around this limit by nesting boolean queries within each other, but the limit is now based on the total number of leaf queries within the query as a whole and this workaround will no longer help.

Specifying indices.query.bool.max_clause_count will have no effect but will generate deprecation warnings. To avoid these warnings, remove the setting from elasticsearch.yml during an upgrade or node restart.

Details
Setting indices.lifecycle.poll_interval too low can cause excessive load on a cluster. The poll interval must now be at least 1s (one second).

Impact
Set indices.lifecycle.poll_interval setting to 1s or greater in elasticsearch.yml or through the cluster update settings API.

Setting indices.lifecycle.poll_interval to less than 1s in elasticsearch.yml will result in an error on startup. Cluster update settings API requests that set indices.lifecycle.poll_interval to less than 1s will return an error.

Details
The file and native realms are now enabled unless explicitly disabled. If explicitly disabled, the file and native realms remain disabled at all times.

Previously, the file and native realms had the following implicit behaviors:

Impact
To explicitly disable the file or native realm, set the respective file.<realm-name>.enabled or native.<realm-name>.enabled setting to false under the xpack.security.authc.realms namespace in elasticsearch.yml.

The following configuration example disables the native realm and the file realm.

xpack.security.authc.realms:

  native.realm1.enabled: false
  file.realm2.enabled: false

  ...

Details
The xpack.security.authc.realms.{type}.{name}.order setting is now required and must be specified for each explicitly configured realm. Each value must be unique.

Impact
The cluster will fail to start if the requirements are not met.

For example, the following configuration is invalid:

xpack.security.authc.realms.kerberos.kerb1:
  keytab.path: es.keytab
  remove_realm_name: false

And must be configured as:

xpack.security.authc.realms.kerberos.kerb1:
  order: 0
  keytab.path: es.keytab
  remove_realm_name: false

Details
Elasticsearch now always accounts for the sizes of relocating shards when making allocation decisions based on the disk usage of the nodes in the cluster. In earlier versions, you could disable this by setting cluster.routing.allocation.disk.include_relocations to false. That could result in poor allocation decisions that could overshoot watermarks and require significant extra work to correct. The cluster.routing.allocation.disk.include_relocations setting has been removed.

Impact
Remove the cluster.routing.allocation.disk.include_relocations setting. Specifying this setting in elasticsearch.yml will result in an error on startup.

Details
The cluster.join.timeout setting has been removed. Join attempts no longer time out.

Impact
Remove cluster.join.timeout from elasticsearch.yml.

Details
All settings under the discovery.zen namespace are no longer supported. They existed only only for BWC reasons in 7.x. This includes:

Impact
Remove the discovery.zen settings from elasticsearch.yml. Specifying these settings will result in an error on startup.

Details
The http.content_type.required setting was deprecated in Elasticsearch 6.0 and has been removed in Elasticsearch 8.0. The setting was introduced in Elasticsearch 5.3 to prepare users for Elasticsearch 6.0, where content type auto detection was removed for HTTP requests.

Impact
Remove the http.content_type.required setting from elasticsearch.yml. Specifying this setting will result in an error on startup.

Details
The http.tcp_no_delay setting was deprecated in 7.x and has been removed in 8.0. Use`http.tcp.no_delay` instead.

Impact
Replace the http.tcp_no_delay setting with http.tcp.no_delay. Specifying http.tcp_no_delay in elasticsearch.yml will result in an error on startup.

Details
The network.tcp.connect_timeout setting was deprecated in 7.x and has been removed in 8.0. This setting was a fallback setting for transport.connect_timeout.

Impact
Remove the`network.tcp.connect_timeout` setting. Use the transport.connect_timeout setting to change the default connection timeout for client connections. Specifying network.tcp.connect_timeout in elasticsearch.yml will result in an error on startup.

Details
The node.max_local_storage_nodes setting was deprecated in 7.x and has been removed in 8.0. Nodes should be run on separate data paths to ensure that each node is consistently assigned to the same data path.

Impact
Remove the node.max_local_storage_nodes setting. Specifying this setting in elasticsearch.yml will result in an error on startup.

Details
The xpack.security.authc.accept_default_password setting has not had any affect since the 6.0 release of Elasticsearch and is no longer allowed.

Impact
Remove the xpack.security.authc.accept_default_password setting from elasticsearch.yml. Specifying this setting will result in an error on startup.

Details
The xpack.security.authz.store.roles.index.cache.max_size and xpack.security.authz.store.roles.index.cache.ttl settings have been removed. These settings have been redundant and deprecated since the 5.2 release of Elasticsearch.

Impact
Remove the xpack.security.authz.store.roles.index.cache.max_size and xpack.security.authz.store.roles.index.cache.ttl settings from elasticsearch.yml . Specifying these settings will result in an error on startup.

Details
The transport.profiles.*.xpack.security.type setting is no longer supported. The Transport Client has been removed and all client traffic now uses the HTTP transport. Transport profiles using this setting should be removed.

Impact
Remove the transport.profiles.*.xpack.security.type setting from elasticsearch.yml. Specifying this setting in a transport profile will result in an error on startup.

Details
In SAML, Identity Providers (IdPs) can either be explicitly configured to release a NameID with a specific format, or configured to attempt to conform with the requirements of a Service Provider (SP). The SP declares its requirements in the NameIDPolicy element of a SAML Authentication Request. In Elasticsearch, the nameid_format SAML realm setting controls the NameIDPolicy value.

Previously, the default value for nameid_format was urn:oasis:names:tc:SAML:2.0:nameid-format:transient. This setting created authentication requests that required the IdP to release NameID with a transient format.

The default value has been removed, which means that Elasticsearch will create SAML Authentication Requests by default that don’t put this requirement on the IdP. If you want to retain the previous behavior, set nameid_format to urn:oasis:names:tc:SAML:2.0:nameid-format:transient.

Impact
If you currently don’t configure nameid_format explicitly, it’s possible that your IdP will reject authentication requests from Elasticsearch because the requests do not specify a NameID format (and your IdP is configured to expect one). This mismatch can result in a broken SAML configuration. If you’re unsure whether your IdP is explicitly configured to use a certain NameID format and you want to retain current behavior , try setting nameid_format to urn:oasis:names:tc:SAML:2.0:nameid-format:transient explicitly.

Details
It is now an error to configure any SSL settings for xpack.security.transport.ssl without also configuring xpack.security.transport.ssl.enabled.

Impact
If using other xpack.security.transport.ssl settings, you must explicitly specify the xpack.security.transport.ssl.enabled setting.

If you do not want to enable SSL and are currently using other xpack.security.transport.ssl settings, do one of the following:

If you want to enable SSL, follow the instructions in Encrypting communications between nodes in a cluster. As part of this configuration, explicitly specify xpack.security.transport.ssl.enabled as true.

For example, the following configuration is invalid:

xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

And must be configured as:

xpack.security.transport.ssl.enabled: true 
xpack.security.transport.ssl.keystore.path: elastic-certificates.p12
xpack.security.transport.ssl.truststore.path: elastic-certificates.p12

Details
It is now an error to configure any SSL settings for xpack.security.http.ssl without also configuring xpack.security.http.ssl.enabled.

Impact
If using other xpack.security.http.ssl settings, you must explicitly specify the xpack.security.http.ssl.enabled setting.

If you do not want to enable SSL and are currently using other xpack.security.http.ssl settings, do one of the following:

If you want to enable SSL, follow the instructions in Encrypting HTTP client communications. As part of this configuration, explicitly specify xpack.security.http.ssl.enabled as true.

For example, the following configuration is invalid:

xpack.security.http.ssl.certificate: elasticsearch.crt
xpack.security.http.ssl.key: elasticsearch.key
xpack.security.http.ssl.certificate_authorities: [ "corporate-ca.crt" ]

And must be configured as either:

xpack.security.http.ssl.enabled: true 
xpack.security.http.ssl.certificate: elasticsearch.crt
xpack.security.http.ssl.key: elasticsearch.key
xpack.security.http.ssl.certificate_authorities: [ "corporate-ca.crt" ]

Details
It is now an error to enable SSL for the transport interface without also configuring a certificate and key through use of the xpack.security.transport.ssl.keystore.path setting or the xpack.security.transport.ssl.certificate and xpack.security.transport.ssl.key settings.

Impact
If xpack.security.transport.ssl.enabled is set to true, provide a certificate and key using the xpack.security.transport.ssl.keystore.path setting or the xpack.security.transport.ssl.certificate and xpack.security.transport.ssl.key settings. If a certificate and key is not provided, Elasticsearch will return in an error on startup.

Details
It is now an error to enable SSL for the HTTP (Rest) server without also configuring a certificate and key through use of the xpack.security.http.ssl.keystore.path setting or the xpack.security.http.ssl.certificate and xpack.security.http.ssl.key settings.

Impact
If xpack.security.http.ssl.enabled is set to true, provide a certificate and key using the xpack.security.http.ssl.keystore.path setting or the xpack.security.http.ssl.certificate and xpack.security.http.ssl.key settings. If certificate and key is not provided, Elasticsearch will return in an error on startup.

Details
The settings *.ssl.keystore.type and *.ssl.truststore.type no longer accept "PKCS11" as a valid type. This applies to all SSL settings in Elasticsearch, including

As well as SSL settings for security realms, watcher and monitoring.

Use of a PKCS#11 keystore or truststore as the JRE’s default store is not affected.

Impact
If you have a PKCS#11 keystore configured within your elasticsearch.yml file, you must remove that configuration and switch to a supported keystore type, or configure your PKCS#11 keystore as the JRE default store.

Details
The kibana user was historically used to authenticate Kibana to Elasticsearch. The name of this user was confusing, and was often mistakenly used to login to Kibana. This has been renamed to kibana_system in order to reduce confusion, and to better align with other built-in system accounts.

Impact
Replace any use of the kibana user with the kibana_system user. Specifying the kibana user in kibana.yml will result in an error on startup.

If your kibana.yml used to contain:

elasticsearch.username: kibana

then you should update to use the new kibana_system user instead:

elasticsearch.username: kibana_system

Details
In 6.5 these settings were deprecated in favor of cluster.remote. In 7.x we provided automatic upgrading of these settings to their cluster.remote counterparts. In 8.0.0, these settings have been removed. Elasticsearch will refuse to start if you have these settings in your configuration or cluster state.

Impact
Use the replacement cluster.remote settings. Discontinue use of the search.remote.* settings. Specifying these settings in elasticsearch.yml will result in an error on startup.

Details
To ensure that all settings are in a proper namespace, the pidfile setting was previously deprecated in version 7.4.0 of Elasticsearch, and is removed in version 8.0.0. Instead, use node.pidfile.

Impact
Use the node.pidfile setting. Discontinue use of the pidfile setting. Specifying the pidfile setting in elasticsearch.yml will result in an error on startup.

Details
To ensure that all settings are in a proper namespace, the processors setting was previously deprecated in version 7.4.0 of Elasticsearch, and is removed in version 8.0.0. Instead, use node.processors.

Impact
Use the node.processors setting. Discontinue use of the processors setting. Specifying the processors setting in elasticsearch.yml will result in an error on startup.

Details
Previously it was possible to set the number of processors used to set the default sizes for the thread pools to be more than the number of available processors. As this leads to more context switches and more threads but without an increase in the number of physical CPUs on which to schedule these additional threads, the node.processors setting is now bounded by the number of available processors.

Impact
If specified, ensure the value of node.processors setting does not exceed the number of available processors. Setting the node.processors value greater than the number of available processors in elasticsearch.yml will result in an error on startup.

Details
In Elasticsearch 7.7.0, the setting cluster.remote.connect was deprecated in favor of setting node.remote_cluster_client. In Elasticsearch 8.0.0, the setting cluster.remote.connect is removed.

Impact
Use the node.remote_cluster_client setting. Discontinue use of the cluster.remote.connect setting. Specifying the cluster.remote.connect setting in elasticsearch.yml will result in an error on startup.

Details
In Elasticsearch 7.8.0, the setting node.local_storage was deprecated and beginning in Elasticsearch 8.0.0 all nodes will require local storage. Therefore, the node.local_storage setting has been removed.

Impact
Discontinue use of the node.local_storage setting. Specifying this setting in elasticsearch.yml will result in an error on startup.

Details
In Elasticsearch 7.7.0, the setting xpack.monitoring.exporters.<exporterName>.auth.password was deprecated in favor of setting xpack.monitoring.exporters.<exporterName>.auth.secure_password. In Elasticsearch 8.0.0, the setting xpack.monitoring.exporters.<exporterName>.auth.password is removed.

Impact
Use the xpack.monitoring.exporters.<exporterName>.auth.secure_password setting. Discontinue use of the xpack.monitoring.exporters.<exporterName>.auth.password setting. Specifying the xpack.monitoring.exporters.<exporterName>.auth.password setting in elasticsearch.yml will result in an error on startup.

Details
The following settings were deprecated in Elasticsearch 7.8.0 and have been removed in Elasticsearch 8.0.0:

These basic license features are now always enabled.

If you have disabled ILM so that you can use another tool to manage Watcher indices, the newly introduced xpack.watcher.use_ilm_index_management setting may be set to false.

Impact
Discontinue use of the removed settings. Specifying these settings in elasticsearch.yml will result in an error on startup.

Details
The following cluster settings have been removed:

It is safe to recover the cluster as soon as a majority of master-eligible nodes have joined so there is no benefit in waiting for any additional master-eligible nodes to start.

Impact
Discontinue use of the removed settings. If needed, use gateway.expected_data_nodes or gateway.recover_after_data_nodes to defer cluster recovery pending a certain number of data nodes.

Details
The legacy role settings:

have been removed. Instead, use the node.roles setting. If you were previously using the legacy role settings on a 7.13 or later cluster, you will have a deprecation log message on each of your nodes indicating the exact replacement value for node.roles.

Impact
Discontinue use of the removed settings. Specifying these settings in elasticsearch.yml will result in an error on startup.

Details
Elasticsearch uses system call filters to remove its ability to fork another process. This is useful to mitigate remote code exploits. These system call filters are enabled by default, and were previously controlled via the setting bootstrap.system_call_filter. Starting in Elasticsearch 8.0, system call filters will be required. As such, the setting bootstrap.system_call_filter was deprecated in Elasticsearch 7.13.0, and is removed as of Elasticsearch 8.0.0.

Impact
Discontinue use of the removed setting. Specifying this setting in Elasticsearch configuration will result in an error on startup.

Details
The cluster and index level settings ending in ._tier used for filtering the allocation of a shard to a particular set of nodes have been removed. Instead, the tier preference setting, index.routing.allocation.include._tier_preference should be used. The removed settings are:

Cluster level settings:

Index settings:

Impact
Discontinue use of the removed settings. Specifying any of these cluster settings in Elasticsearch configuration will result in an error on startup. Any indices using these settings will have the settings archived (and they will have no effect) when the index metadata is loaded.

Details
Elasticsearch uses the shared data path as the base path of per index data paths. This feature was previously used with shared replicas. Starting in 7.13.0, these settings are deprecated. Starting in 8.0 only existing indices created in 7.x will be capable of using the shared data path and per index data path settings.

Impact
Discontinue use of the deprecated settings.

Details
In 7.14, setting cluster.routing.allocation.disk.watermark.enable_for_single_data_node to false was deprecated. Starting in 8.0, the only legal value will be true. In a future release, the setting will be removed completely, with same behavior as if the setting was true.

If the old behavior is desired for a single data node cluster, disk based allocation can be disabled by setting cluster.routing.allocation.disk.threshold_enabled: false

Impact
Discontinue use of the deprecated setting.

Details
The gateway.auto_import_dangling_indices cluster setting has been removed. Previously, you could use this setting to automatically import dangling indices. However, automatically importing dangling indices is unsafe. Use the dangling indices APIs to manage and import dangling indices instead.

Impact
Discontinue use of the removed setting. Specifying the setting in elasticsearch.yml will result in an error on startup.

Details
Previously, the transport client used the thread pool to ensure listeners aren’t called back on network threads. The transport client has been removed in 8.0, and the thread pool is no longer needed.

Impact
Remove listener thread pool settings from elasticsearch.yml for any nodes. Specifying listener thread pool settings in elasticsearch.yml will result in an error on startup.

Details
The fixed_auto_queue_size thread pool type, previously marked as an experimental feature, was deprecated in 7.x and has been removed in 8.0. The search and search_throttled thread pools have the fixed type now.

Impact
No action needed.

Details
The following settings have been deprecated in 7.x and removed in 8.0. Each setting has a replacement setting that was introduced in 6.7.

Impact
Use the replacement settings. Discontinue use of the removed settings. Specifying the removed settings in elasticsearch.yml will result in an error on startup.

Details
Prior to 8.0, transport compression was disabled by default. Starting in 8.0, transport.compress defaults to indexing_data. This configuration means that the propagation of raw indexing data will be compressed between nodes.

Impact
Inter-node transit will get reduced along the indexing path. In some scenarios, CPU usage could increase.

Details
Prior to 8.0, the transport.compression_scheme setting defaulted to deflate. Starting in 8.0, transport.compress_scheme defaults to lz4.

Prior to 8.0, the cluster.remote.<cluster_alias>.transport.compression_scheme setting defaulted to deflate when cluster.remote.<cluster_alias>.transport.compress was explicitly configured. Starting in 8.0, cluster.remote.<cluster_alias>.transport.compression_scheme will fallback to transport.compression_scheme by default.

Impact
This configuration means that transport compression will produce somewhat lower compression ratios in exchange for lower CPU load.

Details
For shared file system repositories ("type": "fs"), the node level setting repositories.fs.compress could previously be used to enable compression for all shared file system repositories where compress was not specified. The repositories.fs.compress setting has been removed.

Impact
Discontinue use of the repositories.fs.compress node-level setting. Use the repository-specific compress setting to enable compression instead. Refer to Shared file system repository settings.

Details
If xpack.security.fips_mode.enabled is true (see FIPS 140-2), the value of xpack.security.authc.password_hashing.algorithm now defaults to pbkdf2_stretch.

In earlier versions this setting would always default to bcrypt and a runtime check would prevent a node from starting unless the value was explicitly set to a "pbkdf2" variant.

There is no change for clusters that do not enable FIPS 140 mode.

Impact
This change should not have any impact on upgraded nodes. Any node with an explicitly configured value for the password hashing algorithm will continue to use that configured value. Any node that did not have an explicitly configured password hashing algorithm in Elasticsearch 6.x or Elasticsearch 7.x would have failed to start.

Details

Prior to 8.0, Elasticsearch would internally handle removal of all monitoring indices according to the xpack.monitoring.history.duration setting.

When using metricbeat or elastic agent >= 8.0 to collect monitoring data, indices are managed via an ILM policy. If the setting is present, the policy will be created using the xpack.monitoring.history.duration as an initial retention period.

If you need to customize retention settings for monitoring data collected with metricbeat, please update the .monitoring-8-ilm-policy ILM policy directly.

The xpack.monitoring.history.duration setting will only apply to monitoring indices written using (legacy) internal collection, not indices created by metricbeat or agent.

Impact
After upgrading, insure that the .monitoring-8-ilm-policy ILM policy aligns with your desired retention settings.

If you only use metricbeat or agent to collect monitoring data, you can also remove any custom xpack.monitoring.history.duration settings.

Details
The elasticsearch-migrate tool provided a way to convert file realm users and roles into the native realm. It has been deprecated since Elasticsearch 7.2.0. Users and roles should now be created in the native realm directly.

Impact
Discontinue use of the elasticsearch-migrate tool. Attempts to use the elasticsearch-migrate tool will result in an error.

Details
Directly accessing system indices is deprecated, and may be prevented in a future version. If you must access a system index, create a security role with an index permission that targets the specific index and set the allow_restricted_indices permission to true. Refer to indices privileges for information on adding this permission to an index privilege.

Impact
Accessing system indices directly results in warnings in the header of API responses. If available, use Kibana or the associated feature’s Elasticsearch APIs to manage the data that you want to access.

Details
The index.merge.policy.max_merge_at_once_explicit index setting is deprecated and has no effect.

Previously, you could specify index.merge.policy.max_merge_at_once_explicit to set the maximum number of segments to merge at the same time during a force merge or when expunging deleted documents. In 8.0, this number is unlimited, regardless of the setting.

Impact
Specifying index.merge.policy.max_merge_at_once_explicit will have no effect but will generate deprecation warnings.

To avoid these deprecation warnings, discontinue use of the setting. Don’t specify the setting when creating new indices, and remove the setting from index and component templates.

To remove the setting from an existing data stream or index, specify the setting’s value as null using the update index settings API.

PUT my-index-000001/_settings
{
  "index.merge.policy.max_merge_at_once_explicit": null
}

Details
The index.max_adjacency_matrix_filters index setting has been removed. Previously, you could use this setting to configure the maximum number of filters for the adjacency matrix aggregation. The indices.query.bool.max_clause_count index setting now determines the maximum number of filters for the aggregation.

Impact
Discontinue use of the index.max_adjacency_matrix_filters index setting.

Requests that include the index setting will return an error. If you upgrade a cluster with a 7.x index that already contains the setting, Elasticsearch will archive the setting.

Remove the index setting from index and component templates. Attempts to use a template that contains the setting will fail and return an error. This includes automated operations, such the ILM rollover action.

Details
The index.force_memory_term_dictionary setting was introduced in 7.0 as a temporary measure to allow users to opt-out of the optimization that leaves the term dictionary on disk when appropriate. This optimization is now mandatory and the setting is removed.

Impact
Discontinue use of the index.force_memory_term_dictionary index setting. Requests that include this setting will return an error.

Details
Creating indices with soft deletes disabled was deprecated in 7.6 and is no longer supported in 8.0. The index.soft_deletes.enabled setting can no longer be set to false.

Impact
Discontinue use of the index.soft_deletes.enabled index setting. Requests that set index.soft_deletes.enabled to false will return an error.

Details
Translog retention settings index.translog.retention.age and index.translog.retention.size were effectively ignored in 7.4, deprecated in 7.7, and removed in 8.0 in favor of soft deletes.

Impact
Discontinue use of the index.translog.retention.age and index.translog.retention.size index settings. Requests that include these settings will return an error.

Details
In the high level REST client, the indexlifecycle package has been renamed to ilm to match the package rename inside the Elasticsearch code.

Impact
Update your workflow and applications to use the ilm package in place of indexlifecycle.

Details
To create Fuzziness instances, use the fromString and fromEdits method instead of the build method that used to accept both Strings and numeric values. Several fuzziness setters on query builders (e.g. MatchQueryBuilder#fuzziness) now accept only a `Fuzziness`instance instead of an Object.

Fuzziness used to be lenient when it comes to parsing arbitrary numeric values while silently truncating them to one of the three allowed edit distances 0, 1 or 2. This leniency is now removed and the class will throw errors when trying to construct an instance with another value (e.g. floats like 1.3 used to get accepted but truncated to 1).

Impact
Use the available constants (e.g. Fuzziness.ONE, Fuzziness.AUTO) or build your own instance using the above mentioned factory methods. Use only allowed Fuzziness values.

Details
Repository has no dependency on IndexShard anymore. The contract of restoreShard and snapshotShard has been reduced to Store and MappingService in order to improve testability.

Impact
No action needed.

Details
If a node exceeds the flood-stage disk watermark then we add a block to all of its indices to prevent further writes as a last-ditch attempt to prevent the node completely exhausting its disk space. By default, from 7.4 onwards the block is automatically removed when a node drops below the high watermark again, but this behaviour could be disabled by setting the system property es.disk.auto_release_flood_stage_block to false. This behaviour is no longer optional, and this system property must now not be set.

Impact
Discontinue use of the es.disk.auto_release_flood_stage_block system property. Setting this system property will result in an error on startup.

Details
Starting in version 7.4, a + in a URL will be encoded as %2B by all REST API functionality. Prior versions handled a + as a single space. In these previous versions, if your application required handling + as a single space, you could return to the old behaviour by setting the system property es.rest.url_plus_as_space to true. Note that this behaviour is deprecated and setting this system property to true will cease to be supported in version 8.

Impact
Update your application or workflow to assume + in a URL is encoded as %2B.

Details
Elasticsearch has a check that verifies that communicating pairs of nodes of the same version are running exactly the same build and therefore using the same wire format as each other. In previous versions this check can be bypassed by setting the system property es.unsafely_permit_handshake_from_incompatible_builds to true. The use of this system property is now forbidden.

Impact
Discontinue use of the es.unsafely_permit_handshake_from_incompatible_builds system property, and ensure that all nodes of the same version are running exactly the same build. Setting this system property will result in an error on startup.

Details
Elasticsearch’s JSON logs now comply with the Elastic Common Schema (ECS). Previously, Elasticsearch’s JSON logs used a custom schema.

Impact
If your application parses Elasticsearch’s JSON logs, update it to support the new ECS format.

Details
Elasticsearch no longer emits a plaintext version of the following logs:

These logs are now only available in JSON.

Server logs are still available in both a JSON and plaintext format.

Impact
If your application parses Elasticsearch’s plaintext logs, update it to use the new ECS JSON logs.

Details
In addition to the existing daily rollover, the security audit logs are now rolled-over by disk size limit as well. Moreover, the rolled-over logs are also gzip compressed.

Impact
The names of rolled over audit log files (but not the name of the current log) have changed. If you’ve set up automated tools to consume these files, you must configure them to use the new names and to possibly account for gzip archives instead of plain text. The Docker build of Elasticsearch is not affected because it logs on stdout, where rollover is not performed.

Details
Elasticsearch 8.0 can read indices created in version 7.0 or above. An Elasticsearch 8.0 node will not start in the presence of indices created in a version of Elasticsearch before 7.0.

Impact
Reindex indices created in Elasticsearch 6.x or before with Elasticsearch 7.x if they need to be carried forward to Elasticsearch 8.x.

Details
In earlier versions a node would start up even if it had data from indices created in a version before the previous major version, as long as those indices were closed. Elasticsearch now ensures that it is compatible with every index, open or closed, at startup time.

Impact
Reindex closed indices created in Elasticsearch 6.x or before with Elasticsearch 7.x if they need to be carried forward to Elasticsearch 8.x.

Details
The number of completion contexts within a single completion field has been limited to 10.

Impact
Use a maximum of 10 completion contexts in a completion field. Specifying more than 10 completion contexts will return an error.

Details
Previously, it was possible to define a multi-field within a multi-field. Defining chained multi-fields was deprecated in 7.3 and is now no longer supported.

Impact
To migrate mappings, all instances of fields that occur within a fields block should be removed, either by flattening the chained fields blocks into a single level, or by switching to copy_to if appropriate.

Details
The setting has been deprecated with 7.5 and is no longer supported on new indices. Mappings for older indices will continue to work but emit a deprecation warning.

Impact
The enabled setting for _field_names should be removed from templates and mappings. Disabling _field_names is not necessary because it no longer carries a large index overhead.

Details
Index-time boosts have been deprecated since the 5x line, but it was still possible to declare field-specific boosts in the mappings. This is now removed completely. Indexes built in 7x that contain mapping boosts will emit warnings, and the boosts will have no effect in 8.0. New indexes will not permit boosts to be set in their mappings at all.

Impact
The boost setting should be removed from templates and mappings. Use boosts directly on queries instead.

Details
In 7.0, Elasticsearch switched from joda time to java time for date-related parsing, formatting, and calculations. Indices created in 7.0 and later versions are already required to use mappings with java-time date formats. However, earlier indices using joda-time formats must be reindexed to use mappings with java-time formats.

Impact
For a detailed migration guide, see the Java time migration guide.

Details
The following geo_shape mapping parameters were deprecated in 6.6:

These parameters have been removed in 8.0.0.

Impact
In 8.0, you can no longer create mappings that include these parameters. However, 7.x indices that use these mapping parameters will continue to work.

Details
The sparse_vector field type was deprecated in 7.6 and is now removed in 8.0. We have not seen much interest in this experimental field type, and don’t see a clear use case as it’s currently designed. If you have feedback or suggestions around sparse vector functionality, please let us know through GitHub or the discuss forums.

Impact
Discontinue use of the sparse_vector field data type. Requests containing a mapping for this field data type will return an error.

Details
Each node’s data is now stored directly in the data directory set by the path.data setting, rather than in ${path.data}/nodes/0, because the removal of the node.max_local_storage_nodes setting means that nodes may no longer share a data path.

Impact
At startup, Elasticsearch will automatically migrate the data path to the new layout. This automatic migration will not proceed if the data path contains data for more than one node. You should move to a configuration in which each node has its own data path before upgrading.

If you try to upgrade a configuration in which there is data for more than one node in a data path then the automatic migration will fail and Elasticsearch will refuse to start. To resolve this you will need to perform the migration manually. The data for the extra nodes are stored in folders named ${path.data}/nodes/1, ${path.data}/nodes/2 and so on, and you should move each of these folders to an appropriate location and then configure the corresponding node to use this location for its data path. If your nodes each have more than one data path in their path.data settings then you should move all the corresponding subfolders in parallel. Each node uses the same subfolder (e.g. nodes/2) across all its data paths.

Details
The default Maxmind geoip databases that shipped by default with Elasticsearch have been removed. These databases are out dated and stale and using these databases will likely result in incorrect geoip lookups.

By default since 7.13, these pre-packaged geoip databases were used in case no database were specified in the config directory or before the geoip downloader downloaded the geoip databases. When the geoip database downloader completed downloading the new databases then these pre-packaged databases were no longer used.

Impact
If the geoip downloader is disabled and no geoip databases are provided in the config directory of each ingest node then the geoip processor will no longer perform geoip lookups and tag these documents with the fact that the requested database is no longer available.

After a cluster has been started and before the geoip downloader has completed downloading the most up to data databases, the geoip processor will not perform any geoip lookups and tag documents that the requested database is not available. After the geoip downloader has completed downloading the most up to data databases then the geoip processor will function as normal. The window of time that the geoip processor can’t do geoip lookups after cluster startup should be very small.

Details
As a transition from Joda datetime to Java datetime, scripting used an intermediate class called JodaCompatibleZonedDateTime. This class has been removed and is replaced by ZonedDateTime. Any use of casting to a JodaCompatibleZonedDateTime or use of method calls only available in JodaCompatibleZonedDateTime in a script will result in a compilation error, and may not allow the upgraded node to start.

Impact
Before upgrading, replace getDayOfWeek with getDayOfWeekEnum().value in any scripts. Any use of getDayOfWeek expecting a return value of int will result in a compilation error or runtime error and may not allow the upgraded node to start.

The following JodaCompatibleZonedDateTime methods must be replaced using ZonedDateTime methods prior to upgrade:

Details
In previous versions of Elasticsearch, in order to register a snapshot repository backed by Amazon S3, Google Cloud Storage (GCS) or Microsoft Azure Blob Storage, you first had to install the corresponding Elasticsearch plugin, for example repository-s3. These plugins are now included in Elasticsearch by default.

Impact
You no longer need to install the following plugins, and not should attempt to do so.

Elasticsearch and the elasticsearch-plugin CLI tool have been changed to tolerate attempted installation and removal of these plugins in order to avoid breaking any existing automation. In the future, attempting to install these plugins will be an error.

Specifically, the elasticsearch-plugin CLI tool will not fail if you attempt to install any of the above plugins, and will instead print a warning and skip the plugins. If any of these plugins are already installed, for example because you installed them when running an older version of Elasticsearch, then you can still remove them with elasticsearch-plugin. Attempting to remove them if they are not installed will succeed but print a warnings.

If you run Elasticsearch using Docker and you are managing plugins using a configuration file, then when Elasticsearch first starts after you upgrade it, it will remove the above plugins if they already installed. If any of these plugins are specified in your configuration file, Elasticsearch will ignore them and emit a warning log message.

Details
In previous versions of Elasticsearch, third-party plugins could implement the getRestHandlerWrapper method to intercept all REST requests to the node. A common use of this feature was to implement custom security plugins to replace the built-in security features. This extension point is no longer available to third-party plugins.

Impact
Some third-party plugins that were designed to work with earlier versions of Elasticsearch might not be compatible with Elasticsearch version 8.0 or later.

If you depend on any plugins that are not produced and supported by Elastic, check with the plugin author and ensure that the plugin is available for your target version of Elasticsearch before upgrading.

Details
In 7.0, we deprecated REST endpoints that contain _xpack in their path. These endpoints are now removed in 8.0. Each endpoint that was deprecated and removed is replaced with a new endpoint that does not contain _xpack. As an example, /{index}/_xpack/graph/_explore is replaced by /{index}/_graph/explore.

Impact
Use the replacement REST API endpoints. Requests submitted to the _xpack API endpoints will return an error.

Compatibility
When rest-api-compatibility is requested, any requests that include the`_xpack` prefix are rerouted to the corresponding URL without the _xpack prefix.

Details
Mapping types have been removed. API endpoints that contain a mapping type have also been removed. Use a typeless endpoint instead.

Impact
Update your application to use typeless REST API endpoints. Requests to endpoints that contain a mapping type will return an error.

Compatibility
When rest-api-compatibility is requested, if a request includes a custom mapping type it is ignored. The request is rerouted to the corresponding typeless URL. Custom mapping types in request bodies and type related HTTP parameters are ignored, and responses, where warranted, include _type : _doc.

Details
In 8.0+, Elastic supports searches from a local cluster to a remote cluster running:

Elastic also supports searches from a local cluster running the last minor version of a major version to a remote cluster running any minor version in the following major version. For example, a local 7.17 cluster can search any remote 8.x cluster.

For example, a local 8.0 cluster can search a remote 7.17 or any remote 8.x cluster. However, a search from a local 8.0 cluster to a remote 7.16 or 6.8 cluster is not supported.

Previously, we also supported searches on remote clusters running:

However, such searches can result in undefined behavior.

Impact
If you only run cross-cluster searches on remote clusters using the same or a newer version, no changes are needed.

If you previously searched remote clusters running an earlier version of Elasticsearch, see Ensure cross-cluster search support for recommended solutions.

A cross-cluster search using an unsupported configuration may still work. However, such searches aren’t tested by Elastic, and their behavior isn’t guaranteed.

Details
The terms aggregation no longer supports the _term key in order values. To sort buckets by their term, use _key instead.

Impact
Discontinue use of the _term order key. Requests that include a _term order key will return an error.

Compatibility
When rest-api-compatibility is requested, the _term order is ignored and key is used instead.

Details
The date_histogram aggregation no longer supports the _time key in order values. To sort buckets by their key, use _key instead.

Impact
Discontinue use of the _time order key. Requests that include a _time order key will return an error.

Compatibility
When rest-api-compatibility is requested, the _time order is ignored and _key is used instead.

Details
The moving_avg aggregation was deprecated in 6.4 and has been removed. To calculate moving averages, use the moving_fn aggregation instead.

Impact
Discontinue use of the moving_avg aggregation. Requests that include the moving_avg aggregation will return an error.

Details
If you specify the percents parameter with the percentiles aggregation, its values must be unique. Otherwise, an exception occurs.

Impact
Use unique values in the percents parameter of the percentiles aggregation. Requests containing duplicate values in the percents parameter will return an error.

Details
It is now an error to specify the interval parameter to the date_histogram aggregation or the https://www.elastic.co/guide/en/elasticsearch/reference/8.3/search-aggregations-bucket-composite-aggregation.html#_date_histogram[composite date_histogram source. Instead, please use either calendar_interval or fixed_interval as appropriate.

Impact
Uses of the interval parameter in either the date_histogram aggregation or the date_histogram composite source will now generate an error. Instead please use the more specific fixed_interval or calendar_interval parameters.

Compatibility
When rest-api-compatibility is requested, the interval parameter is adapted to either a fixed or calendar interval.

Details
The nGram and edgeNGram token filter names that have been deprecated since version 6.4 have been removed. Both token filters can only be used by their alternative names ngram and edge_ngram since version 7.0.

Impact
Use the equivalent ngram and edge_ngram token filters. Requests containing the nGram and edgeNGram token filter names will return an error.

Details
The nGram and edgeNGram tokenizer names haven been deprecated with 7.6 and are no longer supported on new indices. Mappings for indices created after 7.6 will continue to work but emit a deprecation warning. The tokenizer name should be changed to the fully equivalent ngram or edge_ngram names for new indices and in index templates.

Impact
Use the ngram and edge_ngram tokenizers. Requests to create new indices using the nGram and edgeNGram tokenizer names will return an error.

Details
The name of the in flight requests circuit breaker in log output and diagnostic APIs (such as the node stats API) changes from in_flight_requests to inflight_requests to align it with the name of the corresponding settings.

Impact
Update your workflow and applications to use the inflight_requests stat in place of in_flight_requests.

Details
The POST /_cluster/voting_config_exclusions/{node_filter} API has been removed in favour of POST /_cluster/voting_config_exclusions?node_names=... and POST /_cluster/voting_config_exclusions?node_ids=... which allow you to specify the names or IDs of the nodes to exclude.

Impact
Use POST /_cluster/voting_config_exclusions?node_ids=... and specify the nodes to exclude instead of using a node filter. Requests submitted to the /_cluster/voting_config_exclusions/{node_filter} endpoint will return an error.

Details
Remote system indices matching an auto-follow pattern won’t be configured as a follower index automatically.

Impact
Explicitly create a follower index to follow a remote system index if that’s the wanted behaviour.

Details
The wildcard function was deprecated in Elasticsearch 7.13.0 and has been removed.

Impact
Use the like or regex keywords instead.

Details
The ILM freeze action is now a no-op and performs no action on the index, as the freeze API endpoint has been removed in 8.0.

Impact
Update your ILM policies to remove the freeze action from the cold phase.

Details
Creating or updating an ILM policy now requires that any referenced snapshot repositories and SLM policies exist.

Impact
Update your code or configuration management to ensure that repositories and SLM policies are created before any policies that reference them.

Details
Previously, the _upgrade API upgraded indices from the previous major version to the current version. The _reindex API should be used instead for that purpose.

Impact
Requests made to the old _upgrade API will return an error.

Details
The freeze index API (POST /<index>/_freeze) has been removed. Improvements in heap memory usage have eliminated the reason to freeze indices. You can still unfreeze existing frozen indices using the unfreeze index API. For some use cases, the frozen tier may be a suitable replacement for frozen indices. See data tiers for more information.

Impact
Requests made to the old freeze index API will return an error.

Details
Previously, the force merge API allowed the parameters only_expunge_deletes and max_num_segments to be set to a non default value at the same time. But the max_num_segments was silently ignored when only_expunge_deletes is set to true, leaving the false impression that it has been applied.

Impact
When using the force merge API, do not specify values for both the max_num_segments and only_expunge_deletes parameters. Requests that include values for both parameters will return an error.

Details
In 6.0, we deprecated the template parameter in create or update index template requests in favor of using index_patterns. Support for the template parameter is now removed in 8.0.

Impact
Use the create or update index template API's index_patterns parameter. Requests that include the template parameter will return an error.

Compatibility
When rest-api-compatibility is requested, the template parameter is mapped to index_patterns.

Details
Synced flush was deprecated in 7.6 and is removed in 8.0. Use a regular flush instead as it has the same effect as a synced flush in 7.6 and later.

Impact
Use the flush API. Requests to the /<index>/flush/synced or /flush/synced endpoints will return an error.

Compatibility
When rest-api-compatibility is requested, the request to synced flush is routed to the equivalent non-synced flush URL.

Details
When closing an index in earlier versions, by default Elasticsearch would not wait for the shards of the closed index to be properly assigned before returning. From version 8.0 onwards the default behaviour is to wait for shards to be assigned according to the index.write.wait_for_active_shards index setting.

Impact
Accept the new behaviour, or specify ?wait_for_active_shards=0 to preserve the old behaviour if needed.

Details
The index stats API’s types query parameter has been removed. Previously, you could combine types with the indexing query parameter to return indexing stats for specific mapping types. Mapping types have been removed in 8.0.

Impact
Discontinue use of the types query parameter. Requests that include the parameter will return an error.

Compatibility
When rest-api-compatibility is requested, the types query parameter is ignored and stats are returned for the entire index.

Details
In 7.2, we deprecated the ecs parameter for the user_agent ingest processor. In 8.x, the user_agent ingest processor will only return Elastic Common Schema (ECS) fields, regardless of the ecs value.

Impact
To avoid deprecation warnings, remove the parameter from your ingest pipelines. If a pipeline specifies an ecs value, the value is ignored.

Details
The include_type_name query parameter has been removed from the index creation, index template, and mapping APIs. Previously, you could set include_type_name to true to indicate that requests and responses should include a mapping type name. Mapping types have been removed in 8.x.

Impact
Discontinue use of the include_type_name query parameter. Requests that include the parameter will return an error.

Compatibility
When rest-api-compatibility is requested, the include_type_name query parameter is ignored and any custom mapping types in the request are removed.

Details
Reindex from remote would previously allow URL-encoded index names and not re-encode them when generating the search request for the remote host. This leniency has been removed such that all index names are correctly encoded when reindex generates remote search requests.

Impact
Specify unencoded index names for reindex from remote requests.

Details
Previously, a _reindex request had two different size specifications in the body:

The outer level size parameter has now been renamed to max_docs to avoid confusion and clarify its semantics.

Similarly, the size parameter has been renamed to max_docs for _delete_by_query and _update_by_query to keep the 3 interfaces consistent.

Impact
Use the replacement parameters. Requests containing the size parameter will return an error.

Compatibility
When rest-api-compatibility is requested, the size parameter is mapped to the max_docs parameter.

Details
An update by query API request that includes an unsupported field in the script object now returns an error. Previously, the API would silently ignore these unsupported fields.

Impact
To avoid errors, remove unsupported fields from the script object.

Details
The ?local parameter to the GET _cat/nodes API was deprecated in 7.x and is rejected in 8.0. This parameter caused the API to use the local cluster state to determine the nodes returned by the API rather than the cluster state from the master, but this API requests information from each selected node regardless of the ?local parameter which means this API does not run in a fully node-local fashion.

Impact
Discontinue use of the ?local query parameter. cat node API requests that include this parameter will return an error.

Details
The ?local parameter to the GET _cat/shards API was deprecated in 7.x and is rejected in 8.0. This parameter caused the API to use the local cluster state to determine the nodes returned by the API rather than the cluster state from the master, but this API requests information from each selected node regardless of the ?local parameter which means this API does not run in a fully node-local fashion.

Impact
Discontinue use of the ?local query parameter. cat shards API requests that include this parameter will return an error.

Details
The ?local parameter to the GET _cat/indices API was deprecated in 7.x and is rejected in 8.0. This parameter caused the API to use the local cluster state to determine the nodes returned by the API rather than the cluster state from the master, but this API requests information from each selected node regardless of the ?local parameter which means this API does not run in a fully node-local fashion.

Impact
Discontinue use of the ?local query parameter. cat indices API requests that include this parameter will return an error.

Details
The local parameter for get field mapping API was deprecated in 7.8 and is removed in 8.0. This parameter is a no-op and field mappings are always retrieved locally.

Impact
Discontinue use of the local query parameter. get field mapping API requests that include this parameter will return an error.

Details
The machine learning post data to jobs API is deprecated starting in 7.11.0 and will be removed in a future major version.

Impact
Use datafeeds instead.

Details
The ability to update a job_id in a datafeed was deprecated in 7.3.0. and is removed in 8.0.

Impact
It is not possible to move datafeeds between anomaly detection jobs.

Details
The Create or update snapshot repository API and Delete snapshot repository API return 409 status code when the request is attempting to modify an existing repository that’s in use instead of status code 500.

Impact
Update client code that handles creation and deletion of repositories to reflect this change.

Details
The allow_no_datafeeds property was deprecated in the cat datafeeds, get datafeeds, get datafeed statistics, and stop datafeeds APIs in 7.10.0.

Impact
Use allow_no_match instead.

Details
The allow_no_jobs property was deprecated in the cat anomaly detectors, close anomaly detection jobs, get anomaly detection jobs, get anomaly detection job statistics, and get overall buckets APIs in 7.10.0.

Impact
Use allow_no_match instead.

Details
Previously, attempting to start an already-started rollup job would result in a 500 InternalServerError: Cannot start task for Rollup Job [job] because state was [STARTED] exception.

Now, attempting to start a job that is already started will just return a successful 200 OK: started response.

Impact
Update your workflow and applications to assume that a 200 status in response to attempting to start a rollup job means the job is in an actively started state. The request itself may have started the job, or it was previously running and so the request had no effect.

Details
The create or update stored script API's source parameter cannot be empty.

Impact
Before upgrading, use the delete stored script API to delete any empty stored scripts or search templates. In 8.0, Elasticsearch will drop any empty stored scripts or empty search templates from the cluster state. Requests to create a stored script or search template with an empty source will return an error.

Details
The create or update stored script API's code parameter has been removed. Use the source parameter instead.

Impact
Discontinue use of the code parameter. Requests that include the parameter will return an error.

Details
In 8.x, the _type metadata field has been removed. Elasticsearch now handles a search on the _type field as a search on a non-existent field. A search on a non-existent field matches no documents, regardless of the query string.

In 7.x, a search for _doc in the _type field would match the same documents as a match_all query.

Impact
Remove queries on the _type field from your search requests and search templates. Searches that include these queries may return no results.

Details
The multi search API now parses an empty first line as empty action metadata when you provide a text file as the request body, such as when using curl’s --data-binary flag.

The API no longer supports text files that contain:

Impact
Don’t provide an unsupported text file to the multi search API. Requests that include an unsupported file will return an error.

Details
Search requests no longer support the unmapped_type: string sort option. Instead, use unmapped_type: keyword to handle an unmapped field as if it had the keyword field type but ignore its values for sorting.

Impact
Discontinue use of unmapped_type: string. Search requests that include the unmapped_type: string sort option will return shard failures.

Details
Previously, it was possible to aggregate and sort on the built-in _id field by loading an expensive data structure called fielddata. This was deprecated in 7.6 and is now disallowed by default in 8.0.

Impact
Aggregating and sorting on _id should be avoided. As an alternative, the _id field’s contents can be duplicated into another field with docvalues enabled (note that this does not apply to auto-generated IDs).

Details
The common query, deprecated in 7.x, has been removed in 8.0. The same functionality can be achieved by the match query if the total number of hits is not tracked.

Impact
Discontinue use of the common query. Search requests containing a common query will return an error.

Details
The cutoff_frequency parameter, deprecated in 7.x, has been removed in 8.0 from match and multi_match queries. The same functionality can be achieved without any configuration provided that the total number of hits is not tracked.

Impact
Discontinue use of the cutoff_frequency parameter. Search requests containing this parameter in a match or multi_match query will return an error.

Details
The nested_filter and nested_path options, deprecated in 6.x, have been removed in favor of the nested context.

Impact
Discontinue use of the sort request body parameter’s nested_filter and nested_path properties. Requests containing these properties will return an error.

Details
Elasticsearch will no longer prefer using shards in the same location (with the same awareness attribute values) to process _search and _get requests. Adaptive replica selection (activated by default in this version) will route requests more efficiently using the service time of prior inter-node communications.

Impact
No action needed.

Details
The vector functions of the form function(query, doc['field']) were deprecated in 7.6, and are now removed in 8.x. The form function(query, 'field') should be used instead. For example, cosineSimilarity(query, doc['field']) is replaced by cosineSimilarity(query, 'field').

Impact
Use the function(query, 'field') form. Discontinue use of the function(query, doc['field']) form. Requests containing the function(query, doc['field']) form will return an error.

Details
The indices_boost option in the search request used to accept the boosts both as an object and as an array. The object format has been deprecated since 5.2 and is now removed in 8.0.

Impact
Use only array values in the indices_boost parameter. Requests containing an object value in the indices_boost parameter will return an error.

Details
In 7.0, we began formatting docvalue_fields by default using each field’s mapping definition. To ease the transition from 6.x, we added the format option use_field_mapping. This parameter was deprecated in 7.0, and is now removed in 8.0.

Impact
Discontinue use of the use_field_mapping request body parameter. Requests containing this parameter will return an error.

Compatibility
When rest-api-compatibility is requested, the use_field_mapping parameter is ignored.

Details
Search request used to accept -1 as a from in the search body and the url, treating it as the default value of 0. Other negative values got rejected with an error already. We now also reject -1 as an invalid value.

Impact
Change any use of -1 as from parameter in request body or url parameters by either setting it to 0 or omitting it entirely. Requests containing negative values will return an error.

Details
Range queries on date fields used to misinterpret small numbers (e.g. four digits like 1000) as a year when no additional format was set, but would interpret other numeric values as milliseconds since epoch. We now treat all numeric values in absence of a specific format parameter as milliseconds since epoch. If you want to query for years instead, with a missing format you now need to quote the input value (e.g. "1984").

Impact
If you query date fields without a specified format, check if the values in your queries are actually meant to be milliseconds-since-epoch and use a numeric value in this case. If not, use a string value which gets parsed by either the date format set on the field in the mappings or by strict_date_optional_time by default.

Details
The geo_bounding_box query’s type parameter was deprecated in 7.14.0 and has been removed in 8.0.0. This parameter is a no-op and has no effect on the query.

Impact
Discontinue use of the type parameter. geo_bounding_box queries that include this parameter will return an error.

Details
The type query has been removed. Mapping types have been removed in 8.0.

Impact
Discontinue use of the type query. Requests that include the type query will return an error.

Details
Users who were previously assigned the kibana_user role should instead be assigned the kibana_admin role. This role grants the same set of privileges as kibana_user, but has been renamed to better reflect its intended use.

Impact
Assign users with the kibana_user role to the kibana_admin role. Discontinue use of the kibana_user role.

Details
For snapshot and SLM APIs, the indices parameter no longer resolves to system indices or system data streams. Feature states are now the only way to back up and restore system indices or system data streams from a snapshot.

You can no longer use the indices parameter for the create SLM policy API or the create snapshot API to include a system index in a snapshot. To back up a system index, use the include_global_state and feature_states parameters to include the corresponding feature state instead. By default, the include_global_state and feature_states parameters include all system indices.

Similarly, you can no longer use the restore snapshot API's indices parameter to restore a system index from a snapshot. To restore a system index, use the include_global_state and feature_states parameters to restore the corresponding feature state instead. By default, the include_global_state and feature_states parameters don’t restore any system indices.

Impact
If you previously used the indices parameter to back up or restore system indices, update your SLM policies and application to use the include_global_state and feature_states parameters instead.

An SLM policy that explicitly specifies a system index in the indices parameter will fail to create snapshots. Similarly, a create snapshot API or restore snapshot API request that explicitly specifies a system index in the indices parameter will fail and return an error. If the indices value includes a wildcard (*) pattern, the pattern will no longer match system indices.

Details
Previously, the default value for compress was false. The default has been changed to true.

This change will affect both newly created repositories and existing repositories where compress=false has not been explicitly specified.

Impact
Update your workflow and applications to assume a default value of true for the compress parameter.

Details
Starting in version 7.4, s3 snapshot repositories no longer use the now-deprecated path-style access pattern by default. In versions 7.0, 7.1, 7.2 and 7.3 s3 snapshot repositories always used the path-style access pattern. This is a breaking change for deployments that only support path-style access but which are recognized as supporting DNS-style access by the AWS SDK. This breaking change was made necessary by AWS’s announcement that the path-style access pattern is deprecated and will be unsupported on buckets created after September 30th 2020.

Impact
If your deployment only supports path-style access and is affected by this change then you must configure the S3 client setting path_style_access to true.

Details
In earlier versions, you could pass both settings and index_settings in the body of a restore snapshot request, but the settings value was ignored. The restore snapshot API now rejects requests that include a settings value.

Impact
Discontinue use of the settings parameter in restore snapshot request. Requests that include these parameters will return an error.

Details
The repository stats API has been removed. We deprecated this experimental API in 7.10.0.

Impact
Use the repositories metering APIs instead.

Details
In 8.x, Elasticsearch writes Watcher history to a hidden .watcher-history-<index-template-version> data stream. Previously, Elasticsearch wrote Watcher history to hidden .watcher-history-<index-template-version>-<yyyy-MM-dd> indices.

Impact
Update your requests to target the Watcher history data stream. For example, use the .watcher-history-* wildcard expression. Requests that specifically target non-existent Watcher history indices may return an error.

Details
The cluster health API includes options for waiting for certain health conditions to be satisfied. If the requested conditions are not satisfied within a timeout then Elasticsearch will send back a normal response including the field "timed_out": true. In earlier versions it would also use the HTTP response code 408 Request timeout if the request timed out, and 200 OK otherwise. The 408 Request timeout response code is not appropriate for this situation, so from version 8.0.0 Elasticsearch will use the response code 200 OK for both cases.

Impact
To detect a server timeout, check the timed_out field of the JSON response.

Details
The Content-Type response header no longer specifies the charset. This information is not required when transferring JSON data, because JSON text will always be encoded in Unicode, with UTF-8 being the default encoding.

Impact
Some applications and utilities, such as PowerShell’s Invoke-RestMethod, must receive charset information to display data correctly. If your application or utility relies on charset information in the Content-Type response header, UTF-8 encoded characters will be rendered incorrectly in the response body.

As a workaround, to render non-ASCII characters, include an HTTP Accept header in your requests, specifying the charset:

Accept: application/json; charset=utf-8

Details
To reduce the dependency of the JDBC driver onto Elasticsearch classes, the JDBC driver returns geometry data as strings using the WKT (well-known text) format instead of classes from the org.elasticsearch.geometry. Users can choose the geometry library desired to convert the string representation into a full-blown objects either such as the elasticsearch-geo library (which returned the object org.elasticsearch.geo as before), jts or spatial4j.

Impact
Before upgrading, replace any org.elasticsearch.geo classes on the ResultSet#getObject or ResultSet#setObject Elasticsearch JDBC driver with their WKT representation by simply calling toString or org.elasticsearch.geometry.utils.WellKnownText#toWKT/fromWKT methods.

This change does NOT impact users that do not use geometry classes.

Details
The following operating systems have reached their end of life and are no longer supported by Elasticsearch:

We’ve also removed support for SysV init. No supported operating systems use the SysV init process.

Details
Ensure your nodes use a supported operating system. Running Elasticsearch on an unsupported operating system can result in unexpected errors or failures.

Details
Java 17 or higher is now required to run Elasticsearch and any of its command line tools.

Impact
Use Java 17 or higher. Attempts to run Elasticsearch 8.0 using earlier Java versions will fail.

There is not yet a FIPS-certified security module for Java 17 that you can use when running Elasticsearch 8.0 in FIPS 140-2 mode. If you run in FIPS 140-2 mode, you will either need to request an exception from your security organization to upgrade to Elasticsearch 8.0, or remain on Elasticsearch 7.x until Java 17 is certified.

Details
JAVA_HOME is no longer supported to set the path for the JDK. Instead, use the bundled JDK (preferable), or set ES_JAVA_HOME.

Impact
Use the bundled JDK (preferable), or set ES_JAVA_HOME. JAVA_HOME will be ignored.

Details
Early beta versions of transforms had configuration information in a format that is no longer supported.

Impact
Use the upgrade transforms API to fix your transforms. This upgrade does not affect the source or destination indices.

Details
We no longer recommend using transient cluster settings. Use persistent cluster settings instead. If a cluster becomes unstable, transient settings can clear unexpectedly, resulting in an undesired cluster configuration.

Impact
Transient cluster settings are not yet deprecated, but we plan to deprecate them in a future release. For migration steps, see the Transient settings migration guide.

Details
The elasticsearch-setup-passwords tool is deprecated in 8.0. To manually reset the password for built-in users (including the elastic user), use the elasticsearch-reset-password tool, the Elasticsearch change passwords API, or the User Management features in Kibana. elasticsearch-setup-passwords will be removed in a future release.

Impact
Passwords are generated automatically for the elastic user when you start Elasticsearch for the first time. If you run elasticsearch-setup-passwords after starting Elasticsearch, it will fail because the elastic user password is already configured.