Elasticsearch breaking changes
editElasticsearch breaking changes
editThis list summarizes the most important breaking changes in Elasticsearch 7.0.1. For the complete list, go to Elasticsearch breaking changes.
Indices created before 7.0
editElasticsearch 7.0 can read indices created in version 6.0 or above. An Elasticsearch 7.0 node will not start in the presence of indices created in a version of Elasticsearch before 6.0.
Reindex indices from Elasticsearch 5.x or before
Indices created in Elasticsearch 5.x or before will need to be reindexed with Elasticsearch 6.x in order to be readable by Elasticsearch 7.x.
search.max_buckets
in the cluster setting
editThe dynamic cluster setting named search.max_buckets
now defaults
to 10,000 (instead of unlimited in the previous version).
Requests that try to return more than the limit will fail with an exception.
Internal Versioning is no longer supported for optimistic concurrency control
editElasticsearch maintains a numeric version field for each document it stores. That field is incremented by one with every change to the document. Until 7.0.0 the API allowed using that field for optimistic concurrency control, i.e., making a write operation conditional on the current document version. Sadly, that approach is flawed because the value of the version doesn’t always uniquely represent a change to the document. If a primary fails while handling a write operation, it may expose a version that will then be reused by the new primary.
Due to that issue, internal versioning can no longer be used and is replaced by a new method based on sequence numbers. See Optimistic concurrency control for more details.
Note that the external
versioning type is still fully supported.
Shard preferences _primary
, _primary_first
, _replica
, and _replica_first
are removed
editThese shard preferences are removed in favour of the _prefer_nodes
and _only_nodes
preferences.
Cluster-wide shard soft limit
editClusters now have soft limits on the total number of open shards in the cluster
based on the number of nodes and the cluster.max_shards_per_node
cluster
setting, to prevent accidental operations that would destabilize the cluster.
More information can be found in the
documentation for that setting.
Cluster bootstrapping is required if discovery is configured
editThe first time a cluster is started, cluster.initial_master_nodes
must be set
to perform cluster bootstrapping. It should contain the names of the
master-eligible nodes in the initial cluster and be defined on every
master-eligible node in the cluster. See
the discovery settings summary for an example, and the
cluster bootstrapping reference
documentation describes this setting in more detail.
The discovery.zen.minimum_master_nodes
setting is permitted, but ignored, on
7.x nodes.
Discovery configuration is required in production
editProduction deployments of Elasticsearch now require at least one of the
following settings to be specified in the elasticsearch.yml
configuration
file:
-
discovery.seed_hosts
-
discovery.seed_providers
-
cluster.initial_master_nodes
-
discovery.zen.ping.unicast.hosts
-
discovery.zen.hosts_provider
The first three settings in this list are only available in versions 7.0 and
above. If you are preparing to upgrade from an earlier version, you must set
discovery.zen.ping.unicast.hosts
or discovery.zen.hosts_provider
.
Index creation no longer defaults to five shards
editPrevious versions of Elasticsearch defaulted to creating five shards per index. Starting with 7.0.0, the default is now one shard per index.
Ingest configuration exception information is now transmitted in metadata field
editPreviously, some ingest configuration exception information about ingest processors was sent to the client in the HTTP headers, which is inconsistent with how exceptions are conveyed in other parts of Elasticsearch.
Configuration exception information is now conveyed as a field in the response body.
Replacing Joda-Time with java time
editSince Java 8 there is a dedicated java.time
package, which is superior to
the Joda-Time library, that has been used so far in Elasticsearch. One of
the biggest advantages is the ability to be able to store dates in a higher
resolution than milliseconds for greater precision. Also this will allow us
to remove the Joda-Time dependency in the future.
The mappings, aggregations and search code switched from Joda-Time to java time.
The _default_
mapping is no longer allowed
editThe _default_
mapping has been deprecated in 6.0 and is now no longer allowed
in 7.0. Trying to configure a _default_
mapping on 7.x indices will result in
an error.
Tribe node removed
editTribe node functionality has been removed in favor of cross-cluster search.
hits.total
is now an object in the search response
editThe total hits that match the search request is now returned as an object
with a value
and a relation
. value
indicates the number of hits that
match and relation
indicates whether the value is accurate (eq
) or a lower bound
(gte
):
{ "hits": { "total": { "value": 1000, "relation": "eq" }, ... } }
The total
object in the response indicates that the query matches exactly 1000
documents ("eq"). The value
is always accurate ("relation": "eq"
) when
track_total_hits
is set to true in the request.
You can also retrieve hits.total
as a number in the rest response by adding
rest_total_hits_as_int=true
in the request parameter of the search request.
This parameter has been added to ease the transition to the new format and
will be removed in the next major version (8.0).
track_total_hits
defaults to 10,000
editBy default search request will count the total hits accurately up to 10,000
documents. If the total number of hits that match the query is greater than this
value, the response will indicate that the returned value is a lower bound:
{ "_shards": ... "timed_out": false, "took": 100, "hits": { "max_score": 1.0, "total" : { "value": 10000, "relation": "gte" }, "hits": ... } }
You can force the count to always be accurate by setting "track_total_hits
to true explicitly in the search request.
The default for node.name
is now the hostname
editnode.name
now defaults to the hostname at the time when Elasticsearch
is started. Previously the default node name was the first eight characters
of the node id. It can still be configured explicitly in elasticsearch.yml
.
Security realms settings
editThe settings for all security realms must now include the realm type as part
of the setting name, and the explicit type
setting has been removed.
A realm that was previous configured as:
xpack.security.authc.realms: ldap1: type: ldap order: 1 url: "ldaps://ldap.example.com/"
Must be migrated to:
xpack.security.authc.realms: ldap.ldap1: order: 1 url: "ldaps://ldap.example.com/"
Any realm specific secure settings that have been stored in the elasticsearch keystore (such as ldap bind passwords, or passwords for ssl keys) must be updated in a similar way.
TLS v1.0 disabled
editTLS version 1.0 is now disabled by default as it suffers from known security issues. The default protocols are now TLSv1.3 (if supported), TLSv1.2 and TLSv1.1.
You can enable TLS v1.0 by configuring the relevant ssl.supported_protocols
setting to include "TLSv1"
.
Depending on your local configuration and the TLS protocols that are in use
on your network, you may need to enable TLS v1.0 support in any or all of the
following places:
-
xpack.security.http.ssl.supported_protocols
- For incoming HTTP connections to Elasticsearch’s HTTP (Rest) interface. If there are clients that connect to Elasticsearch and do not support newer TLS versions, you must update this setting.
-
xpack.http.ssl.supported_protocols
- For outgoing HTTP connections from Watcher. If you have watches that connect to external HTTP servers and do not support newer TLS versions, you must update this setting.
-
xpack.security.authc.realms.ldap.{name}.ssl.supported_protocols
- For outgoing LDAP connections from Elasticsearch security features. If you have an LDAP realm enabled and the LDAP directory to which that realm connects does not support newer TLS versions, you must update this setting.
-
xpack.security.authc.realms.active_directory.{name}.ssl.supported_protocols
- For outgoing Active Directory (LDAP) connections from Elasticsearch security features. If you have an AD realm enabled and the directory server to which that realm connects does not support newer TLS versions, you must update this setting.
-
xpack.security.authc.realms.saml.{name}.ssl.supported_protocols
-
For outgoing HTTP connections to retrieve SAML metadata.
If you have a SAML realm enabled and the realm is configured to retrieve
metadata over HTTPS (that is,
idp.metadata.path
is a URL starting withhttps://
) and the web server which hosts the metadata does not support newer TLS versions, you must update this setting. -
xpack.security.authc.realms.oidc.{name}.ssl.supported_protocols
- For outgoing HTTP connections to an OpenId Connect Provider. If you have an OpenId Connect ("oidc") realm enabled and the realm is configured to connect to a remote OpenID Connect Provider which does not support newer TLS versions, you must update this setting.
-
xpack.monitoring.exporters.{name}.ssl.supported_protocols
- For remote monitoring data. If your monitoring data is exported to a remote monitoring cluster and that cluster is configured to only support TLSv1, you must update this setting.
-
reindex.ssl.supported_protocols
-
For reindex from remote.
If you reindex data from a remote Elasticsearch cluster which has SSL enabled on the
http
interface and that cluster is configured to only support TLSv1, you must update this setting. -
xpack.security.transport.ssl.supported_protocols
- For incoming connections between Elasticsearch nodes. If you have specialized network equipment which inspects TLS packets between your nodes, and that equipment enforces TLSv1 you must update this setting.
The following is an example that enables TLS v1.0 for incoming HTTP connections:
xpack.security.http.ssl.supported_protocols: [ "TLSv1.3", "TLSv1.2", "TLSv1.1", "TLSv1" ]