New

The executive guide to generative AI

Read more
About usPartnersSupport|Login
IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Elasticsearch version 7.10.1 Elasticsearch version 7.9.3 »
Elastic Docs ›Elasticsearch Guide [7.11] ›Release notes

Elasticsearch version 7.10.0

edit

Elasticsearch version 7.10.0

edit

Also see Breaking changes in 7.10.

Security updates

edit
  • Elasticsearch versions before 7.10.0 and 6.8.14 have an information disclosure issue when audit logging and the emit_request_body option are enabled. The Elasticsearch audit log could contain sensitive information, such as password hashes or authentication tokens. This could allow an Elasticsearch administrator to view these details. You must upgrade to Elasticsearch version 7.10.0 to obtain the fix. CVE-2020-7021

Known issues

edit

  • SQL: If a WHERE clause contains at least two relational operators joined by AND, of which one is a comparison (<=, <, >=, >) and another one is an inequality (!=, <>), both against literals or foldable expressions, the inequality will be ignored. The workaround is to substitute the inequality with a NOT IN operator.

    We have fixed this issue in Elasticsearch 7.10.1 and later versions. For more details, see #65488.

  • There were several regressions around loading nested documents. These have been addressed in Elasticsearch 7.10.2.

    • With multiple layers of nested inner_hits, we can fail to load the _source. (#66577)
    • With nested inner_hits, the fast vector highlighter may load snippets from the wrong document. (#65533)
    • When _source is disabled, we can fail load nested inner_hits and top_hits. (#66572)

  • Snapshot and restore: If an index is deleted while the cluster is concurrently taking more than one snapshot then there is a risk that one of the snapshots may never complete and also that some shard data may be lost from the repository, causing future restore operations to fail. To mitigate this problem, prevent concurrent snapshot operations by setting snapshot.max_concurrent_operations: 1.

    This issue is fixed in Elasticsearch versions 7.13.1 and later. For more details, see #73456.

Breaking changes

edit
Authentication
  • API key name should always be required for creation #59836 (issue: #59484)
Network
  • Set specific keepalive options by default on supported platforms #59278

Breaking Java changes

edit
Machine Learning
  • Deprecate allow_no_jobs and allow_no_datafeeds in favor of allow_no_match #60601 (issue: #60642)
Mapping
  • Pass SearchLookup supplier through to fielddataBuilder #61430 (issue: #59332)

Deprecations

edit
Cluster Coordination
Machine learning
  • Renames */inference* APIs to */trained_models* #63097

New features

edit
Aggregations
Features/Features
  • Add data tiers (hot, warm, cold, frozen) as custom node roles #60994 (issue: #60848)
  • Allocate newly created indices on data_hot tier nodes #61342 (issue: #60848)
Features/ILM+SLM
Machine Learning
  • Implement AucRoc metric for classification #60502 (issue: #62160)
Mapping
  • Introduce 64-bit unsigned long field type #60050 (issue: #32434)
Search
  • Add search fields option to support high-level field retrieval #60100 (issues: #49028, #55363)

Enhancements

edit
Aggregations
  • Adds hard_bounds to histogram aggregations #59175 (issue: #50109)
  • Allocate slightly less per bucket #59740
  • Improve reduction of terms aggregations #61779 (issue: #51857)
  • Speed up date_histogram by precomputing ranges #61467
Analysis
  • Support ignore_keywords flag for word delimiter graph token filter #59563 (issue: #59491)
Authentication
  • Cache API key doc to reduce traffic to the security index #59376 (issues: #53940, #55836)
  • Include authentication type for the authenticate response #61247 (issue: #61130)
  • Oidc additional client auth types #58708
  • Warn about unlicensed realms if no auth token can be extracted #61402 (issue: #61090)
Authorization
  • Add DEBUG logging for undefined role mapping field #61246 (issue: #48562)
  • Add more context to index access denied errors #60357 (issue: #42166)
CRUD
  • Add configured indexing memory limit to node stats #60342
  • Dedicated threadpool for system index writes #61655
Cluster Coordination
  • Add more useful toString on cluster state observers #60277
  • Fail invalid incremental cluster state writes #61030
  • Provide option to allow writes when master is down #60605
Distributed
  • Detect noop of update index settings #61348
  • Thread safe clean up of LocalNodeModeListeners #59932 (issue: #59801)
Features/CAT APIs
  • Adding Hit counts and Miss counts for QueryCache exposed through REST API #60114 (issue: #48645)
Features/Features
  • Add aggregation list to node info #60074 (issue: #52057)
  • Adding new require_alias option to indexing requests #58917 (issue: #55267)
Features/ILM+SLM
  • Move internal index templates to composable templates #61457
Features/Ingest
  • Add network from MaxMind Geo ASN database #61676
  • Allow_duplicates option for append processor #61916 (issue: #57543)
  • Configurable output format for date processor #61324 (issue: #42523)
  • Enhance the ingest node simulate verbose output #60433 (issue: #56004)
  • Per processor description for verbose simulate #58207 (issue: #57906)
  • Preserve grok pattern ordering and add sort option #61671 (issue: #40819)
Features/Java High Level REST Client
  • HLRC: UpdateByQuery API with wait_for_completion being false #58552 (issues: #35202, #46350)
Infra/Core
  • Add logstash system index APIs #53350
  • Deprecate REST access to System Indices #60945
  • Speed up Compression Logic by Pooling Resources #61358
  • System index reads in separate threadpool #57936 (issues: #37867, #50251)
Infra/Logging
Infra/Packaging
  • Add UBI docker builds #60742
  • Upgrade Centos version in Dockerfile to 8 #59019
Infra/Resiliency
Infra/Scripting
Machine Learning
  • Add a "verbose" option to the data frame analytics stats endpoint #59589 (issue: #59125)
  • Add new include flag to get trained models API for model training metadata #61922
  • Add new feature_processors field for data frame analytics #60528 (issue: #59327)
  • Add new n_gram_encoding custom processor #61578
  • During nightly maintenance delete jobs whose original deletion tasks were lost #60121 (issue: #42840)
  • Suspend persistence of trained model stats when ML upgrade mode is enabled #61143
  • Calculate total feature importance to store with model metadata #1387
  • Change outlier detection feature_influence format to array with nested objects #1475, #62068
  • Add timeouts to named pipe connections #1514, #62993 (issue: #1504)
Mapping
Network
  • Improve deserialization failure logging #60577 (issue: #38939)
  • Log and track open/close of transport connections #60297
Performance
  • Speed up empty highlighting many fields #61860
SQL
  • Add option to provide the delimiter for the CSV format #59907 (issue: #41634)
  • Implement DATE_PARSE function for parsing strings into DATE values #57391 (issue: #54962)
  • Implement FORMAT function #55454 (issue: #54965)
Search
  • Avoid reloading _source for every inner hit #60494 (issue: #32818)
  • Cancel multisearch when http connection closed #61399
  • Enable cancellation for msearch requests #61337
  • Executes incremental reduce in the search thread pool #58461 (issues: #51857, #53411)
  • Introduce point in time APIs in x-pack basic #61062 (issues: #26472, #46523)
  • ParametrizedFieldMapper to run validators against default value #60042 (issue: #59332)
  • Add case insensitive flag for "term" family of queries #61596 (issue: #61546)
  • Add case insensitive support for regex queries #59441
  • Tweak toXContent implementation of ParametrizedFieldMapper #59968
  • Implement fields value fetching for the text, search_as_you_type and token_count field types #63515
  • Make term/prefix/wildcard/regex query parsing more lenient, with respect to the case_insensitive flag #63926 (issue: #63893)
Snapshot/Restore
  • Add repositories metering API #60371
  • Clone Snapshot API #61839
  • Determine shard size before allocating shards recovering from snapshots #61906
  • Introduce index based snapshot blob cache for Searchable Snapshots #60522
  • Validate snapshot UUID during restore #59601 (issue: #50999)
Store
  • Report more details of unobtainable ShardLock #61255 (issue: #38807)
Transform

Bug fixes

edit
Aggregations
  • Fix AOOBE when setting min_doc_count to 0 in significant_terms #60823 (issues: #60683, #60824)
  • Make sure non-collecting aggs include sub-aggs #64214 (issue: #64142)
  • Composite aggregation must check live docs when the index is sorted #63864
  • Fix broken parent and child aggregator #63811
Allocation
  • Fix scheduling of ClusterInfoService#refresh #59880
Authorization
  • Fix doc-update interceptor for indices with DLS and FLS #61516
  • Report anonymous roles in authenticate response #61355 (issues: #47195, #53453, #57711, #57853)
  • Add view_index_metadata privilege over metricbeat-* for monitoring agent #63750 (issue: #63750)
CRUD
  • Propagate forceExecution when acquiring permit #60634 (issue: #60359)
Cluster Coordination
  • Reduce allocations when persisting cluster state #61159
Distributed
  • Fix cluster health rest api wait_for_no_initializing_shards bug #58379
  • Fix cluster health when closing #61709
Engine
  • Fix estimate size of translog operations #59206
Features/ILM+SLM
Features/Java Low Level REST Client
  • Handle non-default port in Cloud-Id #61581
Features/Stats
  • Remove sporadic min/max usage estimates from stats #59755
Features/Watcher
  • Correct the query dsl for watching elasticsearch version #58321 (issue: #58261)
  • Fix passing params to template or script failed in watcher #58559 (issue: #57625)
Geo
Infra/Core
  • Throws IndexNotFoundException in TransportGetAction for unknown System indices #61785 (issue: #57936)
  • Handle missing logstash index exceptions #63698
  • XPack Usage API should run on MANAGEMENT threads #64160
Infra/Packaging
  • Allow running the Docker image with a non-default group #61194 (issue: #60864)
  • Set the systemd initial timeout to 75 seconds #60345 (issue: #60140)
Machine Learning
  • Adjusting inference processor to support foreach usage #60915 (issue: #60867)
  • Get data frame analytics jobs stats API can return multiple responses if more than one error #60900 (issue: #60876)
  • Do not mark the data frame analytics job as FAILED when a failure occurs after the node is shutdown #61331 (issue: #60596)
  • Improve handling of exception while starting data frame analytics process #61838 (issue: #61704)
  • Fix progress on resume after final training has completed for classification and regression. Previously, progress was shown stuck at zero for final training. #1443
  • Avoid potential "Failed to compute quantile" and "No values added to quantile sketch" log errors training regression and classification models if there are features with mostly missing values #1500
  • Correct the anomaly detection job model state min_version #1546
Mapping
  • Improve ignore_malformed handling for dates #60211 (issue: #52634)
Network
  • Let isInetAddress utility understand the scope ID on ipv6 #60172 (issue: #60115)
  • Suppress noisy SSL exceptions #61359
Search
  • Allows nanosecond resolution in search_after #60328 (issue: #52424)
  • Consolidate validation for docvalue_fields #59473
  • Correct how field retrieval handles multifields and copy_to #61309 (issue: #61033)
  • Apply boost only once for distance_feature query #63767
  • Fixed NullPointerException in significant_text aggregation when field does not exist #64144 (issue: #64045)
  • Fix async search to retry updates on version conflict #63652 (issue: #63213)
  • Fix sorted query when date_nanos is used as the numeric_type #64183 (issue: #63719)
Snapshot/Restore
  • Avoid listener call under SparseFileTracker#mutex #61626 (issue: #61520)
  • Ensure repo not in use for wildcard repo deletes #60947
  • Fix Test Failure in testCorrectCountsForDoneShards #60254 (issue: #60247)
  • Minimize cache file locking during prewarming #61837 (issue: #58658)
  • Prevent snapshots to be mounted as system indices #61517 (issue: #60522)
  • Make Searchable Snapshot’s CacheFile Lock less #63911 (issue: #63586)
  • Don’t Generate an Index Setting History UUID unless it’s supported #64213 (issue: #64152)
SQL
  • Allow unescaped wildcard (*) in LIKE pattern #63428 (issue: #55108)
  • Validate integer paramete in string functions #63728 (issue: #58923)
  • Remove filter from field_caps requests #63840 (issue: #63832)

Upgrades

edit
Infra/Packaging
  • Upgrade bundled JDK to 15.0.1 and switch to AdoptOpenJDK #64253
Store
  • Upgrade to Lucene-8.7.0 #64532
« Elasticsearch version 7.10.1 Elasticsearch version 7.9.3 »