WARNING: Version 5.4 of Elasticsearch has passed its EOL date.

This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.

« 5.4.1 Release Notes 5.3.3 Release Notes »
Elastic Docs ›Elasticsearch Reference [5.4] ›Release Notes

5.4.0 Release Notes

edit

5.4.0 Release Notes

edit

See Multi data path bug in Elasticsearch 5.3.0.

Also see Breaking changes in 5.4.

Breaking changes

edit
Settings

Breaking Java changes

edit
Aggregations
  • Move getProperty method out of MultiBucketsAggregation.Bucket interface #23988
  • Remove getProperty method from Aggregations interface and impl #23972
  • Move getProperty method out of Aggregation interface #23949
Java API
  • Fold InternalSearchHits and friends into their interfaces #23042

Deprecations

edit
Aggregations
  • Deprecate Stats#getCountAsString #24292
Java API
  • Add BulkProcessor methods with XContentType parameter #23078 (issue: #22691)
Network
Packaging
  • Add deprecation warnings for $ES_USER and $ES_GROUP #24190 (issue: #23989)
Plugin Delete By Query
  • Deprecate delete_by_query requests without an explicit query #23635
Plugin Repository Azure
  • Deprecate global repositories.azure settings #22856 (issue: #22800)
Plugin Repository S3
REST
  • Deprecate ldjson support and document ndjson for bulk/msearch #23051 (issues: #23025, #23049)
Stats
  • Deprecate _field_stats endpoint #23914

New features

edit
Analysis
  • Adds pattern keyword marker filter support #23600 (issue: #4877)
  • Expose WordDelimiterGraphTokenFilter #23327 (issue: #23104)
Index APIs
  • Add FieldCapabilities (_field_caps) API #23007 (issue: #22438)
Search
  • Introduce incremental reduction of TopDocs #23946
Similarities
  • Adds boolean similarity to Elasticsearch #23637 (issue: #6731)

Enhancements

edit
Aggregations
  • Add BucketMetricValue interface #24188
  • Move aggs CommonFields and TYPED_KEYS_DELIMITER from InternalAggregation to Aggregation #23987
  • Use ParseField for aggs CommonFields rather than String #23717
  • Share XContent rendering code in terms aggs #23680
  • Add unit tests for ParentToChildAggregator #23305 (issue: #22278)
  • First step towards incremental reduction of query responses #23253
Allocation
  • Trigger replica recovery restarts by master when primary relocation completes #23926 (issue: #23904)
  • Makes the same_shard host dynamically updatable #23397 (issue: #22992)
Analysis
  • Support Keyword type in Analyze API #23161
Cluster
  • Prevent nodes from joining if newer indices exist in the cluster #23843
Core
  • Detect remnants of path.data/default.path.data bug #24099 (issues: #23981, #24052, #24074, #24093)
  • Await termination after shutting down executors #23889
  • Add early-access check #23743 (issue: #23668)
  • Adapter action future should restore interrupts #23618 (issue: #23617)
  • Disable bootstrap checks for single-node discovery #23598 (issues: #23585, #23595)
  • Enable explicitly enforcing bootstrap checks #23585 (issue: #21864)
  • Add equals/hashcode method to ReplicationResponse #23215
Dates
  • Improve error handling for epoch format parser with time zone (#22621) #23689
Discovery
  • Introduce single-node discovery #23595
  • UnicastZenPing shouldn’t ping the address of the local node #23567
  • MasterFaultDetection can start after the initial cluster state has been processed #23037 (issue: #22828)
Highlighting
  • Add support for fragment_length in the unified highlighter #23431
  • Add BreakIteratorBoundaryScanner support #23248
Index APIs
  • Wildcard cluster names for cross cluster search #23985 (issue: #23893)
Ingest
  • Lazy load the geoip databases #23337
Internal
  • Add a dedicated TransportRemoteInfoAction for consistency #24040 (issue: #23969)
  • Simplify sorted top docs merging in SearchPhaseController #23881
  • Synchronized CollapseTopFieldDocs with lucenes relatives #23854
  • Cleanup SearchPhaseController interface #23844
  • Do not create String instances in Strings methods accepting StringBuilder #22907
Java API
  • Added types options to DeleteByQueryRequest #23265 (issue: #21984)
Java High Level REST Client
  • Convert suggestion response parsing to use NamedXContentRegistry #23355
  • UpdateRequest implements ToXContent #23289
  • Add javadoc for DocWriteResponse.Builders #23267
  • Expose WriteRequest.RefreshPolicy string representation #23106
  • Use typed_keys parameter to prefix suggester names by type in search responses #23080 (issue: #22965)
  • Add parsing methods to BulkItemResponse #22859
Logging
  • Warn on not enough masters during election #20063 (issue: #8362)
Mapping
  • Improve error message for ipv6 on legacy ip fields #23136 (issue: #23126)
Nested Docs
  • Avoid adding unnecessary nested filters when ranges are used. #23427
Network
  • Adjust default Netty receive predictor size to 64k #23542 (issue: #23185)
  • Keep the pipeline handler queue small initially #23335
  • Set network receive predictor size to 32kb #23284 (issue: #23185)
  • TransportService.connectToNode should validate remote node ID #22828 (issue: #22194)
Packaging
  • Introduce Java version check #23194 (issue: #21102)
  • Cleanup some things after removal of joda-time hack #18959
Percolator
  • Allowing range queries with now ranges inside percolator queries #23921 (issue: #23859)
  • Add term extraction support for MultiPhraseQuery #23176
Plugin Discovery EC2
  • Settings: Migrate ec2 discovery sensitive settings to elasticsearch keystore #23961 (issue: #22475)
Plugin Lang Painless
  • Allow painless to load stored fields #24290
  • Start on custom whitelists for Painless #23563
  • Fix Painless’s implementation of interfaces returning primitives #23298 (issue: #22983)
  • Allow painless to implement more interfaces #22983
Plugin Repository Azure
  • Add Backoff policy to azure repository #23387 (issue: #22728)
Plugin Repository S3
  • Removes the retry mechanism from the S3 blob store #23952 (issue: #22845)
  • S3 Repository: Eagerly load static settings #23910
Plugins
  • Modify permissions dialog for plugins #23742
  • Plugins: Add plugin cli specific exit codes #23599 (issue: #15295)
  • Plugins: Output better error message when existing plugin is incompatible #23562 (issue: #20691)
Query DSL
  • Make it possible to validate a query on all shards instead of a single random shard #23697 (issue: #18254)
REST
  • Validate top-level keys when parsing mget requests #23746 (issue: #23720)
  • Cluster stats should not render empty http/transport types #23735
  • Add parameter to prefix aggs name with type in search responses #22965
Search
  • Set shard count limit to unlimited #24012
  • Streamline shard index availability in all SearchPhaseResults #23788
  • Search took time should use a relative clock #23662
  • Prevent negative from parameter in SearchSourceBuilder #23358 (issue: #23324)
  • Remove unnecessary result sorting in SearchPhaseController #23321
  • Expose batched_reduce_size via _search #23288 (issue: #23253)
  • Adding fromXContent to Suggest and Suggestion class #23226 (issue: #23202)
  • Adding fromXContent to Suggestion.Entry and subclasses #23202
  • Add CollapseSearchPhase as a successor for the FetchSearchPhase #23165
  • Integrate IndexOrDocValuesQuery. #23119
  • Detach SearchPhases from AbstractSearchAsyncAction #23118
  • Fix GraphQuery expectation after Lucene upgrade to 6.5 #23117 (issue: #23102)
  • Nested queries should avoid adding unnecessary filters when possible. #23079 (issue: #20797)
  • Add xcontent parsing to completion suggestion option #23071
  • Add xcontent parsing to suggestion options #23018
  • Separate reduce (aggs, suggest and profile) from merging fetched hits #23017
Settings
  • Add secure file setting to keystore #24001
  • Add a setting which specifies a list of setting #23883
  • Add a property to mark setting as final #23872
  • Remove obsolete index setting index.version.minimum_compatible. #23593
  • Provide a method to retrieve a closeable char[] from a SecureString #23389
  • Update indices settings api to support CBOR and SMILE format #23309 (issues: #23242, #23245)
  • Improve setting deprecation message #23156 (issue: #22849)
Snapshot/Restore
  • Change snapshot status error to use generic SnapshotException #24355 (issue: #24225)
Stats
  • Add cross-cluster search remote cluster info API #23969 (issue: #23925)
Task Manager
  • Allow task to be unregistered by ClusterStateApplier #23931
  • Limit IndexRequest toString() length #22832

Bug fixes

edit
Aggregations
  • Align behavior HDR percentiles iterator with percentile() method #24206
  • The filter and significant_terms aggregations should parse the filter as a filter, not a query. #23797
  • Completion suggestion should also consider text if prefix/regex is missing #23451 (issue: #23340)
  • Fixes the per term error in the terms aggregation #23399
  • Fixes terms error count for multiple reduce phases #23291 (issue: #23286)
  • Restore support for the include/pattern syntax. #23140 (issue: #22933)
Bulk
CRUD
Cluster
  • Don’t set local node on cluster state used for node join validation #23311 (issues: #21830, #3, #4, #6, #9)
  • Allow a cluster state applier to create an observer and wait for a better state #23132 (issue: #21817)
  • Cluster allocation explain to never return empty response body #23054
Core
  • Check for default.path.data included in path.data #24285 (issue: #24283)
  • Improve performance of extracting warning value #24114 (issue: #24018)
  • Reject duplicate settings on the command line #24053
  • Restrict build info loading to ES jar, not any jar #24049 (issue: #21955)
Discovery
  • ZenDiscovery - only validate min_master_nodes values if local node is master #23915 (issue: #23695)
Index APIs
  • Fixes restore of a shrunken index when initial recovery node is gone #24322 (issue: #24257)
  • Honor update request timeout #23825
Ingest
  • Improve missing ingest processor error #23379 (issue: #23392)
  • update _ingest.timestamp to use new ZonedDateTime #23174 (issue: #23168)
Inner Hits
  • Replace NestedChildrenQuery with ParentChildrenBlockJoinQuery #24016 (issue: #24009)
  • Changed DisMaxQueryBuilder to extract inner hits from leaf queries #23512 (issue: #23482)
Internal
  • Add infrastructure to mark contexts as system contexts #23830
  • Always restore the ThreadContext for operations delayed due to a block #23349
Java High Level REST Client
  • Correctly parse BulkItemResponse.Failure’s status #23432
Java REST Client
  • Make buffer limit configurable in HeapBufferedConsumerFactory #23970 (issue: #23958)
  • RestClient asynchronous execution should not throw exceptions #23307
Mapping
  • Preserve response headers when creating an index #23950 (issue: #23947)
  • Improves disabled fielddata error message #23841 (issue: #22768)
  • Switch include_in_all in multifield to warning #23656 (issues: #21971, #23654)
  • Fix MapperService StackOverflowError #23605 (issue: #23604)
  • Fix NPE with scaled floats stats when field is not indexed #23528 (issue: #23487)
Network
  • Fix possible hang in local transport when nodes get concurrently disconnected #23962 (issue: #23942)
  • Respect promises on pipelined responses #23317 (issues: #23310, #23322)
  • Ensure that releasing listener is called #23310
Packaging
  • Fall back to non-atomic move when removing plugins #23548 (issue: #35)
Percolator
  • Fix memory leak when percolator uses bitset or field data cache #24115 (issue: #24108)
Plugin Ingest Attachment
Plugin Lang Painless
  • Fix painless’s regex lexer and error messages #23634
  • Replace Painless’s Cast with casting strategies #23369
  • Fix Bad Casts In Painless #23282 (issue: #23238)
Plugin Repository Azure
  • Azure blob store’s readBlob() method first checks if the blob exists #23483 (issue: #23480)
Plugin Repository S3
  • Handle BlobPath’s trailing separator case. Add test cases to BlobPathTests.java #23091
Plugins
  • Fix delete of plugin directory on remove plugin #24266 (issue: #24252)
  • Use a marker file when removing a plugin #24252 (issue: #24231)
Query DSL
  • FuzzyQueryBuilder should error when parsing array of values #23762 (issue: #23759)
REST
  • [API] change wait_for_completion default according to docs #23672
  • Deprecate request_cache for clear-cache #23638 (issue: #22748)
  • HTTP transport stashes the ThreadContext instead of the RestController #23456
  • Ensure we try to autodetect content type for handlers that support plain text #23452
  • Fix date format in warning headers #23418 (issue: #23275)
  • Align REST specs for HEAD requests #23313 (issue: #21125)
  • Correct warning header to be compliant #23275 (issue: #22986)
  • Fix get HEAD requests #23186 (issue: #21125)
  • Fix search scroll request with a plain text body #23183 (issue: #22691)
  • Handle bad HTTP requests #23153 (issue: #23034)
  • Fix get source HEAD requests #23151 (issue: #21125)
  • Properly encode location header #23133 (issues: #21057, #23115)
  • Fix template HEAD requests #23130 (issue: #21125)
  • Fix index HEAD requests #23112 (issue: #21125)
  • Fix alias HEAD requests #23094 (issue: #21125)
Recovery
  • Provide target allocation id as part of start recovery request #24333 (issue: #24167)
Reindex API
  • Fix throttled reindex_from_remote #23953 (issues: #23828, #23945)
  • Fix reindex with a remote source on a version before 2.0.0 #23805
  • Make reindex wait for cleanup before responding #23677 (issue: #23653)
Scripting
  • Remove unnecessary Groovy deprecation logging #23410 (issue: #23401)
  • Convert script/template objects to json format internally #23308 (issue: #23245)
  • Script: Fix value of ctx._now to be current epoch time in milliseconds #23175 (issue: #23169)
Search
  • Cross Cluster Search: propagate original indices per cluster #24328
  • Query string default field #24214
  • Speed up parsing of large terms queries. #24210
  • IndicesQueryCache should delegate the scorerSupplier method. #24209
  • Fork LRUQueryCache from Lucene to work around LUCENE-7749 #24207
  • Disable graph analysis at query time for shingle and cjk filters producing tokens of different size #23920 (issue: #23918)
  • Fix cross-cluster remote node gateway attributes #23863
  • Use a fixed seed for computing term hashCode in TermsSliceQuery #23795
  • Honor max concurrent searches in multi-search #23538 (issue: #23527)
  • Avoid stack overflow in multi-search #23527 (issue: #23523)
  • Fix query_string_query to transform "foo:*" in an exists query on the field name #23433 (issue: #23356)
  • Factor out filling of TopDocs in SearchPhaseController #23380 (issues: #19356, #23357)
  • Replace blocking calls in ExpandCollapseSearchResponseListener by asynchronous requests #23053 (issue: #23048)
Search Templates
  • No longer add illegal content type option to stored search templates #24251 (issue: #24227)
Settings
  • Do not set path.data in environment if not set #24132 (issue: #24099)
  • Correct handling of default and array settings #24074 (issues: #23981, #24052)
  • Fix merge scheduler config settings #23391
  • Settings: Fix keystore cli prompting for yes/no to handle console returning null #23320
Similarities
  • Fix similarity upgrade when "default" similarity is overridden #23163
Snapshot/Restore
  • Fixes maintaining the shards a snapshot is waiting on #24289
  • Fixes snapshot status on failed snapshots #23833 (issue: #23716)
  • Fixes snapshot deletion handling on in-progress snapshot failure #23703 (issue: #23663)
  • Prioritize listing index-N blobs over index.latest in reading snapshots #23333
Stats
  • Avoid overflow when computing total FS stats #23641
  • Handle existence of cgroup version 2 hierarchy #23493 (issue: #23486)
  • Handle long overflow when adding paths' totals #23293 (issue: #23093)
  • Fix control group pattern #23219 (issue: #23218)
  • Fix total disk bytes returning negative value #23093

Regressions

edit
Bulk

Upgrades

edit
Aggregations
Core
  • Upgrade to Lucene 6.5.0 #23750
  • Upgrade from JNA 4.2.2 to JNA 4.4.0 #23636
  • Upgrade to lucene-6.5.0-snapshot-d00c5ca #23385
  • Upgrade to lucene-6.5.0-snapshot-f919485. #23087
Logging
  • Upgrade to Log4j 2.8.2 #23995
Network
Plugin Repository Azure
« 5.4.1 Release Notes 5.3.3 Release Notes »