WARNING: Version 6.1 of Elasticsearch has passed its EOL date.

This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.

« 6.0.0-beta2 Release Notes 6.0.0-alpha2 Release Notes »
Elastic Docs ›Elasticsearch Reference [6.1] ›Release Notes

6.0.0-beta1 Release Notes

edit

6.0.0-beta1 Release Notes

edit

Also see Breaking changes in 6.0.

Breaking changes

edit
Aggregations
  • Change parsing of numeric to and from parameters in date_range aggregation #25376 (issue: #17920)
Aliases
Highlighting
  • Remove the postings highlighter and make unified the default highlighter choice #25028
Index APIs
  • Remove (deprecated) support for + in index expressions #25274 (issue: #24515)
  • Delete index API to work only against concrete indices #25268 (issues: #2318, #23997)
Indexed Scripts/Templates
  • Scripting: Remove search template actions #25717
Ingest
  • update ingest-user-agent regexes.yml #25608
  • remove ingest.new_date_format #25583
Java REST Client
Packaging
  • Remove support for ES_INCLUDE #25804
  • Setup: Change default heap to 1G #25695
  • Use config directory to find jvm.options #25679 (issue: #23004)
  • Remove implicit 32-bit support #25435
  • Remove default path settings #25408 (issue: #25357)
  • Remove path.conf setting #25392 (issue: #25357)
  • Honor masking of systemd-sysctl.service #24234 (issues: #21899, #806)
Plugin Analysis ICU
  • Upgrade icu4j for the ICU analysis plugin to 59.1 #25243 (issue: #21425)
Plugin Discovery Azure Classic
  • Remove discovery.type BWC layer from the EC2/Azure/GCE plugins #25080 (issue: #24543)
Plugin Repository GCS
  • GCS Repository: Remove specifying credential file on disk #24727
Plugins
  • Make plugin loading stricter #25405
Query DSL
  • Refactor QueryStringQuery for 6.0 #25646 (issue: #25574)
  • Change split_on_whitespace default to false #25570 (issue: #25470)
  • Remove deprecated template query #24577 (issue: #19390)
REST
  • IndexClosedException to return 400 rather than 403 #25752
  • Remove comma-separated feature parsing for GetIndicesAction #24723 (issue: #24437)
  • Improve REST error handling when endpoint does not support HTTP verb, add OPTIONS support #24437 (issues: #0, #15335, #17916)
Scripting
  • remove lang url parameter from stored script requests #25779 (issue: #22887)
  • Disallow lang to be used with Stored Scripts #25610
  • Remove Deprecated Script Settings #24756 (issue: #24532)
  • Scripting: Remove native scripts #24726 (issue: #19966)
  • Scripting: Remove file scripts #24627 (issue: #21798)
Search
  • Make index in TermsLookup mandatory #25753 (issue: #25750)
  • Removes FieldStats API #25628 (issue: #25577)
  • Remove deprecated fielddata_fields from search request #25566 (issue: #25537)
  • Removes deprecated fielddata_fields #25537 (issue: #19027)
Settings
  • Settings: Remove shared setting property #24728
  • Settings: Remove support for yaml and json config files #24664 (issue: #19391)
Similarities
  • Similarity should accept dynamic settings when possible #20339 (issue: #6727)

Breaking Java changes

edit
Aggregations
  • Remove the unused SignificantTerms.compareTerm() method #24714
  • Make SignificantTerms.Bucket an interface rather than an abstract class #24670 (issue: #24492)
Internal
  • Collapses package structure for some bucket aggs #25579 (issue: #22868)
Java API
  • Remove deprecated IdsQueryBuilder ctor #25529
  • Removing unneeded getTookInMillis method #23923
Java High Level REST Client
  • Unify the result interfaces from get and search in Java client #25361 (issue: #16440)
  • Allow RestHighLevelClient to use plugins #25024
Java REST Client
Plugin Delete By Query
  • Move DeleteByQuery and Reindex requests into core #24578
Query DSL
  • Remove QueryParseContext #25486
  • Remove QueryParseContext from parsing QueryBuilders #25448
REST
  • Return index name and empty map for /{index}/_alias with no aliases #25114 (issues: #24723, #25090)

Deprecations

edit
Index APIs
  • Deprecated use of + in index expressions #24585 (issue: #24515)
Indexed Scripts/Templates
  • Scripting: Deprecate stored search template apis #25437 (issue: #24596)
Percolator
  • Deprecate percolate query’s document_type parameter. #25199
Scripting
  • Scripting: Change keys for inline/stored scripts to source/id #25127
  • Scripting: Deprecate native scripts #24692 (issue: #19966)
  • Scripting: Deprecate index lookup #24691 (issue: #19359)
  • Deprecate Fine Grain Settings for Scripts #24573 (issue: #24532)
  • Scripting: Deprecate file script settings #24555 (issue: #21798)
  • Scripting: Deprecate file scripts #24552 (issue: #21798)
Settings
  • Settings: Update settings deprecation from yml to yaml #24663 (issue: #19391)
Tribe Node

New features

edit
Analysis
  • Expose simplepattern and simplepatternsplit tokenizers #25159 (issue: #23363)
  • Parse synonyms with the same analysis chain #8049 (issue: #7199)
Parent/Child
  • Move parent_id query to the parent-join module #25072 (issue: #20257)
  • Introduce ParentJoinFieldMapper, a field mapper that creates parent/child relation within documents of the same index #24978 (issue: #20257)
Search
  • Automatically early terminate search query based on index sorting #24864 (issue: #6720)
Sequence IDs
  • Add a scheduled translog retention check #25622 (issues: #10708, #25294)
  • Initialize sequence numbers on a shrunken index #25321 (issue: #10708)
  • Initialize primary term for shrunk indices #25307 (issue: #10708)
  • Introduce translog size and age based retention policies #25147 (issue: #10708)
Stats
  • Adds nodes usage API to monitor usages of actions #24169
Task Manager
Upgrade API
  • TemplateUpgraders should be called during rolling restart #25263 (issues: #24379, #24680)

Enhancements

edit
Aggregations
  • Add strict parsing of aggregation ranges #25769
  • Adds rewrite phase to aggregations #25495 (issue: #17676)
  • Tweak AggregatorBase.addRequestCircuitBreakerBytes #25162 (issue: #24511)
  • Add superset size to Significant Term REST response #24865
  • Add document count to Matrix Stats aggregation response #24776
  • Adds an implementation of LogLogBeta for the cardinality aggregation #22323 (issue: #22230)
Allocation
  • Adjust status on bad allocation explain requests #25503 (issue: #25458)
  • Promote replica on the highest version node #25277 (issue: #10708)
Analysis
  • [Analysis] Support normalizer in request param #24767 (issue: #23347)
  • Enforce validation for PathHierarchy tokenizer #23510
  • [analysis-icu] Allow setting unicodeSetFilter #20814 (issue: #20820)
CAT API
  • expand /_cat/nodes to return information about hard drive #21775 (issue: #21679)
Cluster
  • Validate a joining node’s version with version of existing cluster nodes #25808
  • Switch indices read-only if a node runs out of disk space #25541 (issue: #24299)
  • Add a cluster block that allows to delete indices that are read-only #24678
Core
  • Add max file size bootstrap check #25974
  • Add compatibility versions to main action response #25799
  • Index ids in binary form. #25352 (issues: #18154, #24615)
  • Explicitly reject duplicate data paths #25178
  • Use SPI in High Level Rest Client to load XContent parsers #25097
  • Upgrade to lucene-7.0.0-snapshot-a0aef2f #24775
  • Speed up PK lookups at index time. #19856
Engine
  • Add refresh stats tracking for realtime get #25052 (issue: #24806)
  • Introducing a translog deletion policy #24950
Exceptions
  • IllegalStateException: Only duplicated jar instead of classpath #24953
Highlighting
  • Picks offset source for the unified highlighter directly from the es mapping #25747 (issue: #25699)
Index APIs
  • Let primary own its replication group #25692 (issue: #25485)
  • Create index request should return the index name #25139 (issue: #23044)
Ingest
  • Add Ingest-Processor specific Rest Endpoints & Add Grok endpoint #25059 (issue: #24725)
  • Port support for commercial GeoIP2 databases from Logstash. #24889
  • add exclude_keys option to KeyValueProcessor #24876 (issue: #23856)
  • Allow removing multiple fields in ingest processor #24750 (issue: #24622)
  • Add target_field parameter to ingest processors #24133 (issues: #23228, #23682)
Inner Hits
Internal
  • Cleanup IndexFieldData visibility #25900
  • Bump the min compat version to 5.6.0 #25805
  • "shard started" should show index and shard ID #25157
  • Break out clear scroll logic from TransportClearScrollAction #25125 (issue: #25094)
  • Add helper methods to TransportActionProxy to identify proxy actions and requests #25124
  • Add remote cluster infrastructure to fetch discovery nodes. #25123 (issue: #25094)
  • Add the ability to set eager_global_ordinals in the new parent-join field #25019
  • Disallow multiple parent-join fields per mapping #25002
  • Remove the need for _UNRELEASED suffix in versions #24798 (issue: #24768)
  • Optimize the order of bytes in uuids for better compression. #24615 (issue: #18209)
Java API
  • Always Accumulate Transport Exceptions #25017 (issue: #23099)
Java High Level REST Client
  • [DOCS] restructure java clients docs pages #25517
  • Use SPI in High Level Rest Client to load XContent parsers #25098 (issues: #25024, #25097)
  • Add support for clear scroll to high level REST client #25038
  • Add search scroll method to high level REST client #24938 (issue: #23331)
  • Add search method to high level REST client #24796 (issues: #24794, #24795)
Java REST Client
  • Shade external dependencies in the rest client jar #25780 (issue: #25208)
  • RestClient uses system properties and system default SSLContext #25757 (issue: #23231)
Logging
  • Prevent excessive disk consumption by log files #25660
  • Use LRU set to reduce repeat deprecation messages #25474 (issue: #25457)
Mapping
  • Better validation of copy_to. #25983
  • Optimize terms queries on ip addresses to use a PointInSetQuery whenever possible. #25669 (issue: #25667)
Network
Packaging
  • Remove memlock suggestion from systemd service #25979
  • Set address space limit in systemd service file #25975
  • Version option should display if snapshot #25970
  • Ignore JVM options before checking Java version #25969
  • Also skip JAVA_TOOL_OPTIONS on Windows #25968
  • Introduce elasticsearch-env for Windows #25958
  • Introduce elasticsearch-env #25815 (issue: #20286)
  • Stop exporting HOSTNAME from scripts #25807
Parent/Child
  • Remove ParentJoinFieldSubFetchPhase #25550 (issue: #25363)
  • Support parent id being specified as number in the _source #25547
Plugin Lang Painless
  • Allow Custom Whitelists in Painless #25557
  • Update Painless to Allow Augmentation from Any Class #25360
  • Add Needs Methods to Painless Script Context Factories #25267
  • Support Script Context Stateful Factory in Painless #25233
  • Generate Painless Factory for Creating Script Instances #25120
  • Update Painless to Use New Script Contexts #25015
  • Optimize instance creation in LambdaBootstrap #24618
Plugin Repository GCS
  • GCS Repository: Add secure storage of credentials #24697
Plugin Repository S3
  • S3 Repository: Add back repository level credentials #24609
Plugins
  • Move tribe to a module #25778
  • Plugins can register pre-configured char filters #25000 (issue: #23658)
  • Add purge option to remove plugin CLI #24981
  • Allow plugins to register pre-configured tokenizers #24751 (issues: #24223, #24572)
  • Move ReindexAction class to core #24684 (issue: #24578)
  • Make PreConfiguredTokenFilter harder to misuse #24572 (issue: #23658)
Query DSL
  • Make slop optional when parsing span_near query #25677 (issue: #25642)
  • Require a field when a seed is provided to the random_score function. #25594 (issue: #25240)
REST
  • Refactor PathTrie and RestController to use a single trie for all methods #25459 (issue: #24437)
  • Make ObjectParser support string to boolean conversion #24668 (issue: #21802)
Recovery
  • Goodbye, Translog Views #25962
  • Disallow multiple concurrent recovery attempts for same target shard #25428
  • Live primary-replica resync (no rollback) #24841 (issue: #10708)
Scripting
  • Scripting: Rename SearchScript.needsScores to needs_score #25235
  • Scripting: Add optional context parameter to put stored script requests #25014
  • Add New Security Script Settings #24637 (issue: #24532)
Search
  • Rewrite search requests on the coordinating nodes #25814 (issue: #25791)
  • Ensure query resources are fetched asynchronously during rewrite #25791
  • Introduce a new Rewriteable interface to streamline rewriting #25788
  • Reduce the scope of QueryRewriteContext #25787
  • Reduce the overhead of timeouts and low-level search cancellation. #25776
  • Reduce profiling overhead. #25772 (issue: #24799)
  • Prevent can_match requests from sending to incompatible nodes #25705 (issue: #25704)
  • Add a shard filter search phase to pre-filter shards based on query rewriting #25658
  • Ensure we rewrite common queries to match_none if possible #25650
  • Limit the number of concurrent shard requests per search request #25632
  • Add cluster name validation to RemoteClusterConnection #25568
  • Speed up sorted scroll when the index sort matches the search sort #25138 (issue: #6720)
  • Leverage scorerSupplier when applicable. #25109
  • Add cross-cluster search support for scroll searches #25094
  • Track EWMA[1] of task execution time in search threadpool executor #24989 (issue: #24915)
  • Query range fields by doc values when they are expected to be more efficient than points #24823 (issue: #24314)
  • Search: Fairer balancing when routing searches by session ID #24671 (issue: #24642)
Sequence IDs
  • Move primary term from ReplicationRequest to ConcreteShardRequest #25822
  • Add reason to global checkpoint updates on replica #25612 (issue: #10708)
  • Introduce primary/replica mode for GlobalCheckPointTracker #25468
  • Throw back replica local checkpoint on new primary #25452 (issues: #10708, #25355)
  • Update global checkpoint when increasing primary term on replica #25422 (issues: #10708, #25355)
  • Enable a long translog retention policy by default #25294 (issues: #10708, #25147)
  • Introduce primary context #25122 (issues: #10708, #25355)
  • Block older operations on primary term transition #24779 (issue: #10708)
Settings
  • Add disk threshold settings validation #25600 (issue: #25560)
  • Enable cross-setting validation #25560 (issue: #25541)
  • Validate transport.profiles.* settings #25508
  • Cleanup network / transport related settings #25489
  • Emit settings deprecation logging at most once #25457
  • IndexMetaData: Introduce internal format index setting #25292
Snapshot/Restore
  • Improves snapshot logging and snapshot deletion error handling #25264
Stats
  • Update IndexShard#refreshMetric via a ReferenceManager.RefreshListener #25083 (issues: #24806, #25052)
Translog
  • Translog file recovery should not rely on lucene commits #25005 (issue: #24950)

Bug fixes

edit
Aggregations
  • Fixes array out of bounds for value count agg #26038 (issue: #17379)
  • Aggregations bug: Significant_text fails on arrays of text. #25030 (issue: #25029)
Aliases
  • mget with an alias shouldn’t ignore alias routing #25697 (issue: #25696)
  • GET aliases should 404 if aliases are missing #25043 (issue: #24644)
Analysis
  • Pre-configured shingle filter should disable graph analysis #25853 (issue: #25555)
Circuit Breakers
  • Checks the circuit breaker before allocating bytes for a new big array #25010 (issue: #24790)
Core
  • Release operation permit on thread-pool rejection #25930 (issue: #25863)
  • Node should start up despite of a lingering .es_temp_file #21210 (issue: #21007)
Discovery
  • MasterNodeChangePredicate should use the node instance to detect master change #25877 (issue: #25471)
Engine
  • Engine - do not index operations with seq# lower than the local checkpoint into lucene #25827 (issues: #1, #2, #25592)
Geo
  • Fix typo in GeoUtils#isValidLongitude #25121
Highlighting
  • FastVectorHighlighter should not cache the field query globally #25197 (issue: #25171)
  • Higlighters: Fix MultiPhrasePrefixQuery rewriting #25103 (issue: #25088)
Index APIs
  • Shrink API should ignore templates #25380 (issue: #25035)
  • Rollover max docs should only count primaries #24977 (issue: #24217)
Ingest
  • Sort Processor does not have proper behavior with targetField #25237 (issue: #24133)
  • fix grok’s pattern parsing to validate pattern names in expression #25063 (issue: #22831)
Inner Hits
  • When fetching nested inner hits only access stored fields when needed #25864 (issue: #6)
Internal
  • Fix BytesReferenceStreamInput#skip with offset #25634
  • Fix race condition in RemoteClusterConnection node supplier #25432
  • Initialise empty lists in BaseTaskResponse constructor #25290
  • Extract a common base class for scroll executions #24979 (issue: #16555)
  • Obey lock order if working with store to get metadata snapshots #24787 (issue: #24481)
  • Fix Version based BWC and set correct minCompatVersion #24732
  • Fix _field_caps serialization in order to support cross-cluster search #24722
  • Avoid race when shutting down controller processes #24579
Mapping
  • Fix parsing of ip range queries. #25768 (issue: #25636)
  • Disable date field mapping changing #25285 (issue: #25271)
  • Correctly enable _all for older 5.x indices #25087 (issue: #25068)
  • token_count datatype should handle null value #25046 (issue: #24928)
  • keep _parent field while updating child type mapping #24407 (issue: #23381)
More Like This
  • Pass over _routing value with more_like_this items to be retrieved #24679 (issue: #23699)
Nested Docs
  • In case of a single type the _id field should be added to the nested document instead of _uid field #25149
Network
  • Ensure pending transport handlers are invoked for all channel failures #25150
  • Notify onConnectionClosed rather than onNodeDisconnect to prune transport handlers #24639 (issues: #24557, #24575, #24632)
Packaging
  • Exit Windows scripts promptly on failure #25959
  • Pass config path as a system property #25943
  • ES_HOME needs to be made absolute before attempt at traversal #25865
  • Fix elasticsearch-keystore handling of path.conf #25811
  • Stop disabling explicit GC #25759
  • Avoid failing install if system-sysctl is masked #25657 (issue: #24234)
  • Get short path name for native controllers #25344
  • When stopping via systemd only kill the JVM, not its control group #25195
  • remove remaining references to scripts directory #24771
  • Handle parentheses in batch file path #24731 (issue: #24712)
Parent/Child
  • The default _parent field should not try to load global ordinals #25851 (issue: #25849)
Percolator
  • Fix range queries with date range based on current time in percolator queries. #24666 (issue: #23921)
Plugin Lang Painless
  • Painless: allow doubles to be casted to longs. #25936
Plugin Repository Azure
  • Make calls to CloudBlobContainer#exists privileged #25937 (issue: #25931)
Plugin Repository GCS
  • Ensure that gcs client creation is privileged #25938 (issue: #25932)
Plugin Repository HDFS
  • Upgrading HDFS Repository Plugin to use HDFS 2.8.1 Client #25497 (issue: #25450)
Plugin Repository S3
  • Avoid SecurityException in repository-S3 on DefaultS3OutputStream.flush() #25254 (issue: #25192)
Plugins
  • X-Pack plugin download fails on Windows desktop #24570
Query DSL
  • SpanNearQueryBuilder should return the inner clause when a single clause is provided #25856 (issue: #25630)
  • Refactor field expansion for match, multi_match and query_string query #25726 (issues: #25551, #25556)
  • WrapperQueryBuilder should also rewrite the parsed query #25480
REST
  • Fix handling of invalid error trace parameter #25785 (issue: #25774)
  • Fix handling of exceptions thrown on HEAD requests #25172 (issue: #21125)
  • Fixed NPEs caused by requests without content. #23497 (issue: #24701)
  • Fix get mappings HEAD requests #23192 (issue: #21125)
Recovery
  • Close translog view after primary-replica resync #25862 (issue: #24841)
Reindex API
  • Reindex: don’t duplicate _source parameter #24629 (issue: #24628)
  • Add qa module that tests reindex-from-remote against pre-5.0 versions of Elasticsearch #24561 (issues: #23828, #24520)
Search
  • Caching a MinDocQuery can lead to wrong results. #25909
  • Fix random score generation when no seed is provided. #25908
  • Merge FunctionScoreQuery and FiltersFunctionScoreQuery #25889 (issues: #15709, #23628)
  • Respect cluster alias in _index aggs and queries #25885 (issue: #25606)
  • First increment shard stats before notifying and potentially sending response #25818
  • Remove assertion about deviation when casting to a float. #25806 (issue: #25330)
  • Prevent skipping shards if a suggest builder is present #25739 (issue: #25658)
  • Ensure remote cluster alias is preserved in inner hits aggs #25627 (issue: #25606)
  • Do not search locally if remote index pattern resolves to no indices #25436 (issue: #25426)
  • Adds check for negative search request size #25397 (issue: #22530)
  • Make sure range queries are correctly profiled. #25108
  • Fix RangeFieldMapper rangeQuery to properly handle relations #24808 (issue: #24744)
  • Fix ExpandSearchPhase when response contains no hits #24688 (issue: #24672)
Sequence IDs
  • Fix pre-6.0 response to unknown replication actions #25744 (issue: #10708)
  • Track local checkpoint on primary immediately #25434 (issues: #10708, #25355, #25415)
  • Initialize max unsafe auto ID timestamp on shrink #25356 (issues: #10708, #25355)
  • Use correct primary term for replicating NOOPs #25128
  • Handle already closed while filling gaps #25021 (issue: #24925)
Settings
  • Fix settings serialization to not serialize secure settings or not take the total size into account #25323
  • Keystore CLI should use the AddFileKeyStoreCommand for files #25298
  • Allow resetting settings that use an IP validator #24713 (issue: #24709)
Snapshot/Restore
  • Snapshot/Restore: Ensure that shard failure reasons are correctly stored in CS #25941 (issue: #25878)
  • Output all empty snapshot info fields if in verbose mode #25455 (issue: #24477)
  • Remove redundant and broken MD5 checksum from repository-s3 #25270 (issue: #25269)
  • Consolidates the logic for cleaning up snapshots on master election #24894 (issue: #24605)
  • Removes completed snapshot from cluster state on master change #24605 (issue: #24452)
Stats
  • _nodes/stats should not fail due to concurrent AlreadyClosedException #25016 (issue: #23099)
Suggesters
  • Context suggester should filter doc values field #25858 (issue: #25404)

Regressions

edit
Highlighting
  • Fix Fast Vector Highlighter NPE on match phrase prefix #25116 (issue: #25088)
Search
  • Always use DisjunctionMaxQuery to build cross fields disjunction #25115 (issue: #23966)

Upgrades

edit
Network
Upgrade API
  • Improve stability and logging of TemplateUpgradeServiceIT tests #25386 (issue: #25382)
« 6.0.0-beta2 Release Notes 6.0.0-alpha2 Release Notes »