WARNING: Version 6.2 of Elasticsearch has passed its EOL date.

This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.

« Elasticsearch version 6.0.1 Elasticsearch version 6.0.0-rc2 »
Elastic Docs ›Elasticsearch Reference [6.2] ›Elasticsearch Release Notes

Elasticsearch version 6.0.0

edit

Elasticsearch version 6.0.0

edit

Breaking Changes

edit
Aggregations
  • Change parsing of numeric to and from parameters in date_range aggregation #25376 (issue: #17920)
Aliases
Allocation
  • Remove cluster.routing.allocation.snapshot.relocation_enabled setting #20994
Analysis
  • Do not allow custom analyzers to have the same names as built-in analyzers #22349 (issue: #22263)
  • Removing query-string parameters in _analyze API #20704 (issue: #20246)
CAT API
  • Write -1 on unbounded queue in cat thread pool #21342 (issue: #21187)
CRUD
  • Disallow VersionType.FORCE for GetRequest #21079 (issue: #20995)
  • Disallow VersionType.FORCE versioning for 6.x indices #20995 (issue: #20377)
  • If the index does not exist, delete document will not auto create it #24518 (issue: #15425)
Cluster
  • Disallow : in cluster and index/alias names #26247 (issue: #23892)
  • No longer allow cluster name in data path #20433 (issue: #20391)
Core
  • Simplify file store #24402 (issue: #24390)
  • Make boolean conversion strict #22200
  • Remove the default store type. #21616
  • Remove store throttling. #21573
Geo
  • Remove deprecated geo search features #22876
  • Reduce GeoDistance Insanity #19846
Highlighting
  • Remove the postings highlighter and make unified the default highlighter choice #25028
Index APIs
  • Remove (deprecated) support for + in index expressions #25274 (issue: #24515)
  • Delete index API to work only against concrete indices #25268 (issues: #2318, #23997)
  • Open/Close index api to allow_no_indices by default #24401 (issues: #24031, #24341)
  • Remove support for controversial ignore_unavailable and allow_no_indices from indices exists api #20712
Index Templates
  • Allows multiple patterns to be specified for index templates #21009 (issue: #20690)
Indexed Scripts/Templates
  • Scripting: Remove search template actions #25717
Ingest
  • update ingest-user-agent regexes.yml #25608
  • remove ingest.new_date_format #25583
Inner Hits
  • Return the _source of inner hit nested as is without wrapping it into its full path context #26982 (issues: #26102, #26944)
Java API
  • Enforce Content-Type requirement on the rest layer and remove deprecated methods #23146 (issue: #19388)
Java REST Client
Mapping
  • Reject out of range numbers for float, double and half_float #25826 (issue: #25534)
  • Enforce at most one type. #24428 (issue: #24317)
  • Disallow include_in_all for 6.0+ indices #22970 (issue: #22923)
  • Disable _all by default, disallow configuring _all on 6.0+ indices #22144 (issues: #19784, #20925, #21341)
  • Throw an exception on unrecognized "match_mapping_type" #22090 (issue: #17285)
Network
  • Remove unused Netty-related settings #26161
  • Remove blocking TCP clients and servers #22639
  • Remove modules/transport_netty_3 in favor of netty_4 #21590
  • Remove LocalTransport in favor of MockTcpTransport #20695
Packaging
  • Configure heap dump path out of the box #26755 (issue: #26665)
  • Remove support for ES_INCLUDE #25804
  • Setup: Change default heap to 1G #25695
  • Use config directory to find jvm.options #25679 (issue: #23004)
  • Remove implicit 32-bit support #25435
  • Remove default path settings #25408 (issue: #25357)
  • Remove path.conf setting #25392 (issue: #25357)
  • Honor masking of systemd-sysctl.service #24234 (issues: #21899, #806)
  • Rename CONF_DIR to ES_PATH_CONF #26197 (issue: #26154)
  • Remove customization of ES_USER and ES_GROUP #23989 (issue: #23848)
Percolator
  • Remove deprecated percolate and mpercolate apis #22331
Plugin Analysis ICU
  • Upgrade icu4j for the ICU analysis plugin to 59.1 #25243 (issue: #21425)
  • Upgrade icu4j to latest version #24821
Plugin Delete By Query
  • Require explicit query in _delete_by_query API #23632 (issue: #23629)
Plugin Discovery Azure Classic
  • Remove discovery.type BWC layer from the EC2/Azure/GCE plugins #25080 (issue: #24543)
Plugin Discovery EC2
  • Ec2 Discovery: Cleanup deprecated settings #24150
  • Discovery EC2: Remove region setting #23991 (issue: #22758)
  • AWS Plugins: Remove signer type setting #23984 (issue: #22599)
Plugin Lang JS
  • Remove lang-python and lang-javascript #20734 (issue: #20698)
Plugin Mapper Attachment
Plugin Repository Azure
  • Remove global repositories.azure settings #23262 (issues: #22800, #22856)
  • Remove auto creation of container for azure repository #22858 (issue: #22857)
Plugin Repository GCS
  • GCS Repository: Remove specifying credential file on disk #24727
Plugin Repository S3
  • S3 Repository: Cleanup deprecated settings #24097
  • S3 Repository: Remove region setting #22853 (issue: #22758)
  • S3 Repository: Remove bucket auto create #22846 (issue: #22761)
  • S3 Repository: Remove env var and sysprop credentials support #22842
  • Remove deprecated S3 settings #24445
Plugins
  • Make plugin loading stricter #25405
Query DSL
  • Remove deprecated type and slop field in match query #26720
  • Remove several parse field deprecations in query builders #26711
  • Remove deprecated parameters from ids_query #26508
  • Refactor QueryStringQuery for 6.0 #25646 (issue: #25574)
  • Change split_on_whitespace default to false #25570 (issue: #25470)
  • Remove deprecated template query #24577 (issue: #19390)
  • Throw exception in scroll requests using from #26235 (issue: #9373)
  • Remove deprecated minimum_number_should_match in BoolQueryBuilder #22416
  • Remove support for empty queries #22092 (issue: #17624)
  • Remove deprecated query names: in, geo_bbox, mlt, fuzzy_match and match_fuzzy #21852
  • The terms query should always map to a Lucene TermsQuery. #21786
  • Be strict when parsing values searching for booleans #21555 (issue: #21545)
  • Remove collect payloads parameter #20385
REST
  • IndexClosedException to return 400 rather than 403 #25752
  • Remove comma-separated feature parsing for GetIndicesAction #24723 (issue: #24437)
  • Improve REST error handling when endpoint does not support HTTP verb, add OPTIONS support #24437 (issues: #0, #15335, #17916)
  • Remove ldjson support and document ndjson for bulk/msearch #23049 (issue: #23025)
  • Enable strict duplicate checks for all XContent types #22225 (issues: #19614, #22073)
  • Enable strict duplicate checks for JSON content #22073 (issue: #19614)
  • Remove lenient stats parsing #21417 (issues: #20722, #21410)
  • Remove allow unquoted JSON #20388 (issues: #17674, #17801)
  • Remove FORCE version_type #20377 (issue: #19769)
Scripting
  • remove lang url parameter from stored script requests #25779 (issue: #22887)
  • Disallow lang to be used with Stored Scripts #25610
  • Remove Deprecated Script Settings #24756 (issue: #24532)
  • Scripting: Remove native scripts #24726 (issue: #19966)
  • Scripting: Remove file scripts #24627 (issue: #21798)
  • Make dates be ReadableDateTimes in scripts #22948 (issue: #22875)
  • Remove groovy scripting language #21607
  • Remove script access to term statistics #19462 (issue: #19359)
Search
  • Make index in TermsLookup mandatory #25753 (issue: #25750)
  • Removes FieldStats API #25628 (issue: #25577)
  • Remove deprecated fielddata_fields from search request #25566 (issue: #25537)
  • Removes deprecated fielddata_fields #25537 (issue: #19027)
  • ProfileResult and CollectorResult should print machine readable timing information #22561
  • Remove indices query #21837 (issue: #17710)
  • Remove ignored type parameter in search_shards api #21688
Sequence IDs
  • Change certain replica failures not to fail the replica shard #22874 (issue: #10708)
Settings
  • Settings: Remove shared setting property #24728
  • Settings: Remove support for yaml and json config files #24664 (issue: #19391)
Shadow Replicas
Similarities
  • Similarity should accept dynamic settings when possible #20339 (issue: #6727)

Breaking Java Changes

edit
Aggregations
  • Remove the unused SignificantTerms.compareTerm() method #24714
  • Make SignificantTerms.Bucket an interface rather than an abstract class #24670 (issue: #24492)
  • Fix NPE when values is omitted on percentile_ranks agg #26046
  • Make Terms.Bucket an interface rather than an abstract class #24492
  • Compound order for histogram aggregations #22343 (issues: #14771, #20003, #23613)
Internal
  • Collapses package structure for some bucket aggs #25579 (issue: #22868)
Java API
  • Remove deprecated IdsQueryBuilder ctor #25529
  • Removing unneeded getTookInMillis method #23923
  • Java api: ActionRequestBuilder#execute to return a PlainActionFuture #24415 (issues: #24412, #9201)
Java High Level REST Client
  • Unify the result interfaces from get and search in Java client #25361 (issue: #16440)
  • Allow RestHighLevelClient to use plugins #25024
Java REST Client
Network
  • Simplify TransportAddress #20798
Plugin Delete By Query
  • Move DeleteByQuery and Reindex requests into core #24578
Plugins
  • Drop name from TokenizerFactory #24869
Query DSL
  • Remove QueryParseContext #25486
  • Remove QueryParseContext from parsing QueryBuilders #25448
REST
  • Return index name and empty map for /{index}/_alias with no aliases #25114 (issues: #24723, #25090)

Deprecations

edit
Index APIs
  • Deprecated use of + in index expressions #24585 (issue: #24515)
Index Templates
  • Restore deprecation warning for invalid match_mapping_type values #22304
Indexed Scripts/Templates
  • Scripting: Deprecate stored search template apis #25437 (issue: #24596)
Internal
  • Deprecate XContentType auto detection methods in XContentFactory #22181 (issue: #19388)
Percolator
  • Deprecate percolate query’s document_type parameter. #25199
Plugins
  • Plugins: Add backcompat for sha1 checksums #26748 (issue: #26746)
Scripting
  • Scripting: Change keys for inline/stored scripts to source/id #25127
  • Scripting: Deprecate native scripts #24692 (issue: #19966)
  • Scripting: Deprecate index lookup #24691 (issue: #19359)
  • Deprecate Fine Grain Settings for Scripts #24573 (issue: #24532)
  • Scripting: Deprecate file script settings #24555 (issue: #21798)
  • Scripting: Deprecate file scripts #24552 (issue: #21798)
Settings
  • Settings: Update settings deprecation from yml to yaml #24663 (issue: #19391)
  • Deprecate settings in .yml and .json #24059 (issue: #19391)
Tribe Node

New Features

edit
Aggregations
  • SignificantText aggregation - like significant_terms, but for text #24432 (issue: #23674)
Analysis
  • Expose simplepattern and simplepatternsplit tokenizers #25159 (issue: #23363)
  • Parse synonyms with the same analysis chain #8049 (issue: #7199)
Core
Internal
  • Automatically adjust search threadpool queue_size #23884 (issue: #3890)
Mapping
  • Add new ip_range field type #24433
Parent/Child
  • Move parent_id query to the parent-join module #25072 (issue: #20257)
  • Introduce ParentJoinFieldMapper, a field mapper that creates parent/child relation within documents of the same index #24978 (issue: #20257)
Plugin Analysis ICU
  • Add ICUCollationFieldMapper #24126
Search
  • Automatically early terminate search query based on index sorting #24864 (issue: #6720)
Sequence IDs
  • Add a scheduled translog retention check #25622 (issues: #10708, #25294)
  • Initialize sequence numbers on a shrunken index #25321 (issue: #10708)
  • Initialize primary term for shrunk indices #25307 (issue: #10708)
  • Introduce translog size and age based retention policies #25147 (issue: #10708)
Stats
  • Adds nodes usage API to monitor usages of actions #24169
Task Manager
  • Task Management [ISSUE] #15117
Upgrade API
  • TemplateUpgraders should be called during rolling restart #25263 (issues: #24379, #24680)

Enhancements

edit
Aggregations
  • Add strict parsing of aggregation ranges #25769
  • Adds rewrite phase to aggregations #25495 (issue: #17676)
  • Tweak AggregatorBase.addRequestCircuitBreakerBytes #25162 (issue: #24511)
  • Add superset size to Significant Term REST response #24865
  • Add document count to Matrix Stats aggregation response #24776
  • Adds an implementation of LogLogBeta for the cardinality aggregation #22323 (issue: #22230)
  • Support distance units in GeoHashGrid aggregation precision #26291 (issue: #5042)
  • Reject multiple methods in percentiles aggregation #26163 (issue: #26095)
  • Use global_ordinals_hash execution mode when sorting by sub aggregations. #26014 (issue: #24359)
  • Add a specialized deferring collector for terms aggregator #25190
  • Agg builder accessibility fixes #24323
  • Remove support for the include/pattern syntax. #23141 (issue: #22933)
  • Promote longs to doubles when a terms agg mixes decimal and non-decimal numbers #22449 (issue: #22232)
Allocation
  • Adjust status on bad allocation explain requests #25503 (issue: #25458)
  • Promote replica on the highest version node #25277 (issue: #10708)
Analysis
  • [Analysis] Support normalizer in request param #24767 (issue: #23347)
  • Enforce validation for PathHierarchy tokenizer #23510
  • [analysis-icu] Allow setting unicodeSetFilter #20814 (issue: #20820)
  • Match- and MultiMatchQueryBuilder should only allow setting analyzer on string values #23684 (issue: #21665)
Bulk
  • Simplify bulk request execution #20109
CAT API
  • expand /_cat/nodes to return information about hard drive #21775 (issue: #21679)
CRUD
  • Added validation for upsert request #24282 (issue: #16671)
Circuit Breakers
  • ScriptService: Replace max compilation per minute setting with max compilation rate #26399
Cluster
  • Validate a joining node’s version with version of existing cluster nodes #25808
  • Switch indices read-only if a node runs out of disk space #25541 (issue: #24299)
  • Add a cluster block that allows to delete indices that are read-only #24678
  • Separate publishing from applying cluster states #24236
  • Adds cluster state size to /_cluster/state response #23440 (issue: #3415)
Core
  • Allow InputStreamStreamInput array size validation where applicable #26692
  • Refactor bootstrap check results and error messages #26637
  • Add BootstrapContext to expose settings and recovered state to bootstrap checks #26628
  • Unit testable index creation task on MetaDataCreateIndexService #25961
  • Ignore .DS_Store files on macOS #27108 (issue: #23982)
  • Add max file size bootstrap check #25974
  • Add compatibility versions to main action response #25799
  • Index ids in binary form. #25352 (issues: #18154, #24615)
  • Explicitly reject duplicate data paths #25178
  • Use SPI in High Level Rest Client to load XContent parsers #25097
  • Upgrade to lucene-7.0.0-snapshot-a0aef2f #24775
  • Speed up PK lookups at index time. #19856
  • Use Java 9 FilePermission model #26302 (issue: #21534)
  • Add friendlier message on bad keystore permissions #26284
  • Epoch millis and second formats accept float implicitly #26119 (issue: #14641)
  • Remove connect SocketPermissions from core #22797
  • Add repository-url module and move URLRepository #22752 (issue: #22116)
  • Remove accept SocketPermissions from core #22622 (issue: #22116)
  • Move IfConfig.logIfNecessary call into bootstrap #22455 (issue: #22116)
  • Remove artificial default processors limit #20874 (issue: #20828)
  • Simplify write failure handling #19105 (issue: #20109)
  • Improve bootstrap checks error messages #24548
Discovery
  • Allow plugins to validate cluster-state on join #26595
Engine
  • Add refresh stats tracking for realtime get #25052 (issue: #24806)
  • Introducing a translog deletion policy #24950
  • Fill missing sequence IDs up to max sequence ID when recovering from store #24238 (issue: #10708)
  • Use sequence numbers to identify out of order delivery in replicas & recovery #24060 (issue: #10708)
  • Add replica ops with version conflict to translog #22626
  • Clarify global checkpoint recovery #21934 (issue: #21254)
  • Move the IndexDeletionPolicy to be engine internal #24930 (issue: #10708)
Exceptions
  • IllegalStateException: Only duplicated jar instead of classpath #24953
Highlighting
  • Picks offset source for the unified highlighter directly from the es mapping #25747 (issue: #25699)
Index APIs
  • Let primary own its replication group #25692 (issue: #25485)
  • Create index request should return the index name #25139 (issue: #23044)
Index Templates
  • Fix error message for a put index template request without index_patterns #27102 (issue: #27100)
Ingest
  • Add Ingest-Processor specific Rest Endpoints & Add Grok endpoint #25059 (issue: #24725)
  • Port support for commercial GeoIP2 databases from Logstash. #24889
  • add exclude_keys option to KeyValueProcessor #24876 (issue: #23856)
  • Allow removing multiple fields in ingest processor #24750 (issue: #24622)
  • Add target_field parameter to ingest processors #24133 (issues: #23228, #23682)
Inner Hits
Internal
  • TemplateUpgradeService should only run on the master #27294
  • Cleanup IndexFieldData visibility #25900
  • Bump the min compat version to 5.6.0 #25805
  • "shard started" should show index and shard ID #25157
  • Break out clear scroll logic from TransportClearScrollAction #25125 (issue: #25094)
  • Add helper methods to TransportActionProxy to identify proxy actions and requests #25124
  • Add remote cluster infrastructure to fetch discovery nodes. #25123 (issue: #25094)
  • Add the ability to set eager_global_ordinals in the new parent-join field #25019
  • Disallow multiple parent-join fields per mapping #25002
  • Remove the need for _UNRELEASED suffix in versions #24798 (issue: #24768)
  • Optimize the order of bytes in uuids for better compression. #24615 (issue: #18209)
  • Prevent cluster internal ClusterState.Custom impls to leak to a client #26232
  • Use holder pattern for lazy deprecation loggers #26218 (issue: #26210)
  • Allow ClusterState.Custom to be created on initial cluster states #26144
  • Try to convince the JVM not to lose stacktraces #24426 (issue: #24376)
  • Make document write requests immutable #23038
  • Add assertions enabled helper #24834
Java API
  • Always Accumulate Transport Exceptions #25017 (issue: #23099)
Java High Level REST Client
Java REST Client
  • Shade external dependencies in the rest client jar #25780 (issue: #25208)
  • RestClient uses system properties and system default SSLContext #25757 (issue: #23231)
  • Wrap rest httpclient with doPrivileged blocks #22603 (issue: #22116)
Logging
  • Prevent excessive disk consumption by log files #25660
  • Use LRU set to reduce repeat deprecation messages #25474 (issue: #25457)
Mapping
  • More efficient encoding of range fields. #26470 (issue: #26443)
  • Don’t detect source’s XContentType in DocumentParser.parseDocument() #26880
  • Better validation of copy_to. #25983
  • Optimize terms queries on ip addresses to use a PointInSetQuery whenever possible. #25669 (issue: #25667)
  • Loosen the restrictions on disabling _all in 6.x #26259
  • Date detection should not rely on a hardcoded set of characters. #22171 (issue: #1694)
  • Identify documents by their _id. #24460
Network
  • Add additional low-level logging handler #26887
  • Unwrap causes when maybe dying #26884
  • Move TransportStats accounting into TcpTransport #25251
  • Simplify connection closing and cleanups in TcpTransport #25250
  • Disable the Netty recycler in the client #24793 (issues: #22452, #24721)
  • Remove Netty logging hack #24653 (issues: #24469, #5624, #6568, #6696)
  • Isolate SocketPermissions to Netty #23057
  • Wrap netty accept/connect ops with doPrivileged #22572 (issue: #22116)
  • Replace Socket, ServerSocket, and HttpServer usages in tests with mocksocket versions #22287 (issue: #22116)
Packaging
  • Remove memlock suggestion from systemd service #25979
  • Set address space limit in systemd service file #25975
  • Version option should display if snapshot #25970
  • Ignore JVM options before checking Java version #25969
  • Also skip JAVA_TOOL_OPTIONS on Windows #25968
  • Introduce elasticsearch-env for Windows #25958
  • Introduce elasticsearch-env #25815 (issue: #20286)
  • Stop exporting HOSTNAME from scripts #25807
  • Set number of processes in systemd unit file #24970 (issue: #20874)
Parent/Child
  • Remove ParentJoinFieldSubFetchPhase #25550 (issue: #25363)
  • Support parent id being specified as number in the _source #25547
Percolator
  • Store the QueryBuilder’s Writable representation instead of its XContent representation #25456
  • Add support for selecting percolator query candidate matches containing wildcard / prefix queries #25351
Plugin Discovery EC2
  • Read ec2 discovery address from aws instance tags #22743 (issue: #22566)
Plugin Lang Painless
  • Allow Custom Whitelists in Painless #25557
  • Update Painless to Allow Augmentation from Any Class #25360
  • Add Needs Methods to Painless Script Context Factories #25267
  • Support Script Context Stateful Factory in Painless #25233
  • Generate Painless Factory for Creating Script Instances #25120
  • Update Painless to Use New Script Contexts #25015
  • Optimize instance creation in LambdaBootstrap #24618
  • Make Painless Compiler Use an Instance Per Context #24972
  • Make PainlessScript An Interface #24966
Plugin Repository GCS
  • GCS Repository: Add secure storage of credentials #24697
Plugin Repository HDFS
  • Add permission checks before reading from HDFS stream #26716 (issue: #26714)
  • Add doPrivilege blocks for socket connect ops in repository-hdfs #22793 (issue: #22116)
  • Add Kerberos support for Repo HDFS plugin [ISSUE] #21990
Plugin Repository S3
  • S3 Repository: Add back repository level credentials #24609
Plugins
  • Adjust SHA-512 supported format on plugin install #27093
  • Move tribe to a module #25778
  • Plugins can register pre-configured char filters #25000 (issue: #23658)
  • Add purge option to remove plugin CLI #24981
  • Allow plugins to register pre-configured tokenizers #24751 (issues: #24223, #24572)
  • Move ReindexAction class to core #24684 (issue: #24578)
  • Make PreConfiguredTokenFilter harder to misuse #24572 (issue: #23658)
  • Plugins: Remove leniency for missing plugins dir #24173
  • Add doPrivilege blocks for socket connect operations in plugins #22534 (issue: #22116)
Query DSL
  • Make slop optional when parsing span_near query #25677 (issue: #25642)
  • Require a field when a seed is provided to the random_score function. #25594 (issue: #25240)
  • Add support for auto_generate_synonyms_phrase_query in match_query, multi_match_query, query_string and simple_query_string #23147
REST
  • Cat shards bytes #26952
  • Refactor PathTrie and RestController to use a single trie for all methods #25459 (issue: #24437)
  • Make ObjectParser support string to boolean conversion #24668 (issue: #21802)
Recovery
  • Introduce a History UUID as a requirement for ops based recovery #26577 (issue: #10708)
  • Goodbye, Translog Views #25962
  • Disallow multiple concurrent recovery attempts for same target shard #25428
  • Live primary-replica resync (no rollback) #24841 (issue: #10708)
  • Peer Recovery: remove maxUnsafeAutoIdTimestamp hand off #24243 (issue: #24149)
  • Introduce sequence-number-based recovery #22484 (issue: #10708)
Scripting
  • Scripting: Rename SearchScript.needsScores to needs_score #25235
  • Scripting: Add optional context parameter to put stored script requests #25014
  • Add New Security Script Settings #24637 (issue: #24532)
  • Add StatefulFactoryType as optional intermediate factory in script contexts #24974 (issue: #20426)
  • Make contexts available to ScriptEngine construction #24896
  • Make ScriptEngine.compile generic on the script context #24873
  • Add instance and compiled classes to script contexts #24868
Search
  • Add soft limit on allowed number of script fields in request #26598 (issue: #26390)
  • Add a soft limit for the number of requested doc-value fields #26574 (issue: #26390)
  • Rewrite search requests on the coordinating nodes #25814 (issue: #25791)
  • Ensure query resources are fetched asynchronously during rewrite #25791
  • Introduce a new Rewriteable interface to streamline rewriting #25788
  • Reduce the scope of QueryRewriteContext #25787
  • Reduce the overhead of timeouts and low-level search cancellation. #25776
  • Reduce profiling overhead. #25772 (issue: #24799)
  • Prevent can_match requests from sending to incompatible nodes #25705 (issue: #25704)
  • Add a shard filter search phase to pre-filter shards based on query rewriting #25658
  • Ensure we rewrite common queries to match_none if possible #25650
  • Limit the number of concurrent shard requests per search request #25632
  • Add cluster name validation to RemoteClusterConnection #25568
  • Speed up sorted scroll when the index sort matches the search sort #25138 (issue: #6720)
  • Leverage scorerSupplier when applicable. #25109
  • Add cross-cluster search support for scroll searches #25094
  • Track EWMA[1] of task execution time in search threadpool executor #24989 (issue: #24915)
  • Query range fields by doc values when they are expected to be more efficient than points #24823 (issue: #24314)
  • Search: Fairer balancing when routing searches by session ID #24671 (issue: #24642)
  • Add parsing from xContent to Suggest #22903
  • Add parsing from xContent to ShardSearchFailure #22699
  • Eliminate array access in tight loops when profiling is enabled. #24959
  • Support Multiple Inner Hits on a Field Collapse Request #24517
  • Expand cross-cluster search indices for search requests to the concrete index or to it’s aliases #24502
Search Templates
Sequence IDs
  • Roll translog generation on primary promotion #27313
  • Restoring from snapshot should force generation of a new history uuid #26694 (issues: #10708, #26544, #26557, #26577)
  • Add global checkpoint tracking on the primary #26666 (issue: #26591)
  • Introduce global checkpoint background sync #26591 (issues: #26573, #26630, #26666)
  • Move UNASSIGNED_SEQ_NO and NO_OPS_PERFORMED to SequenceNumbers` #26494 (issue: #10708)
  • Move primary term from ReplicationRequest to ConcreteShardRequest #25822
  • Add reason to global checkpoint updates on replica #25612 (issue: #10708)
  • Introduce primary/replica mode for GlobalCheckPointTracker #25468
  • Throw back replica local checkpoint on new primary #25452 (issues: #10708, #25355)
  • Update global checkpoint when increasing primary term on replica #25422 (issues: #10708, #25355)
  • Enable a long translog retention policy by default #25294 (issues: #10708, #25147)
  • Introduce primary context #25122 (issues: #10708, #25355)
  • Block older operations on primary term transition #24779 (issue: #10708)
  • Block global checkpoint advances when recovering #24404 (issue: #10708)
  • Add primary term to doc write response #24171 (issue: #10708)
  • Preserve multiple translog generations #24015 (issue: #10708)
  • Introduce translog generation rolling #23606 (issue: #10708)
  • Replicate write failures #23314
  • Introduce sequence-number-aware translog #22822 (issue: #10708)
  • Introduce translog no-op #22291 (issue: #10708)
  • Tighten sequence numbers recovery #22212 (issue: #10708)
  • Add BWC layer to seq no infra and enable BWC tests #22185 (issue: #21670)
  • Add internal _primary_term doc values field, fix _seq_no indexing #21637 (issues: #10708, #21480)
  • Add global checkpoint to translog checkpoints #21254
  • Sequence numbers commit data for Lucene uses Iterable interface #20793 (issue: #10708)
  • Simplify GlobalCheckpointService and properly hook it for cluster state updates #20720
  • Fill gaps on primary promotion #24945 (issue: #10708)
  • Introduce clean transition on primary promotion #24925 (issue: #10708)
  • Guarantee that translog generations are seqNo conflict free #24825 (issues: #10708, #24779)
  • Inline global checkpoints #24513 (issue: #10708)
Settings
  • Add disk threshold settings validation #25600 (issue: #25560)
  • Enable cross-setting validation #25560 (issue: #25541)
  • Validate transport.profiles.* settings #25508
  • Cleanup network / transport related settings #25489
  • Emit settings deprecation logging at most once #25457
  • IndexMetaData: Introduce internal format index setting #25292
  • Persist created keystore on startup unless keystore is present #26253 (issue: #26126)
  • Settings: Add keystore.seed auto generated secure setting #26149
  • Settings: Add keystore creation to add commands #26126
Snapshot/Restore
  • Fixed references to Multi Index Syntax #27283
  • Improves snapshot logging and snapshot deletion error handling #25264
  • Enhances get snapshots API to allow retrieving repository index only #24477 (issue: #24288)
Stats
  • Update IndexShard#refreshMetric via a ReferenceManager.RefreshListener #25083 (issues: #24806, #25052)
  • Expose disk usage estimates in nodes stats #22081 (issue: #8686)
Store
  • Remote support for lucene versions without checksums #24021
Suggesters
  • Remove deprecated _suggest endpoint #22203 (issue: #20305)
Task Manager
Translog
  • Translog file recovery should not rely on lucene commits #25005 (issue: #24950)

Bug Fixes

edit
Aggregations
  • Do not delegate a null scorer to LeafBucketCollectors #26747 (issue: #26611)
  • Create weights lazily in filter and filters aggregation #26983
  • Fix IndexOutOfBoundsException in histograms for NaN doubles (#26787) #26856 (issue: #26787)
  • Scripted_metric _agg parameter disappears if params are provided #19863 (issue: #19768)
  • Fixes array out of bounds for value count agg #26038 (issue: #17379)
  • Aggregations bug: Significant_text fails on arrays of text. #25030 (issue: #25029)
  • Check bucket metric ages point to a multi bucket agg #26215 (issue: #25775)
  • Terms aggregation should remap global ordinal buckets when a sub-aggregator is used to sort the terms #24941 (issue: #24788)
  • Correctly set doc_count when MovAvg "predicts" values on existing buckets #24892 (issue: #24327)
  • DateHistogram: Fix extended_bounds with offset #23789 (issue: #23776)
  • Fix ArrayIndexOutOfBoundsException when no ranges are specified in the query #23241 (issue: #22881)
Aliases
  • mget with an alias shouldn’t ignore alias routing #25697 (issue: #25696)
  • GET aliases should 404 if aliases are missing #25043 (issue: #24644)
Allocation
Analysis
  • Pre-configured shingle filter should disable graph analysis #25853 (issue: #25555)
  • PatternAnalyzer should lowercase wildcard queries when lowercase is true. #24967
CAT API
  • Fix NPE for /_cat/indices when no primary shard #26953 (issue: #26942)
CRUD
  • Serialize and expose timeout of acknowledged requests in REST layer #26189 (issue: #26213)
  • Fix silent loss of last command to _bulk and _msearch due to missing newline #25740 (issue: #7601)
Cache
  • Reduce the default number of cached queries. #26949 (issue: #26938)
  • fix bug of weight computation #24856
Circuit Breakers
  • Checks the circuit breaker before allocating bytes for a new big array #25010 (issue: #24790)
Cluster
  • Register setting cluster.indices.tombstones.size #26193 (issue: #26191)
Core
  • Correctly encode warning headers #27269 (issue: #27244)
  • Fix cache compute if absent for expired entries #26516
  • Timed runnable should delegate to abstract runnable #27095 (issue: #27069)
  • Stop invoking non-existent syscall #27016 (issue: #20179)
  • MetaData Builder doesn’t properly prevent an alias with the same name as an index #26804
  • Release operation permit on thread-pool rejection #25930 (issue: #25863)
  • Node should start up despite of a lingering .es_temp_file #21210 (issue: #21007)
  • Fix cache expire after access #24546
Dates
Discovery
  • MasterNodeChangePredicate should use the node instance to detect master change #25877 (issue: #25471)
Engine
  • Die with dignity while merging #27265 (issue: #19272)
  • Engine - do not index operations with seq# lower than the local checkpoint into lucene #25827 (issues: #1, #2, #25592)
Geo
  • Fix typo in GeoUtils#isValidLongitude #25121
Highlighting
  • Fix percolator highlight sub fetch phase to not highlight query twice #26622
  • FastVectorHighlighter should not cache the field query globally #25197 (issue: #25171)
  • Higlighters: Fix MultiPhrasePrefixQuery rewriting #25103 (issue: #25088)
  • Fix nested query highlighting #26305 (issue: #26230)
Index APIs
  • Shrink API should ignore templates #25380 (issue: #25035)
  • Rollover max docs should only count primaries #24977 (issue: #24217)
  • Validates updated settings on closed indices #24487 (issue: #23787)
Ingest
  • date processor should not fail if timestamp is specified as json number #26986 (issue: #26967)
  • date_index_name processor should not fail if timestamp is specified as json number #26910 (issue: #26890)
  • Sort Processor does not have proper behavior with targetField #25237 (issue: #24133)
  • fix grok’s pattern parsing to validate pattern names in expression #25063 (issue: #22831)
  • Remove support for Visio and potm files #22079 (issue: #22077)
  • Fix floating-point error when DateProcessor parses UNIX #24947
  • add option for _ingest.timestamp to use new ZonedDateTime (5.x backport) #24030 (issues: #23168, #23174)
Inner Hits
  • Do not allow inner hits that fetch _source and have a non nested object field as parent #25749 (issue: #25315)
  • When fetching nested inner hits only access stored fields when needed #25864 (issue: #6)
  • If size / offset are out of bounds just do a plain count #20556 (issue: #20501)
  • Fix Source filtering in new field collapsing feature #24068 (issue: #24063)
Internal
  • Bump version to 6.0.1 [OPEN] #27386
  • IndexShard.routingEntry should only be updated once all internal state is ready #26776
  • Catch exceptions and inform handler in RemoteClusterConnection#collectNodes #26725 (issue: #26700)
  • Internal: Add versionless alias for rest client codebase in policy files #26521
  • Upgrade Lucene to version 7.0.1 #26926
  • Fix BytesReferenceStreamInput#skip with offset #25634
  • Fix race condition in RemoteClusterConnection node supplier #25432
  • Initialise empty lists in BaseTaskResponse constructor #25290
  • Extract a common base class for scroll executions #24979 (issue: #16555)
  • Obey lock order if working with store to get metadata snapshots #24787 (issue: #24481)
  • Fix Version based BWC and set correct minCompatVersion #24732
  • Fix _field_caps serialization in order to support cross-cluster search #24722
  • Avoid race when shutting down controller processes #24579
  • Fix handling of document failure exception in InternalEngine #22718
  • Ensure remote cluster is connected before fetching _field_caps #24845 (issue: #24763)
Java API
  • BulkProcessor flush runnable preserves the thread context from creation time #26718 (issue: #26596)
Java High Level REST Client
  • Make RestHighLevelClient’s Request class public #26627 (issue: #26455)
  • Forbid direct usage of ContentType.create() methods #26457 (issues: #22769, #26438)
  • Make ShardSearchTarget optional when parsing ShardSearchFailure #27078 (issue: #27055)
Java REST Client
  • Better message text for ResponseException #26564
  • rest-client-sniffer: configurable threadfactory #26897
Logging
Mapping
  • Allow copying from a field to another field that belongs to the same nested object. #26774 (issue: #26763)
  • Fixed bug that mapper_parsing_exception is thrown for numeric field with ignore_malformed=true when inserting "NaN" #25967 (issue: #25289)
  • Coerce decimal strings for whole number types by truncating the decimal part #25835 (issue: #25819)
  • Fix parsing of ip range queries. #25768 (issue: #25636)
  • Disable date field mapping changing #25285 (issue: #25271)
  • Correctly enable _all for older 5.x indices #25087 (issue: #25068)
  • token_count datatype should handle null value #25046 (issue: #24928)
  • keep _parent field while updating child type mapping #24407 (issue: #23381)
  • ICUCollationKeywordFieldMapper use SortedSetDocValuesField #26267
  • Fix serialization of the _all field. #26143 (issue: #26136)
More Like This
  • Pass over _routing value with more_like_this items to be retrieved #24679 (issue: #23699)
NOT CLASSIFIED
  • DocumentMissingException during Logstash scripted upsert [ISSUE] #27148
  • An assertion trips when master opens an index from before 5.x [ISSUE] #24809
Nested Docs
  • In case of a single type the _id field should be added to the nested document instead of _uid field #25149
  • Inner hits source filtering not working [ISSUE] #23090
Network
  • Fixed ByteBuf leaking in org.elasticsearch.http.netty4.Netty4HttpRequestHandler #27222 (issues: #3, #4, #5, #6)
  • Check for closed connection while opening #26932
  • Ensure pending transport handlers are invoked for all channel failures #25150
  • Notify onConnectionClosed rather than onNodeDisconnect to prune transport handlers #24639 (issues: #24557, #24575, #24632)
  • Release pipelined http responses on close #26226
  • Fix error message if an incompatible node connects #24884
Packaging
  • Fix handling of Windows paths containing parentheses #26916 (issue: #26454)
  • Exit Windows scripts promptly on failure #25959
  • Pass config path as a system property #25943
  • ES_HOME needs to be made absolute before attempt at traversal #25865
  • Fix elasticsearch-keystore handling of path.conf #25811
  • Stop disabling explicit GC #25759
  • Avoid failing install if system-sysctl is masked #25657 (issue: #24234)
  • Get short path name for native controllers #25344
  • When stopping via systemd only kill the JVM, not its control group #25195
  • remove remaining references to scripts directory #24771
  • Handle parentheses in batch file path #24731 (issue: #24712)
  • Detect modified keystore on package removal #26300
  • Create keystore on RPM and Debian package install #26282
  • Add safer empty variable checking for Windows #26268 (issue: #26261)
  • Export HOSTNAME environment variable #26262 (issues: #25807, #26255)
  • Fix daemonization command status test #26196 (issue: #26080)
  • Set RuntimeDirectory in systemd service #23526
Parent/Child
  • The default _parent field should not try to load global ordinals #25851 (issue: #25849)
Percolator
  • Also support query extraction for queries wrapped inside a ESToParentBlockJoinQuery #26754
  • Fix range queries with date range based on current time in percolator queries. #24666 (issue: #23921)
Plugin Analysis Kuromoji
Plugin Analysis Phonetic
  • Fix beidermorse phonetic token filter for unspecified languageset #27112 (issue: #26771)
Plugin Discovery File
  • Fix discovery-file plugin to use custom config path #26662 (issue: #26660)
Plugin Ingest Attachment
Plugin Lang Painless
  • Painless: allow doubles to be casted to longs. #25936
Plugin Repository Azure
  • Azure snapshots can not be restored anymore #26778 (issues: #22858, #26751, #26777)
  • Snapshot : azure module - accelerate the listing of files (used in delete snapshot) #25710 (issue: #25424)
  • Use Azure upload method instead of our own implementation #26751
  • Make calls to CloudBlobContainer#exists privileged #25937 (issue: #25931)
Plugin Repository GCS
  • Ensure that gcs client creation is privileged #25938 (issue: #25932)
Plugin Repository HDFS
  • Add Log4j to SLF4J binding for repository-hdfs #26514 (issue: #26512)
  • Upgrading HDFS Repository Plugin to use HDFS 2.8.1 Client #25497 (issue: #25450)
Plugin Repository S3
  • Avoid SecurityException in repository-S3 on DefaultS3OutputStream.flush() #25254 (issue: #25192)
  • Wrap getCredentials() in a doPrivileged() block #23297 (issues: #22534, #23271)
Plugins
  • X-Pack plugin download fails on Windows desktop [ISSUE] #24570
  • Fix plugin installation permissions #24527 (issue: #24480)
Query DSL
  • Fixed incomplete JSON body on count request making org.elasticsearch.rest.action.RestActions#parseTopLevelQueryBuilder go into endless loop #26680 (issue: #26083)
  • SpanNearQueryBuilder should return the inner clause when a single clause is provided #25856 (issue: #25630)
  • Refactor field expansion for match, multi_match and query_string query #25726 (issues: #25551, #25556)
  • WrapperQueryBuilder should also rewrite the parsed query #25480
REST
  • Rest test fixes #27354
  • Fix inconsistencies in the rest api specs for cat.snapshots #26996 (issues: #25737, #26923)
  • Fix inconsistencies in the rest api specs for *_script #26971 (issue: #26923)
  • exists template needs a template name #25988
  • Fix handling of invalid error trace parameter #25785 (issue: #25774)
  • Fix handling of exceptions thrown on HEAD requests #25172 (issue: #21125)
  • Fixed NPEs caused by requests without content. #23497 (issue: #24701)
  • Fix get mappings HEAD requests #23192 (issue: #21125)
Recovery
  • Close translog view after primary-replica resync #25862 (issue: #24841)
Reindex API
  • Fix update_by_query’s default size parameter #26784 (issue: #26761)
  • Reindex: don’t duplicate _source parameter #24629 (issue: #24628)
  • Add qa module that tests reindex-from-remote against pre-5.0 versions of Elasticsearch #24561 (issues: #23828, #24520)
Scroll
  • Fix single shard scroll within a cluster with nodes in version >= 5.3 and <= 5.3 #24512
Search
  • Fail query when a sort is provided in conjunction with rescorers #26510
  • Let search phases override max concurrent requests #26484 (issue: #26198)
  • Avoid stack overflow on search phases #27069 (issue: #27042)
  • Fix search_after with geo distance sorting #26891
  • Fix serialization errors when cross-cluster search goes to a single shard #26881 (issue: #26833)
  • Early termination with index sorting should not set terminated_early in the response #26597 (issue: #26408)
  • Format doc values fields. #22146
  • Fix term(s) query for range field #25918
  • Caching a MinDocQuery can lead to wrong results. #25909
  • Fix random score generation when no seed is provided. #25908
  • Merge FunctionScoreQuery and FiltersFunctionScoreQuery #25889 (issues: #15709, #23628)
  • Respect cluster alias in _index aggs and queries #25885 (issue: #25606)
  • First increment shard stats before notifying and potentially sending response #25818
  • Remove assertion about deviation when casting to a float. #25806 (issue: #25330)
  • Prevent skipping shards if a suggest builder is present #25739 (issue: #25658)
  • Ensure remote cluster alias is preserved in inner hits aggs #25627 (issue: #25606)
  • Do not search locally if remote index pattern resolves to no indices #25436 (issue: #25426)
  • Adds check for negative search request size #25397 (issue: #22530)
  • Make sure range queries are correctly profiled. #25108
  • Fix RangeFieldMapper rangeQuery to properly handle relations #24808 (issue: #24744)
  • Fix ExpandSearchPhase when response contains no hits #24688 (issue: #24672)
  • Refactor simple_query_string to handle text part like multi_match and query_string #26145 (issue: #25726)
  • Fix _exists_ in query_string on empty indices. #25993 (issue: #25956)
  • Fix script field sort returning Double.MAX_VALUE for all documents #24942 (issue: #24940)
  • Compute the took time of the query after the expand phase of field collapsing #24902 (issue: #24900)
Sequence IDs
  • Fire global checkpoint sync under system context #26984
  • Fix pre-6.0 response to unknown replication actions #25744 (issue: #10708)
  • Track local checkpoint on primary immediately #25434 (issues: #10708, #25355, #25415)
  • Initialize max unsafe auto ID timestamp on shrink #25356 (issues: #10708, #25355)
  • Use correct primary term for replicating NOOPs #25128
  • Handle already closed while filling gaps #25021 (issue: #24925)
  • TranslogWriter.assertNoSeqNumberConflict failure [ISSUE] #26710
  • Avoid losing ops in file-based recovery #22945 (issue: #22484)
  • Handle primary failure handling replica response #24926 (issue: #24935)
Settings
  • Emit settings deprecation logging on empty update #27017 (issue: #26419)
  • Fix filtering for ListSetting #26914
  • Fix settings serialization to not serialize secure settings or not take the total size into account #25323
  • Keystore CLI should use the AddFileKeyStoreCommand for files #25298
  • Allow resetting settings that use an IP validator #24713 (issue: #24709)
  • Updating an unrecognized setting should error out with that reason [ISSUE] #25607
  • Settings: Fix setting groups to include secure settings #25076 (issue: #25069)
Similarities
  • Add boolean similarity to built in similarity types #26613
Snapshot/Restore
  • Snapshot/Restore: better handle incorrect chunk_size settings in FS repo #26844 (issue: #26843)
  • Snapshot/Restore: Ensure that shard failure reasons are correctly stored in CS #25941 (issue: #25878)
  • Output all empty snapshot info fields if in verbose mode #25455 (issue: #24477)
  • Remove redundant and broken MD5 checksum from repository-s3 #25270 (issue: #25269)
  • Consolidates the logic for cleaning up snapshots on master election #24894 (issue: #24605)
  • Removes completed snapshot from cluster state on master change #24605 (issue: #24452)
  • Keep snapshot restore state and routing table in sync #20836 (issue: #19774)
  • Master failover during snapshotting could leave the snapshot incomplete [OPEN] [ISSUE] #25281
  • Fix inefficient (worst case exponential) loading of snapshot repository #24510 (issue: #24509)
Stats
  • Fix RestGetAction name typo #27266
  • Keep cumulative elapsed scroll time in microseconds #27068 (issue: #27046)
  • _nodes/stats should not fail due to concurrent AlreadyClosedException #25016 (issue: #23099)
  • Avoid double decrement on current query counter #24922 (issues: #22996, #24872)
  • Adjust available and free bytes to be non-negative on huge FSes #24911 (issues: #23093, #24453)
Suggesters
  • Fix division by zero in phrase suggester that causes assertion to fail #27149
  • Context suggester should filter doc values field #25858 (issue: #25404)
  • Fix context suggester to read values from keyword type field #24200 (issue: #24129)
Templates
  • Tests: Fix FullClusterRestartIT.testSnapshotRestore test failing in 6.x #27218 (issue: #27213)
Translog
  • Fix Translog.Delete serialization for sequence numbers #22543
Upgrade API
  • Upgrade API: fix excessive logging and unnecessary template updates #26698 (issue: #26673)

Regressions

edit
Bulk
  • Only re-parse operation if a mapping update was needed #23832 (issue: #23665)
Highlighting
  • Fix Fast Vector Highlighter NPE on match phrase prefix #25116 (issue: #25088)
Search
  • Always use DisjunctionMaxQuery to build cross fields disjunction #25115 (issue: #23966)
Sequence IDs
  • Indexing performance degradation in 6.0.0-beta1 [ISSUE] #26339

Upgrades

edit
Core
Logging
Network
Plugin Ingest Attachment
Upgrade API
  • Improve stability and logging of TemplateUpgradeServiceIT tests #25386 (issue: #25382)
« Elasticsearch version 6.0.1 Elasticsearch version 6.0.0-rc2 »