IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Elasticsearch version 6.0.1 Elasticsearch version 6.0.0-rc2 »
Elastic Docs ›Elasticsearch Guide [6.7] ›Release Notes

Elasticsearch version 6.0.0

edit

Elasticsearch version 6.0.0

edit

Breaking Changes

edit
Aggregations
  • Change parsing of numeric to and from parameters in date_range aggregation #25376 (issue: #17920)
Aliases
Allocation
  • Remove cluster.routing.allocation.snapshot.relocation_enabled setting #20994
Analysis
  • Do not allow custom analyzers to have the same names as built-in analyzers #22349 (issue: #22263)
  • Removing query-string parameters in _analyze API #20704 (issue: #20246)
CAT API
  • Write -1 on unbounded queue in cat thread pool #21342 (issue: #21187)
CRUD
  • Disallow VersionType.FORCE for GetRequest #21079 (issue: #20995)
  • Disallow VersionType.FORCE versioning for 6.x indices #20995 (issue: #20377)
  • If the index does not exist, delete document will not auto create it #24518 (issue: #15425)
Cluster
  • Disallow : in cluster and index/alias names #26247 (issue: #23892)
  • No longer allow cluster name in data path #20433 (issue: #20391)
Core
  • Simplify file store #24402 (issue: #24390)
  • Make boolean conversion strict #22200
  • Remove the default store type. #21616
  • Remove store throttling. #21573
Geo
  • Remove deprecated geo search features #22876
  • Reduce GeoDistance Insanity #19846
Highlighting
  • Remove the postings highlighter and make unified the default highlighter choice #25028
Index APIs
  • Remove (deprecated) support for + in index expressions #25274 (issue: #24515)
  • Delete index API to work only against concrete indices #25268 (issues: #2318, #23997)
  • Open/Close index api to allow_no_indices by default #24401 (issues: #24031, #24341)
  • Remove support for controversial ignore_unavailable and allow_no_indices from indices exists api #20712
Index Templates
  • Allows multiple patterns to be specified for index templates #21009 (issue: #20690)
Indexed Scripts/Templates
  • Scripting: Remove search template actions #25717
Ingest
  • update ingest-user-agent regexes.yml #25608
  • remove ingest.new_date_format #25583
Inner Hits
  • Return the _source of inner hit nested as is without wrapping it into its full path context #26982 (issues: #26102, #26944)
Java API
  • Enforce Content-Type requirement on the rest layer and remove deprecated methods #23146 (issue: #19388)
Java REST Client
Mapping
  • Reject out of range numbers for float, double and half_float #25826 (issue: #25534)
  • Enforce at most one type. #24428 (issue: #24317)
  • Disallow include_in_all for 6.0+ indices #22970 (issue: #22923)
  • Disable _all by default, disallow configuring _all on 6.0+ indices #22144 (issues: #19784, #20925, #21341)
  • Throw an exception on unrecognized "match_mapping_type" #22090 (issue: #17285)
Network
  • Remove unused Netty-related settings #26161
  • Remove blocking TCP clients and servers #22639
  • Remove modules/transport_netty_3 in favor of netty_4 #21590
  • Remove LocalTransport in favor of MockTcpTransport #20695
Packaging
  • Configure heap dump path out of the box #26755 (issue: #26665)
  • Remove support for ES_INCLUDE #25804
  • Setup: Change default heap to 1G #25695
  • Use config directory to find jvm.options #25679 (issue: #23004)
  • Remove implicit 32-bit support #25435
  • Remove default path settings #25408 (issue: #25357)
  • Remove path.conf setting #25392 (issue: #25357)
  • Honor masking of systemd-sysctl.service #24234 (issues: #21899, #806)
  • Rename CONF_DIR to ES_PATH_CONF #26197 (issue: #26154)
  • Remove customization of ES_USER and ES_GROUP #23989 (issue: #23848)
Percolator
  • Remove deprecated percolate and mpercolate apis #22331
Plugin Analysis ICU
  • Upgrade icu4j for the ICU analysis plugin to 59.1 #25243 (issue: #21425)
  • Upgrade icu4j to latest version #24821
Plugin Delete By Query
  • Require explicit query in _delete_by_query API #23632 (issue: #23629)
Plugin Discovery Azure Classic
  • Remove discovery.type BWC layer from the EC2/Azure/GCE plugins #25080 (issue: #24543)
Plugin Discovery EC2
  • Ec2 Discovery: Cleanup deprecated settings #24150
  • Discovery EC2: Remove region setting #23991 (issue: #22758)
  • AWS Plugins: Remove signer type setting #23984 (issue: #22599)
Plugin Lang JS
  • Remove lang-python and lang-javascript #20734 (issue: #20698)
Plugin Mapper Attachment
Plugin Repository Azure
  • Remove global repositories.azure settings #23262 (issues: #22800, #22856)
  • Remove auto creation of container for azure repository #22858 (issue: #22857)
Plugin Repository GCS
  • GCS Repository: Remove specifying credential file on disk #24727
Plugin Repository S3
  • S3 Repository: Cleanup deprecated settings #24097
  • S3 Repository: Remove region setting #22853 (issue: #22758)
  • S3 Repository: Remove bucket auto create #22846 (issue: #22761)
  • S3 Repository: Remove env var and sysprop credentials support #22842
  • Remove deprecated S3 settings #24445
Plugins
  • Make plugin loading stricter #25405
Query DSL
  • Remove deprecated type and slop field in match query #26720
  • Remove several parse field deprecations in query builders #26711
  • Remove deprecated parameters from ids_query #26508
  • Refactor QueryStringQuery for 6.0 #25646 (issue: #25574)
  • Change split_on_whitespace default to false #25570 (issue: #25470)
  • Remove deprecated template query #24577 (issue: #19390)
  • Throw exception in scroll requests using from #26235 (issue: #9373)
  • Remove deprecated minimum_number_should_match in BoolQueryBuilder #22416
  • Remove support for empty queries #22092 (issue: #17624)
  • Remove deprecated query names: in, geo_bbox, mlt, fuzzy_match and match_fuzzy #21852
  • The terms query should always map to a Lucene TermsQuery. #21786
  • Be strict when parsing values searching for booleans #21555 (issue: #21545)
  • Remove collect payloads parameter #20385
REST
  • IndexClosedException to return 400 rather than 403 #25752
  • Remove comma-separated feature parsing for GetIndicesAction #24723 (issue: #24437)
  • Improve REST error handling when endpoint does not support HTTP verb, add OPTIONS support #24437 (issues: #0, #15335, #17916)
  • Remove ldjson support and document ndjson for bulk/msearch #23049 (issue: #23025)
  • Enable strict duplicate checks for all XContent types #22225 (issues: #19614, #22073)
  • Enable strict duplicate checks for JSON content #22073 (issue: #19614)
  • Remove lenient stats parsing #21417 (issues: #20722, #21410)
  • Remove allow unquoted JSON #20388 (issues: #17674, #17801)
  • Remove FORCE version_type #20377 (issue: #19769)
Scripting
  • remove lang url parameter from stored script requests #25779 (issue: #22887)
  • Disallow lang to be used with Stored Scripts #25610
  • Remove Deprecated Script Settings #24756 (issue: #24532)
  • Scripting: Remove native scripts #24726 (issue: #19966)
  • Scripting: Remove file scripts #24627 (issue: #21798)
  • Make dates be ReadableDateTimes in scripts #22948 (issue: #22875)
  • Remove groovy scripting language #21607
  • Remove script access to term statistics #19462 (issue: #19359)
Search
  • Make index in TermsLookup mandatory #25753 (issue: #25750)
  • Removes FieldStats API #25628 (issue: #25577)
  • Remove deprecated fielddata_fields from search request #25566 (issue: #25537)
  • Removes deprecated fielddata_fields #25537 (issue: #19027)
  • ProfileResult and CollectorResult should print machine readable timing information #22561
  • Remove indices query #21837 (issue: #17710)
  • Remove ignored type parameter in search_shards api #21688
Security

  • Added new security limitations:

    • When a user’s role enables document level security for an index and suggesters are specified, the specified suggesters are ignored. For more information about suggesters, see Suggesters.
    • When document level security is enabled, search requests cannot be profiled. For more information about profiling, see the Profile API.
Sequence IDs
  • Change certain replica failures not to fail the replica shard #22874 (issue: #10708)
Settings
  • Settings: Remove shared setting property #24728
  • Settings: Remove support for yaml and json config files #24664 (issue: #19391)
Shadow Replicas
Similarities
  • Similarity should accept dynamic settings when possible #20339 (issue: #6727)

Breaking Java Changes

edit
Aggregations
  • Remove the unused SignificantTerms.compareTerm() method #24714
  • Make SignificantTerms.Bucket an interface rather than an abstract class #24670 (issue: #24492)
  • Fix NPE when values is omitted on percentile_ranks agg #26046
  • Make Terms.Bucket an interface rather than an abstract class #24492
  • Compound order for histogram aggregations #22343 (issues: #14771, #20003, #23613)
Internal
  • Collapses package structure for some bucket aggs #25579 (issue: #22868)
Java API
  • Remove deprecated IdsQueryBuilder ctor #25529
  • Removing unneeded getTookInMillis method #23923
  • Java api: ActionRequestBuilder#execute to return a PlainActionFuture #24415 (issues: #24412, #9201)
Java High Level REST Client
  • Unify the result interfaces from get and search in Java client #25361 (issue: #16440)
  • Allow RestHighLevelClient to use plugins #25024
Java REST Client
Network
  • Simplify TransportAddress #20798
Plugin Delete By Query
  • Move DeleteByQuery and Reindex requests into core #24578
Plugins
  • Drop name from TokenizerFactory #24869
Query DSL
  • Remove QueryParseContext #25486
  • Remove QueryParseContext from parsing QueryBuilders #25448
REST
  • Return index name and empty map for /{index}/_alias with no aliases #25114 (issues: #24723, #25090)

Deprecations

edit
Index APIs
  • Deprecated use of + in index expressions #24585 (issue: #24515)
Index Templates
  • Restore deprecation warning for invalid match_mapping_type values #22304
Indexed Scripts/Templates
  • Scripting: Deprecate stored search template apis #25437 (issue: #24596)
Internal
  • Deprecate XContentType auto detection methods in XContentFactory #22181 (issue: #19388)
Percolator
  • Deprecate percolate query’s document_type parameter. #25199
Plugins
  • Plugins: Add backcompat for sha1 checksums #26748 (issue: #26746)
Scripting
  • Scripting: Change keys for inline/stored scripts to source/id #25127
  • Scripting: Deprecate native scripts #24692 (issue: #19966)
  • Scripting: Deprecate index lookup #24691 (issue: #19359)
  • Deprecate Fine Grain Settings for Scripts #24573 (issue: #24532)
  • Scripting: Deprecate file script settings #24555 (issue: #21798)
  • Scripting: Deprecate file scripts #24552 (issue: #21798)
Settings
  • Settings: Update settings deprecation from yml to yaml #24663 (issue: #19391)
  • Deprecate settings in .yml and .json #24059 (issue: #19391)
Tribe Node

New Features

edit
Aggregations
  • SignificantText aggregation - like significant_terms, but for text #24432 (issue: #23674)
Analysis
  • Expose simplepattern and simplepatternsplit tokenizers #25159 (issue: #23363)
  • Parse synonyms with the same analysis chain #8049 (issue: #7199)
Core
Internal
  • Automatically adjust search threadpool queue_size #23884 (issue: #3890)
Mapping
  • Add new ip_range field type #24433
Parent/Child
  • Move parent_id query to the parent-join module #25072 (issue: #20257)
  • Introduce ParentJoinFieldMapper, a field mapper that creates parent/child relation within documents of the same index #24978 (issue: #20257)
Plugin Analysis ICU
  • Add ICUCollationFieldMapper #24126
Search
  • Automatically early terminate search query based on index sorting #24864 (issue: #6720)
Sequence IDs
  • Add a scheduled translog retention check #25622 (issues: #10708, #25294)
  • Initialize sequence numbers on a shrunken index #25321 (issue: #10708)
  • Initialize primary term for shrunk indices #25307 (issue: #10708)
  • Introduce translog size and age based retention policies #25147 (issue: #10708)
Stats
  • Adds nodes usage API to monitor usages of actions #24169
Task Manager
  • Task Management [ISSUE] #15117
Upgrade API
  • TemplateUpgraders should be called during rolling restart #25263 (issues: #24379, #24680)

Enhancements

edit
Aggregations
  • Add strict parsing of aggregation ranges #25769
  • Adds rewrite phase to aggregations #25495 (issue: #17676)
  • Tweak AggregatorBase.addRequestCircuitBreakerBytes #25162 (issue: #24511)
  • Add superset size to Significant Term REST response #24865
  • Add document count to Matrix Stats aggregation response #24776
  • Adds an implementation of LogLogBeta for the cardinality aggregation #22323 (issue: #22230)
  • Support distance units in GeoHashGrid aggregation precision #26291 (issue: #5042)
  • Reject multiple methods in percentiles aggregation #26163 (issue: #26095)
  • Use global_ordinals_hash execution mode when sorting by sub aggregations. #26014 (issue: #24359)
  • Add a specialized deferring collector for terms aggregator #25190
  • Agg builder accessibility fixes #24323
  • Remove support for the include/pattern syntax. #23141 (issue: #22933)
  • Promote longs to doubles when a terms agg mixes decimal and non-decimal numbers #22449 (issue: #22232)
Allocation
  • Adjust status on bad allocation explain requests #25503 (issue: #25458)
  • Promote replica on the highest version node #25277 (issue: #10708)
Analysis
  • [Analysis] Support normalizer in request param #24767 (issue: #23347)
  • Enforce validation for PathHierarchy tokenizer #23510
  • [analysis-icu] Allow setting unicodeSetFilter #20814 (issue: #20820)
  • Match- and MultiMatchQueryBuilder should only allow setting analyzer on string values #23684 (issue: #21665)
Bulk
  • Simplify bulk request execution #20109
CAT API
  • expand /_cat/nodes to return information about hard drive #21775 (issue: #21679)
CRUD
  • Added validation for upsert request #24282 (issue: #16671)
Circuit Breakers
  • ScriptService: Replace max compilation per minute setting with max compilation rate #26399
Cluster
  • Validate a joining node’s version with version of existing cluster nodes #25808
  • Switch indices read-only if a node runs out of disk space #25541 (issue: #24299)
  • Add a cluster block that allows to delete indices that are read-only #24678
  • Separate publishing from applying cluster states #24236
  • Adds cluster state size to /_cluster/state response #23440 (issue: #3415)
Core
  • Allow InputStreamStreamInput array size validation where applicable #26692
  • Refactor bootstrap check results and error messages #26637
  • Add BootstrapContext to expose settings and recovered state to bootstrap checks #26628
  • Unit testable index creation task on MetaDataCreateIndexService #25961
  • Ignore .DS_Store files on macOS #27108 (issue: #23982)
  • Add max file size bootstrap check #25974
  • Add compatibility versions to main action response #25799
  • Index ids in binary form. #25352 (issues: #18154, #24615)
  • Explicitly reject duplicate data paths #25178
  • Use SPI in High Level Rest Client to load XContent parsers #25097
  • Upgrade to lucene-7.0.0-snapshot-a0aef2f #24775
  • Speed up PK lookups at index time. #19856
  • Use Java 9 FilePermission model #26302 (issue: #21534)
  • Add friendlier message on bad keystore permissions #26284
  • Epoch millis and second formats accept float implicitly #26119 (issue: #14641)
  • Remove connect SocketPermissions from core #22797
  • Add repository-url module and move URLRepository #22752 (issue: #22116)
  • Remove accept SocketPermissions from core #22622 (issue: #22116)
  • Move IfConfig.logIfNecessary call into bootstrap #22455 (issue: #22116)
  • Remove artificial default processors limit #20874 (issue: #20828)
  • Simplify write failure handling #19105 (issue: #20109)
  • Improve bootstrap checks error messages #24548
Discovery
  • Allow plugins to validate cluster-state on join #26595
Engine
  • Add refresh stats tracking for realtime get #25052 (issue: #24806)
  • Introducing a translog deletion policy #24950
  • Fill missing sequence IDs up to max sequence ID when recovering from store #24238 (issue: #10708)
  • Use sequence numbers to identify out of order delivery in replicas & recovery #24060 (issue: #10708)
  • Add replica ops with version conflict to translog #22626
  • Clarify global checkpoint recovery #21934 (issue: #21254)
  • Move the IndexDeletionPolicy to be engine internal #24930 (issue: #10708)
Exceptions
  • IllegalStateException: Only duplicated jar instead of classpath #24953
Highlighting
  • Picks offset source for the unified highlighter directly from the es mapping #25747 (issue: #25699)
Index APIs
  • Let primary own its replication group #25692 (issue: #25485)
  • Create index request should return the index name #25139 (issue: #23044)
Index Templates
  • Fix error message for a put index template request without index_patterns #27102 (issue: #27100)
Ingest
  • Add Ingest-Processor specific Rest Endpoints & Add Grok endpoint #25059 (issue: #24725)
  • Port support for commercial GeoIP2 databases from Logstash. #24889
  • add exclude_keys option to KeyValueProcessor #24876 (issue: #23856)
  • Allow removing multiple fields in ingest processor #24750 (issue: #24622)
  • Add target_field parameter to ingest processors #24133 (issues: #23228, #23682)
Inner Hits
Internal
  • TemplateUpgradeService should only run on the master #27294
  • Cleanup IndexFieldData visibility #25900
  • Bump the min compat version to 5.6.0 #25805
  • "shard started" should show index and shard ID #25157
  • Break out clear scroll logic from TransportClearScrollAction #25125 (issue: #25094)
  • Add helper methods to TransportActionProxy to identify proxy actions and requests #25124
  • Add remote cluster infrastructure to fetch discovery nodes. #25123 (issue: #25094)
  • Add the ability to set eager_global_ordinals in the new parent-join field #25019
  • Disallow multiple parent-join fields per mapping #25002
  • Remove the need for _UNRELEASED suffix in versions #24798 (issue: #24768)
  • Optimize the order of bytes in uuids for better compression. #24615 (issue: #18209)
  • Prevent cluster internal ClusterState.Custom impls to leak to a client #26232
  • Use holder pattern for lazy deprecation loggers #26218 (issue: #26210)
  • Allow ClusterState.Custom to be created on initial cluster states #26144
  • Try to convince the JVM not to lose stacktraces #24426 (issue: #24376)
  • Make document write requests immutable #23038
  • Add assertions enabled helper #24834
Java API
  • Always Accumulate Transport Exceptions #25017 (issue: #23099)
Java High Level REST Client
Java REST Client
  • Shade external dependencies in the rest client jar #25780 (issue: #25208)
  • RestClient uses system properties and system default SSLContext #25757 (issue: #23231)
  • Wrap rest httpclient with doPrivileged blocks #22603 (issue: #22116)
Logging
  • Prevent excessive disk consumption by log files #25660
  • Use LRU set to reduce repeat deprecation messages #25474 (issue: #25457)
Mapping
  • More efficient encoding of range fields. #26470 (issue: #26443)
  • Don’t detect source’s XContentType in DocumentParser.parseDocument() #26880
  • Better validation of copy_to. #25983
  • Optimize terms queries on ip addresses to use a PointInSetQuery whenever possible. #25669 (issue: #25667)
  • Loosen the restrictions on disabling _all in 6.x #26259
  • Date detection should not rely on a hardcoded set of characters. #22171 (issue: #1694)
  • Identify documents by their _id. #24460
Network
  • Add additional low-level logging handler #26887
  • Unwrap causes when maybe dying #26884
  • Move TransportStats accounting into TcpTransport #25251
  • Simplify connection closing and cleanups in TcpTransport #25250
  • Disable the Netty recycler in the client #24793 (issues: #22452, #24721)
  • Remove Netty logging hack #24653 (issues: #24469, #5624, #6568, #6696)
  • Isolate SocketPermissions to Netty #23057
  • Wrap netty accept/connect ops with doPrivileged #22572 (issue: #22116)
  • Replace Socket, ServerSocket, and HttpServer usages in tests with mocksocket versions #22287 (issue: #22116)
Packaging
  • Remove memlock suggestion from systemd service #25979
  • Set address space limit in systemd service file #25975
  • Version option should display if snapshot #25970
  • Ignore JVM options before checking Java version #25969
  • Also skip JAVA_TOOL_OPTIONS on Windows #25968
  • Introduce elasticsearch-env for Windows #25958
  • Introduce elasticsearch-env #25815 (issue: #20286)
  • Stop exporting HOSTNAME from scripts #25807
  • Set number of processes in systemd unit file #24970 (issue: #20874)
Parent/Child
  • Remove ParentJoinFieldSubFetchPhase #25550 (issue: #25363)
  • Support parent id being specified as number in the _source #25547
Percolator
  • Store the QueryBuilder’s Writable representation instead of its XContent representation #25456
  • Add support for selecting percolator query candidate matches containing wildcard / prefix queries #25351
Plugin Discovery EC2
  • Read ec2 discovery address from aws instance tags #22743 (issue: #22566)
Plugin Lang Painless
  • Allow Custom Whitelists in Painless #25557
  • Update Painless to Allow Augmentation from Any Class #25360
  • Add Needs Methods to Painless Script Context Factories #25267
  • Support Script Context Stateful Factory in Painless #25233
  • Generate Painless Factory for Creating Script Instances #25120
  • Update Painless to Use New Script Contexts #25015
  • Optimize instance creation in LambdaBootstrap #24618
  • Make Painless Compiler Use an Instance Per Context #24972
  • Make PainlessScript An Interface #24966
Plugin Repository GCS
  • GCS Repository: Add secure storage of credentials #24697
Plugin Repository HDFS
  • Add permission checks before reading from HDFS stream #26716 (issue: #26714)
  • Add doPrivilege blocks for socket connect ops in repository-hdfs #22793 (issue: #22116)
  • Add Kerberos support for Repo HDFS plugin [ISSUE] #21990
Plugin Repository S3
  • S3 Repository: Add back repository level credentials #24609
Plugins
  • Adjust SHA-512 supported format on plugin install #27093
  • Move tribe to a module #25778
  • Plugins can register pre-configured char filters #25000 (issue: #23658)
  • Add purge option to remove plugin CLI #24981
  • Allow plugins to register pre-configured tokenizers #24751 (issues: #24223, #24572)
  • Move ReindexAction class to core #24684 (issue: #24578)
  • Make PreConfiguredTokenFilter harder to misuse #24572 (issue: #23658)
  • Plugins: Remove leniency for missing plugins dir #24173
  • Add doPrivilege blocks for socket connect operations in plugins #22534 (issue: #22116)
Query DSL
  • Make slop optional when parsing span_near query #25677 (issue: #25642)
  • Require a field when a seed is provided to the random_score function. #25594 (issue: #25240)
  • Add support for auto_generate_synonyms_phrase_query in match_query, multi_match_query, query_string and simple_query_string #23147
REST
  • Cat shards bytes #26952
  • Refactor PathTrie and RestController to use a single trie for all methods #25459 (issue: #24437)
  • Make ObjectParser support string to boolean conversion #24668 (issue: #21802)
Recovery
  • Introduce a History UUID as a requirement for ops based recovery #26577 (issue: #10708)
  • Goodbye, Translog Views #25962
  • Disallow multiple concurrent recovery attempts for same target shard #25428
  • Live primary-replica resync (no rollback) #24841 (issue: #10708)
  • Peer Recovery: remove maxUnsafeAutoIdTimestamp hand off #24243 (issue: #24149)
  • Introduce sequence-number-based recovery #22484 (issue: #10708)
Scripting
  • Scripting: Rename SearchScript.needsScores to needs_score #25235
  • Scripting: Add optional context parameter to put stored script requests #25014
  • Add New Security Script Settings #24637 (issue: #24532)
  • Add StatefulFactoryType as optional intermediate factory in script contexts #24974 (issue: #20426)
  • Make contexts available to ScriptEngine construction #24896
  • Make ScriptEngine.compile generic on the script context #24873
  • Add instance and compiled classes to script contexts #24868
Search
  • Add soft limit on allowed number of script fields in request #26598 (issue: #26390)
  • Add a soft limit for the number of requested doc-value fields #26574 (issue: #26390)
  • Rewrite search requests on the coordinating nodes #25814 (issue: #25791)
  • Ensure query resources are fetched asynchronously during rewrite #25791
  • Introduce a new Rewriteable interface to streamline rewriting #25788
  • Reduce the scope of QueryRewriteContext #25787
  • Reduce the overhead of timeouts and low-level search cancellation. #25776
  • Reduce profiling overhead. #25772 (issue: #24799)
  • Prevent can_match requests from sending to incompatible nodes #25705 (issue: #25704)
  • Add a shard filter search phase to pre-filter shards based on query rewriting #25658
  • Ensure we rewrite common queries to match_none if possible #25650
  • Limit the number of concurrent shard requests per search request #25632
  • Add cluster name validation to RemoteClusterConnection #25568
  • Speed up sorted scroll when the index sort matches the search sort #25138 (issue: #6720)
  • Leverage scorerSupplier when applicable. #25109
  • Add Cross Cluster Search support for scroll searches #25094
  • Track EWMA[1] of task execution time in search threadpool executor #24989 (issue: #24915)
  • Query range fields by doc values when they are expected to be more efficient than points #24823 (issue: #24314)
  • Search: Fairer balancing when routing searches by session ID #24671 (issue: #24642)
  • Add parsing from xContent to Suggest #22903
  • Add parsing from xContent to ShardSearchFailure #22699
  • Eliminate array access in tight loops when profiling is enabled. #24959
  • Support Multiple Inner Hits on a Field Collapse Request #24517
  • Expand cross cluster search indices for search requests to the concrete index or to it’s aliases #24502
Search Templates
Sequence IDs
  • Roll translog generation on primary promotion #27313
  • Restoring from snapshot should force generation of a new history uuid #26694 (issues: #10708, #26544, #26557, #26577)
  • Add global checkpoint tracking on the primary #26666 (issue: #26591)
  • Introduce global checkpoint background sync #26591 (issues: #26573, #26630, #26666)
  • Move UNASSIGNED_SEQ_NO and NO_OPS_PERFORMED to SequenceNumbers` #26494 (issue: #10708)
  • Move primary term from ReplicationRequest to ConcreteShardRequest #25822
  • Add reason to global checkpoint updates on replica #25612 (issue: #10708)
  • Introduce primary/replica mode for GlobalCheckPointTracker #25468
  • Throw back replica local checkpoint on new primary #25452 (issues: #10708, #25355)
  • Update global checkpoint when increasing primary term on replica #25422 (issues: #10708, #25355)
  • Enable a long translog retention policy by default #25294 (issues: #10708, #25147)
  • Introduce primary context #25122 (issues: #10708, #25355)
  • Block older operations on primary term transition #24779 (issue: #10708)
  • Block global checkpoint advances when recovering #24404 (issue: #10708)
  • Add primary term to doc write response #24171 (issue: #10708)
  • Preserve multiple translog generations #24015 (issue: #10708)
  • Introduce translog generation rolling #23606 (issue: #10708)
  • Replicate write failures #23314
  • Introduce sequence-number-aware translog #22822 (issue: #10708)
  • Introduce translog no-op #22291 (issue: #10708)
  • Tighten sequence numbers recovery #22212 (issue: #10708)
  • Add BWC layer to seq no infra and enable BWC tests #22185 (issue: #21670)
  • Add internal _primary_term doc values field, fix _seq_no indexing #21637 (issues: #10708, #21480)
  • Add global checkpoint to translog checkpoints #21254
  • Sequence numbers commit data for Lucene uses Iterable interface #20793 (issue: #10708)
  • Simplify GlobalCheckpointService and properly hook it for cluster state updates #20720
  • Fill gaps on primary promotion #24945 (issue: #10708)
  • Introduce clean transition on primary promotion #24925 (issue: #10708)
  • Guarantee that translog generations are seqNo conflict free #24825 (issues: #10708, #24779)
  • Inline global checkpoints #24513 (issue: #10708)
Settings
  • Add disk threshold settings validation #25600 (issue: #25560)
  • Enable cross-setting validation #25560 (issue: #25541)
  • Validate transport.profiles.* settings #25508
  • Cleanup network / transport related settings #25489
  • Emit settings deprecation logging at most once #25457
  • IndexMetaData: Introduce internal format index setting #25292
  • Persist created keystore on startup unless keystore is present #26253 (issue: #26126)
  • Settings: Add keystore.seed auto generated secure setting #26149
  • Settings: Add keystore creation to add commands #26126
Snapshot/Restore
  • Fixed references to Multi Index Syntax #27283
  • Improves snapshot logging and snapshot deletion error handling #25264
  • Enhances get snapshots API to allow retrieving repository index only #24477 (issue: #24288)
Stats
  • Update IndexShard#refreshMetric via a ReferenceManager.RefreshListener #25083 (issues: #24806, #25052)
  • Expose disk usage estimates in nodes stats #22081 (issue: #8686)
Store
  • Remote support for lucene versions without checksums #24021
Suggesters
  • Remove deprecated _suggest endpoint #22203 (issue: #20305)
Task Manager
Translog
  • Translog file recovery should not rely on lucene commits #25005 (issue: #24950)

Bug Fixes

edit
Aggregations
  • Do not delegate a null scorer to LeafBucketCollectors #26747 (issue: #26611)
  • Create weights lazily in filter and filters aggregation #26983
  • Fix IndexOutOfBoundsException in histograms for NaN doubles (#26787) #26856 (issue: #26787)
  • Scripted_metric _agg parameter disappears if params are provided #19863 (issue: #19768)
  • Fixes array out of bounds for value count agg #26038 (issue: #17379)
  • Aggregations bug: Significant_text fails on arrays of text. #25030 (issue: #25029)
  • Check bucket metric ages point to a multi bucket agg #26215 (issue: #25775)
  • Terms aggregation should remap global ordinal buckets when a sub-aggregator is used to sort the terms #24941 (issue: #24788)
  • Correctly set doc_count when MovAvg "predicts" values on existing buckets #24892 (issue: #24327)
  • DateHistogram: Fix extended_bounds with offset #23789 (issue: #23776)
  • Fix ArrayIndexOutOfBoundsException when no ranges are specified in the query #23241 (issue: #22881)
Aliases
  • mget with an alias shouldn’t ignore alias routing #25697 (issue: #25696)
  • GET aliases should 404 if aliases are missing #25043 (issue: #24644)
Allocation
Analysis
  • Pre-configured shingle filter should disable graph analysis #25853 (issue: #25555)
  • PatternAnalyzer should lowercase wildcard queries when lowercase is true. #24967
CAT API
  • Fix NPE for /_cat/indices when no primary shard #26953 (issue: #26942)
CRUD
  • Serialize and expose timeout of acknowledged requests in REST layer #26189 (issue: #26213)
  • Fix silent loss of last command to _bulk and _msearch due to missing newline #25740 (issue: #7601)
Cache
  • Reduce the default number of cached queries. #26949 (issue: #26938)
  • fix bug of weight computation #24856
Circuit Breakers
  • Checks the circuit breaker before allocating bytes for a new big array #25010 (issue: #24790)
Cluster
  • Register setting cluster.indices.tombstones.size #26193 (issue: #26191)
Core
  • Correctly encode warning headers #27269 (issue: #27244)
  • Fix cache compute if absent for expired entries #26516
  • Timed runnable should delegate to abstract runnable #27095 (issue: #27069)
  • Stop invoking non-existent syscall #27016 (issue: #20179)
  • MetaData Builder doesn’t properly prevent an alias with the same name as an index #26804
  • Release operation permit on thread-pool rejection #25930 (issue: #25863)
  • Node should start up despite of a lingering .es_temp_file #21210 (issue: #21007)
  • Fix cache expire after access #24546
Dates
Discovery
  • MasterNodeChangePredicate should use the node instance to detect master change #25877 (issue: #25471)
Engine
  • Die with dignity while merging #27265 (issue: #19272)
  • Engine - do not index operations with seq# lower than the local checkpoint into lucene #25827 (issues: #1, #2, #25592)
Geo
  • Fix typo in GeoUtils#isValidLongitude #25121
Highlighting
  • Fix percolator highlight sub fetch phase to not highlight query twice #26622
  • FastVectorHighlighter should not cache the field query globally #25197 (issue: #25171)
  • Higlighters: Fix MultiPhrasePrefixQuery rewriting #25103 (issue: #25088)
  • Fix nested query highlighting #26305 (issue: #26230)
Index APIs
  • Shrink API should ignore templates #25380 (issue: #25035)
  • Rollover max docs should only count primaries #24977 (issue: #24217)
  • Validates updated settings on closed indices #24487 (issue: #23787)
Ingest
  • date processor should not fail if timestamp is specified as json number #26986 (issue: #26967)
  • date_index_name processor should not fail if timestamp is specified as json number #26910 (issue: #26890)
  • Sort Processor does not have proper behavior with targetField #25237 (issue: #24133)
  • fix grok’s pattern parsing to validate pattern names in expression #25063 (issue: #22831)
  • Remove support for Visio and potm files #22079 (issue: #22077)
  • Fix floating-point error when DateProcessor parses UNIX #24947
  • add option for _ingest.timestamp to use new ZonedDateTime (5.x backport) #24030 (issues: #23168, #23174)
Inner Hits
  • Do not allow inner hits that fetch _source and have a non nested object field as parent #25749 (issue: #25315)
  • When fetching nested inner hits only access stored fields when needed #25864 (issue: #6)
  • If size / offset are out of bounds just do a plain count #20556 (issue: #20501)
  • Fix Source filtering in new field collapsing feature #24068 (issue: #24063)
Internal
  • Bump version to 6.0.1 [OPEN] #27386
  • IndexShard.routingEntry should only be updated once all internal state is ready #26776
  • Catch exceptions and inform handler in RemoteClusterConnection#collectNodes #26725 (issue: #26700)
  • Internal: Add versionless alias for rest client codebase in policy files #26521
  • Upgrade Lucene to version 7.0.1 #26926
  • Fix BytesReferenceStreamInput#skip with offset #25634
  • Fix race condition in RemoteClusterConnection node supplier #25432
  • Initialise empty lists in BaseTaskResponse constructor #25290
  • Extract a common base class for scroll executions #24979 (issue: #16555)
  • Obey lock order if working with store to get metadata snapshots #24787 (issue: #24481)
  • Fix Version based BWC and set correct minCompatVersion #24732
  • Fix _field_caps serialization in order to support cross cluster search #24722
  • Avoid race when shutting down controller processes #24579
  • Fix handling of document failure exception in InternalEngine #22718
  • Ensure remote cluster is connected before fetching _field_caps #24845 (issue: #24763)
Java API
  • BulkProcessor flush runnable preserves the thread context from creation time #26718 (issue: #26596)
Java High Level REST Client
  • Make RestHighLevelClient’s Request class public #26627 (issue: #26455)
  • Forbid direct usage of ContentType.create() methods #26457 (issues: #22769, #26438)
  • Make ShardSearchTarget optional when parsing ShardSearchFailure #27078 (issue: #27055)
Java REST Client
  • Better message text for ResponseException #26564
  • rest-client-sniffer: configurable threadfactory #26897
Logging
Machine Learning
  • Fixed a race condition when simultaneous close requests are made for the same job.
Mapping
  • Allow copying from a field to another field that belongs to the same nested object. #26774 (issue: #26763)
  • Fixed bug that mapper_parsing_exception is thrown for numeric field with ignore_malformed=true when inserting "NaN" #25967 (issue: #25289)
  • Coerce decimal strings for whole number types by truncating the decimal part #25835 (issue: #25819)
  • Fix parsing of ip range queries. #25768 (issue: #25636)
  • Disable date field mapping changing #25285 (issue: #25271)
  • Correctly enable _all for older 5.x indices #25087 (issue: #25068)
  • token_count datatype should handle null value #25046 (issue: #24928)
  • keep _parent field while updating child type mapping #24407 (issue: #23381)
  • ICUCollationKeywordFieldMapper use SortedSetDocValuesField #26267
  • Fix serialization of the _all field. #26143 (issue: #26136)
More Like This
  • Pass over _routing value with more_like_this items to be retrieved #24679 (issue: #23699)
NOT CLASSIFIED
  • DocumentMissingException during Logstash scripted upsert [ISSUE] #27148
  • An assertion trips when master opens an index from before 5.x [ISSUE] #24809
Nested Docs
  • In case of a single type the _id field should be added to the nested document instead of _uid field #25149
  • Inner hits source filtering not working [ISSUE] #23090
Network
  • Fixed ByteBuf leaking in org.elasticsearch.http.netty4.Netty4HttpRequestHandler #27222 (issues: #3, #4, #5, #6)
  • Check for closed connection while opening #26932
  • Ensure pending transport handlers are invoked for all channel failures #25150
  • Notify onConnectionClosed rather than onNodeDisconnect to prune transport handlers #24639 (issues: #24557, #24575, #24632)
  • Release pipelined http responses on close #26226
  • Fix error message if an incompatible node connects #24884
Packaging
  • Fix handling of Windows paths containing parentheses #26916 (issue: #26454)
  • Exit Windows scripts promptly on failure #25959
  • Pass config path as a system property #25943
  • ES_HOME needs to be made absolute before attempt at traversal #25865
  • Fix elasticsearch-keystore handling of path.conf #25811
  • Stop disabling explicit GC #25759
  • Avoid failing install if system-sysctl is masked #25657 (issue: #24234)
  • Get short path name for native controllers #25344
  • When stopping via systemd only kill the JVM, not its control group #25195
  • remove remaining references to scripts directory #24771
  • Handle parentheses in batch file path #24731 (issue: #24712)
  • Detect modified keystore on package removal #26300
  • Create keystore on RPM and Debian package install #26282
  • Add safer empty variable checking for Windows #26268 (issue: #26261)
  • Export HOSTNAME environment variable #26262 (issues: #25807, #26255)
  • Fix daemonization command status test #26196 (issue: #26080)
  • Set RuntimeDirectory in systemd service #23526
Parent/Child
  • The default _parent field should not try to load global ordinals #25851 (issue: #25849)
Percolator
  • Also support query extraction for queries wrapped inside a ESToParentBlockJoinQuery #26754
  • Fix range queries with date range based on current time in percolator queries. #24666 (issue: #23921)
Plugin Analysis Kuromoji
Plugin Analysis Phonetic
  • Fix beidermorse phonetic token filter for unspecified languageset #27112 (issue: #26771)
Plugin Discovery File
  • Fix discovery-file plugin to use custom config path #26662 (issue: #26660)
Plugin Ingest Attachment
Plugin Lang Painless
  • Painless: allow doubles to be casted to longs. #25936
Plugin Repository Azure
  • Azure snapshots can not be restored anymore #26778 (issues: #22858, #26751, #26777)
  • Snapshot : azure module - accelerate the listing of files (used in delete snapshot) #25710 (issue: #25424)
  • Use Azure upload method instead of our own implementation #26751
  • Make calls to CloudBlobContainer#exists privileged #25937 (issue: #25931)
Plugin Repository GCS
  • Ensure that gcs client creation is privileged #25938 (issue: #25932)
Plugin Repository HDFS
  • Add Log4j to SLF4J binding for repository-hdfs #26514 (issue: #26512)
  • Upgrading HDFS Repository Plugin to use HDFS 2.8.1 Client #25497 (issue: #25450)
Plugin Repository S3
  • Avoid SecurityException in repository-S3 on DefaultS3OutputStream.flush() #25254 (issue: #25192)
  • Wrap getCredentials() in a doPrivileged() block #23297 (issues: #22534, #23271)
Plugins
  • X-Pack plugin download fails on Windows desktop [ISSUE] #24570
  • Fix plugin installation permissions #24527 (issue: #24480)
Query DSL
  • Fixed incomplete JSON body on count request making org.elasticsearch.rest.action.RestActions#parseTopLevelQueryBuilder go into endless loop #26680 (issue: #26083)
  • SpanNearQueryBuilder should return the inner clause when a single clause is provided #25856 (issue: #25630)
  • Refactor field expansion for match, multi_match and query_string query #25726 (issues: #25551, #25556)
  • WrapperQueryBuilder should also rewrite the parsed query #25480
REST
  • Rest test fixes #27354
  • Fix inconsistencies in the rest api specs for cat.snapshots #26996 (issues: #25737, #26923)
  • Fix inconsistencies in the rest api specs for *_script #26971 (issue: #26923)
  • exists template needs a template name #25988
  • Fix handling of invalid error trace parameter #25785 (issue: #25774)
  • Fix handling of exceptions thrown on HEAD requests #25172 (issue: #21125)
  • Fixed NPEs caused by requests without content. #23497 (issue: #24701)
  • Fix get mappings HEAD requests #23192 (issue: #21125)
Recovery
  • Close translog view after primary-replica resync #25862 (issue: #24841)
Reindex API
  • Fix update_by_query’s default size parameter #26784 (issue: #26761)
  • Reindex: don’t duplicate _source parameter #24629 (issue: #24628)
  • Add qa module that tests reindex-from-remote against pre-5.0 versions of Elasticsearch #24561 (issues: #23828, #24520)
Scroll
  • Fix single shard scroll within a cluster with nodes in version >= 5.3 and <= 5.3 #24512
Search
  • Fail query when a sort is provided in conjunction with rescorers #26510
  • Let search phases override max concurrent requests #26484 (issue: #26198)
  • Avoid stack overflow on search phases #27069 (issue: #27042)
  • Fix search_after with geo distance sorting #26891
  • Fix serialization errors when cross cluster search goes to a single shard #26881 (issue: #26833)
  • Early termination with index sorting should not set terminated_early in the response #26597 (issue: #26408)
  • Format doc values fields. #22146
  • Fix term(s) query for range field #25918
  • Caching a MinDocQuery can lead to wrong results. #25909
  • Fix random score generation when no seed is provided. #25908
  • Merge FunctionScoreQuery and FiltersFunctionScoreQuery #25889 (issues: #15709, #23628)
  • Respect cluster alias in _index aggs and queries #25885 (issue: #25606)
  • First increment shard stats before notifying and potentially sending response #25818
  • Remove assertion about deviation when casting to a float. #25806 (issue: #25330)
  • Prevent skipping shards if a suggest builder is present #25739 (issue: #25658)
  • Ensure remote cluster alias is preserved in inner hits aggs #25627 (issue: #25606)
  • Do not search locally if remote index pattern resolves to no indices #25436 (issue: #25426)
  • Adds check for negative search request size #25397 (issue: #22530)
  • Make sure range queries are correctly profiled. #25108
  • Fix RangeFieldMapper rangeQuery to properly handle relations #24808 (issue: #24744)
  • Fix ExpandSearchPhase when response contains no hits #24688 (issue: #24672)
  • Refactor simple_query_string to handle text part like multi_match and query_string #26145 (issue: #25726)
  • Fix _exists_ in query_string on empty indices. #25993 (issue: #25956)
  • Fix script field sort returning Double.MAX_VALUE for all documents #24942 (issue: #24940)
  • Compute the took time of the query after the expand phase of field collapsing #24902 (issue: #24900)
Security
  • Prevented 6.0 nodes from joining clusters with un-upgraded version 5 .security indices. For upgrade instructions, see Upgrading the Elastic Stack.
  • Enabled read-only access to the index audit log by the _xpack internal user. For more information, see Internal users.
  • Updated the concrete security index such that it is now always named .security-6. In 6.0 beta and RC releases, it was sometimes named .security-v6.
  • Fixed handling of exceptions when retrieving roles from a native roles store. For more information about configuring a native realm, see Native user authentication.
Sequence IDs
  • Fire global checkpoint sync under system context #26984
  • Fix pre-6.0 response to unknown replication actions #25744 (issue: #10708)
  • Track local checkpoint on primary immediately #25434 (issues: #10708, #25355, #25415)
  • Initialize max unsafe auto ID timestamp on shrink #25356 (issues: #10708, #25355)
  • Use correct primary term for replicating NOOPs #25128
  • Handle already closed while filling gaps #25021 (issue: #24925)
  • TranslogWriter.assertNoSeqNumberConflict failure [ISSUE] #26710
  • Avoid losing ops in file-based recovery #22945 (issue: #22484)
  • Handle primary failure handling replica response #24926 (issue: #24935)
Settings
  • Emit settings deprecation logging on empty update #27017 (issue: #26419)
  • Fix filtering for ListSetting #26914
  • Fix settings serialization to not serialize secure settings or not take the total size into account #25323
  • Keystore CLI should use the AddFileKeyStoreCommand for files #25298
  • Allow resetting settings that use an IP validator #24713 (issue: #24709)
  • Updating an unrecognized setting should error out with that reason [ISSUE] #25607
  • Settings: Fix setting groups to include secure settings #25076 (issue: #25069)
Similarities
  • Add boolean similarity to built in similarity types #26613
Snapshot/Restore
  • Snapshot/Restore: better handle incorrect chunk_size settings in FS repo #26844 (issue: #26843)
  • Snapshot/Restore: Ensure that shard failure reasons are correctly stored in CS #25941 (issue: #25878)
  • Output all empty snapshot info fields if in verbose mode #25455 (issue: #24477)
  • Remove redundant and broken MD5 checksum from repository-s3 #25270 (issue: #25269)
  • Consolidates the logic for cleaning up snapshots on master election #24894 (issue: #24605)
  • Removes completed snapshot from cluster state on master change #24605 (issue: #24452)
  • Keep snapshot restore state and routing table in sync #20836 (issue: #19774)
  • Master failover during snapshotting could leave the snapshot incomplete [OPEN] [ISSUE] #25281
  • Fix inefficient (worst case exponential) loading of snapshot repository #24510 (issue: #24509)
Stats
  • Fix RestGetAction name typo #27266
  • Keep cumulative elapsed scroll time in microseconds #27068 (issue: #27046)
  • _nodes/stats should not fail due to concurrent AlreadyClosedException #25016 (issue: #23099)
  • Avoid double decrement on current query counter #24922 (issues: #22996, #24872)
  • Adjust available and free bytes to be non-negative on huge FSes #24911 (issues: #23093, #24453)
Suggesters
  • Fix division by zero in phrase suggester that causes assertion to fail #27149
  • Context suggester should filter doc values field #25858 (issue: #25404)
  • Fix context suggester to read values from keyword type field #24200 (issue: #24129)
Templates
  • Tests: Fix FullClusterRestartIT.testSnapshotRestore test failing in 6.x #27218 (issue: #27213)
Translog
  • Fix Translog.Delete serialization for sequence numbers #22543
Upgrade API
  • Upgrade API: fix excessive logging and unnecessary template updates #26698 (issue: #26673)

Regressions

edit
Bulk
  • Only re-parse operation if a mapping update was needed #23832 (issue: #23665)
Highlighting
  • Fix Fast Vector Highlighter NPE on match phrase prefix #25116 (issue: #25088)
Search
  • Always use DisjunctionMaxQuery to build cross fields disjunction #25115 (issue: #23966)
Sequence IDs
  • Indexing performance degradation in 6.0.0-beta1 [ISSUE] #26339

Upgrades

edit
Core
Logging
Network
Plugin Ingest Attachment
Upgrade API
  • Improve stability and logging of TemplateUpgradeServiceIT tests #25386 (issue: #25382)
« Elasticsearch version 6.0.1 Elasticsearch version 6.0.0-rc2 »