IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Elasticsearch version 6.0.0-beta2 Elasticsearch version 6.0.0-alpha2 »
Elastic Docs ›Elasticsearch Guide [6.5] ›Release Notes

Elasticsearch version 6.0.0-beta1

edit

Elasticsearch version 6.0.0-beta1

edit

Breaking Changes

edit
Aggregations
  • Change parsing of numeric to and from parameters in date_range aggregation #25376 (issue: #17920)
Aliases
Highlighting
  • Remove the postings highlighter and make unified the default highlighter choice #25028
Index APIs
  • Remove (deprecated) support for + in index expressions #25274 (issue: #24515)
  • Delete index API to work only against concrete indices #25268 (issues: #2318, #23997)
Indexed Scripts/Templates
  • Scripting: Remove search template actions #25717
Ingest
  • update ingest-user-agent regexes.yml #25608
  • remove ingest.new_date_format #25583
Java REST Client
Packaging
  • Remove support for ES_INCLUDE #25804
  • Setup: Change default heap to 1G #25695
  • Use config directory to find jvm.options #25679 (issue: #23004)
  • Remove implicit 32-bit support #25435
  • Remove default path settings #25408 (issue: #25357)
  • Remove path.conf setting #25392 (issue: #25357)
  • Honor masking of systemd-sysctl.service #24234 (issues: #21899, #806)
Plugin Analysis ICU
  • Upgrade icu4j for the ICU analysis plugin to 59.1 #25243 (issue: #21425)
Plugin Discovery Azure Classic
  • Remove discovery.type BWC layer from the EC2/Azure/GCE plugins #25080 (issue: #24543)
Plugin Repository GCS
  • GCS Repository: Remove specifying credential file on disk #24727
Plugins
  • Make plugin loading stricter #25405
Query DSL
  • Refactor QueryStringQuery for 6.0 #25646 (issue: #25574)
  • Change split_on_whitespace default to false #25570 (issue: #25470)
  • Remove deprecated template query #24577 (issue: #19390)
REST
  • IndexClosedException to return 400 rather than 403 #25752
  • Remove comma-separated feature parsing for GetIndicesAction #24723 (issue: #24437)
  • Improve REST error handling when endpoint does not support HTTP verb, add OPTIONS support #24437 (issues: #0, #15335, #17916)
Scripting
  • remove lang url parameter from stored script requests #25779 (issue: #22887)
  • Disallow lang to be used with Stored Scripts #25610
  • Remove Deprecated Script Settings #24756 (issue: #24532)
  • Scripting: Remove native scripts #24726 (issue: #19966)
  • Scripting: Remove file scripts #24627 (issue: #21798)
Search
  • Make index in TermsLookup mandatory #25753 (issue: #25750)
  • Removes FieldStats API #25628 (issue: #25577)
  • Remove deprecated fielddata_fields from search request #25566 (issue: #25537)
  • Removes deprecated fielddata_fields #25537 (issue: #19027)
Security
  • A new bootstrap check enforces that TLS/SSL is required for inter-node communication when running in production mode. See Encrypting Communications.
  • A new bootstrap check enforces that HTTPS is used by the built-in token service when running in production mode. To disable the token service, set xpack.security.authc.token.enabled to false in your elasticsearch.yml. See Token service settings.
Settings
  • Settings: Remove shared setting property #24728
  • Settings: Remove support for yaml and json config files #24664 (issue: #19391)
Similarities
  • Similarity should accept dynamic settings when possible #20339 (issue: #6727)

Breaking Java Changes

edit
Aggregations
  • Remove the unused SignificantTerms.compareTerm() method #24714
  • Make SignificantTerms.Bucket an interface rather than an abstract class #24670 (issue: #24492)
Internal
  • Collapses package structure for some bucket aggs #25579 (issue: #22868)
Java API
  • Remove deprecated IdsQueryBuilder ctor #25529
  • Removing unneeded getTookInMillis method #23923
Java High Level REST Client
  • Unify the result interfaces from get and search in Java client #25361 (issue: #16440)
  • Allow RestHighLevelClient to use plugins #25024
Java REST Client
Plugin Delete By Query
  • Move DeleteByQuery and Reindex requests into core #24578
Query DSL
  • Remove QueryParseContext #25486
  • Remove QueryParseContext from parsing QueryBuilders #25448
REST
  • Return index name and empty map for /{index}/_alias with no aliases #25114 (issues: #24723, #25090)

Deprecations

edit
Index APIs
  • Deprecated use of + in index expressions #24585 (issue: #24515)
Indexed Scripts/Templates
  • Scripting: Deprecate stored search template apis #25437 (issue: #24596)
Percolator
  • Deprecate percolate query’s document_type parameter. #25199
Scripting
  • Scripting: Change keys for inline/stored scripts to source/id #25127
  • Scripting: Deprecate native scripts #24692 (issue: #19966)
  • Scripting: Deprecate index lookup #24691 (issue: #19359)
  • Deprecate Fine Grain Settings for Scripts #24573 (issue: #24532)
  • Scripting: Deprecate file script settings #24555 (issue: #21798)
  • Scripting: Deprecate file scripts #24552 (issue: #21798)
Settings
  • Settings: Update settings deprecation from yml to yaml #24663 (issue: #19391)
Tribe Node

New Features

edit
Analysis
  • Expose simplepattern and simplepatternsplit tokenizers #25159 (issue: #23363)
  • Parse synonyms with the same analysis chain #8049 (issue: #7199)
Parent/Child
  • Move parent_id query to the parent-join module #25072 (issue: #20257)
  • Introduce ParentJoinFieldMapper, a field mapper that creates parent/child relation within documents of the same index #24978 (issue: #20257)
Search
  • Automatically early terminate search query based on index sorting #24864 (issue: #6720)
Sequence IDs
  • Add a scheduled translog retention check #25622 (issues: #10708, #25294)
  • Initialize sequence numbers on a shrunken index #25321 (issue: #10708)
  • Initialize primary term for shrunk indices #25307 (issue: #10708)
  • Introduce translog size and age based retention policies #25147 (issue: #10708)
Stats
  • Adds nodes usage API to monitor usages of actions #24169
Task Manager
Upgrade API
  • TemplateUpgraders should be called during rolling restart #25263 (issues: #24379, #24680)

Enhancements

edit
Aggregations
  • Add strict parsing of aggregation ranges #25769
  • Adds rewrite phase to aggregations #25495 (issue: #17676)
  • Tweak AggregatorBase.addRequestCircuitBreakerBytes #25162 (issue: #24511)
  • Add superset size to Significant Term REST response #24865
  • Add document count to Matrix Stats aggregation response #24776
  • Adds an implementation of LogLogBeta for the cardinality aggregation #22323 (issue: #22230)
Allocation
  • Adjust status on bad allocation explain requests #25503 (issue: #25458)
  • Promote replica on the highest version node #25277 (issue: #10708)
Analysis
  • [Analysis] Support normalizer in request param #24767 (issue: #23347)
  • Enforce validation for PathHierarchy tokenizer #23510
  • [analysis-icu] Allow setting unicodeSetFilter #20814 (issue: #20820)
CAT API
  • expand /_cat/nodes to return information about hard drive #21775 (issue: #21679)
Cluster
  • Validate a joining node’s version with version of existing cluster nodes #25808
  • Switch indices read-only if a node runs out of disk space #25541 (issue: #24299)
  • Add a cluster block that allows to delete indices that are read-only #24678
Core
  • Add max file size bootstrap check #25974
  • Add compatibility versions to main action response #25799
  • Index ids in binary form. #25352 (issues: #18154, #24615)
  • Explicitly reject duplicate data paths #25178
  • Use SPI in High Level Rest Client to load XContent parsers #25097
  • Upgrade to lucene-7.0.0-snapshot-a0aef2f #24775
  • Speed up PK lookups at index time. #19856
Engine
  • Add refresh stats tracking for realtime get #25052 (issue: #24806)
  • Introducing a translog deletion policy #24950
Exceptions
  • IllegalStateException: Only duplicated jar instead of classpath #24953
Highlighting
  • Picks offset source for the unified highlighter directly from the es mapping #25747 (issue: #25699)
Index APIs
  • Let primary own its replication group #25692 (issue: #25485)
  • Create index request should return the index name #25139 (issue: #23044)
Ingest
  • Add Ingest-Processor specific Rest Endpoints & Add Grok endpoint #25059 (issue: #24725)
  • Port support for commercial GeoIP2 databases from Logstash. #24889
  • add exclude_keys option to KeyValueProcessor #24876 (issue: #23856)
  • Allow removing multiple fields in ingest processor #24750 (issue: #24622)
  • Add target_field parameter to ingest processors #24133 (issues: #23228, #23682)
Inner Hits
Internal
  • Cleanup IndexFieldData visibility #25900
  • Bump the min compat version to 5.6.0 #25805
  • "shard started" should show index and shard ID #25157
  • Break out clear scroll logic from TransportClearScrollAction #25125 (issue: #25094)
  • Add helper methods to TransportActionProxy to identify proxy actions and requests #25124
  • Add remote cluster infrastructure to fetch discovery nodes. #25123 (issue: #25094)
  • Add the ability to set eager_global_ordinals in the new parent-join field #25019
  • Disallow multiple parent-join fields per mapping #25002
  • Remove the need for _UNRELEASED suffix in versions #24798 (issue: #24768)
  • Optimize the order of bytes in uuids for better compression. #24615 (issue: #18209)
Java API
  • Always Accumulate Transport Exceptions #25017 (issue: #23099)
Java High Level REST Client
  • [DOCS] restructure java clients docs pages #25517
  • Use SPI in High Level Rest Client to load XContent parsers #25098 (issues: #25024, #25097)
  • Add support for clear scroll to high level REST client #25038
  • Add search scroll method to high level REST client #24938 (issue: #23331)
  • Add search method to high level REST client #24796 (issues: #24794, #24795)
Java REST Client
  • Shade external dependencies in the rest client jar #25780 (issue: #25208)
  • RestClient uses system properties and system default SSLContext #25757 (issue: #23231)
Logging
  • Prevent excessive disk consumption by log files #25660
  • Use LRU set to reduce repeat deprecation messages #25474 (issue: #25457)
Mapping
  • Better validation of copy_to. #25983
  • Optimize terms queries on ip addresses to use a PointInSetQuery whenever possible. #25669 (issue: #25667)
Network
Packaging
  • Remove memlock suggestion from systemd service #25979
  • Set address space limit in systemd service file #25975
  • Version option should display if snapshot #25970
  • Ignore JVM options before checking Java version #25969
  • Also skip JAVA_TOOL_OPTIONS on Windows #25968
  • Introduce elasticsearch-env for Windows #25958
  • Introduce elasticsearch-env #25815 (issue: #20286)
  • Stop exporting HOSTNAME from scripts #25807
Parent/Child
  • Remove ParentJoinFieldSubFetchPhase #25550 (issue: #25363)
  • Support parent id being specified as number in the _source #25547
Plugin Lang Painless
  • Allow Custom Whitelists in Painless #25557
  • Update Painless to Allow Augmentation from Any Class #25360
  • Add Needs Methods to Painless Script Context Factories #25267
  • Support Script Context Stateful Factory in Painless #25233
  • Generate Painless Factory for Creating Script Instances #25120
  • Update Painless to Use New Script Contexts #25015
  • Optimize instance creation in LambdaBootstrap #24618
Plugin Repository GCS
  • GCS Repository: Add secure storage of credentials #24697
Plugin Repository S3
  • S3 Repository: Add back repository level credentials #24609
Plugins
  • Move tribe to a module #25778
  • Plugins can register pre-configured char filters #25000 (issue: #23658)
  • Add purge option to remove plugin CLI #24981
  • Allow plugins to register pre-configured tokenizers #24751 (issues: #24223, #24572)
  • Move ReindexAction class to core #24684 (issue: #24578)
  • Make PreConfiguredTokenFilter harder to misuse #24572 (issue: #23658)
Query DSL
  • Make slop optional when parsing span_near query #25677 (issue: #25642)
  • Require a field when a seed is provided to the random_score function. #25594 (issue: #25240)
REST
  • Refactor PathTrie and RestController to use a single trie for all methods #25459 (issue: #24437)
  • Make ObjectParser support string to boolean conversion #24668 (issue: #21802)
Recovery
  • Goodbye, Translog Views #25962
  • Disallow multiple concurrent recovery attempts for same target shard #25428
  • Live primary-replica resync (no rollback) #24841 (issue: #10708)
Scripting
  • Scripting: Rename SearchScript.needsScores to needs_score #25235
  • Scripting: Add optional context parameter to put stored script requests #25014
  • Add New Security Script Settings #24637 (issue: #24532)
Search
  • Rewrite search requests on the coordinating nodes #25814 (issue: #25791)
  • Ensure query resources are fetched asynchronously during rewrite #25791
  • Introduce a new Rewriteable interface to streamline rewriting #25788
  • Reduce the scope of QueryRewriteContext #25787
  • Reduce the overhead of timeouts and low-level search cancellation. #25776
  • Reduce profiling overhead. #25772 (issue: #24799)
  • Prevent can_match requests from sending to incompatible nodes #25705 (issue: #25704)
  • Add a shard filter search phase to pre-filter shards based on query rewriting #25658
  • Ensure we rewrite common queries to match_none if possible #25650
  • Limit the number of concurrent shard requests per search request #25632
  • Add cluster name validation to RemoteClusterConnection #25568
  • Speed up sorted scroll when the index sort matches the search sort #25138 (issue: #6720)
  • Leverage scorerSupplier when applicable. #25109
  • Add Cross Cluster Search support for scroll searches #25094
  • Track EWMA[1] of task execution time in search threadpool executor #24989 (issue: #24915)
  • Query range fields by doc values when they are expected to be more efficient than points #24823 (issue: #24314)
  • Search: Fairer balancing when routing searches by session ID #24671 (issue: #24642)
Sequence IDs
  • Move primary term from ReplicationRequest to ConcreteShardRequest #25822
  • Add reason to global checkpoint updates on replica #25612 (issue: #10708)
  • Introduce primary/replica mode for GlobalCheckPointTracker #25468
  • Throw back replica local checkpoint on new primary #25452 (issues: #10708, #25355)
  • Update global checkpoint when increasing primary term on replica #25422 (issues: #10708, #25355)
  • Enable a long translog retention policy by default #25294 (issues: #10708, #25147)
  • Introduce primary context #25122 (issues: #10708, #25355)
  • Block older operations on primary term transition #24779 (issue: #10708)
Settings
  • Add disk threshold settings validation #25600 (issue: #25560)
  • Enable cross-setting validation #25560 (issue: #25541)
  • Validate transport.profiles.* settings #25508
  • Cleanup network / transport related settings #25489
  • Emit settings deprecation logging at most once #25457
  • IndexMetaData: Introduce internal format index setting #25292
Snapshot/Restore
  • Improves snapshot logging and snapshot deletion error handling #25264
Stats
  • Update IndexShard#refreshMetric via a ReferenceManager.RefreshListener #25083 (issues: #24806, #25052)
Translog
  • Translog file recovery should not rely on lucene commits #25005 (issue: #24950)

Bug Fixes

edit
Aggregations
  • Fixes array out of bounds for value count agg #26038 (issue: #17379)
  • Aggregations bug: Significant_text fails on arrays of text. #25030 (issue: #25029)
Aliases
  • mget with an alias shouldn’t ignore alias routing #25697 (issue: #25696)
  • GET aliases should 404 if aliases are missing #25043 (issue: #24644)
Analysis
  • Pre-configured shingle filter should disable graph analysis #25853 (issue: #25555)
Circuit Breakers
  • Checks the circuit breaker before allocating bytes for a new big array #25010 (issue: #24790)
Core
  • Release operation permit on thread-pool rejection #25930 (issue: #25863)
  • Node should start up despite of a lingering .es_temp_file #21210 (issue: #21007)
Discovery
  • MasterNodeChangePredicate should use the node instance to detect master change #25877 (issue: #25471)
Engine
  • Engine - do not index operations with seq# lower than the local checkpoint into lucene #25827 (issues: #1, #2, #25592)
Geo
  • Fix typo in GeoUtils#isValidLongitude #25121
Highlighting
  • FastVectorHighlighter should not cache the field query globally #25197 (issue: #25171)
  • Higlighters: Fix MultiPhrasePrefixQuery rewriting #25103 (issue: #25088)
Index APIs
  • Shrink API should ignore templates #25380 (issue: #25035)
  • Rollover max docs should only count primaries #24977 (issue: #24217)
Ingest
  • Sort Processor does not have proper behavior with targetField #25237 (issue: #24133)
  • fix grok’s pattern parsing to validate pattern names in expression #25063 (issue: #22831)
Inner Hits
  • When fetching nested inner hits only access stored fields when needed #25864 (issue: #6)
Internal
  • Fix BytesReferenceStreamInput#skip with offset #25634
  • Fix race condition in RemoteClusterConnection node supplier #25432
  • Initialise empty lists in BaseTaskResponse constructor #25290
  • Extract a common base class for scroll executions #24979 (issue: #16555)
  • Obey lock order if working with store to get metadata snapshots #24787 (issue: #24481)
  • Fix Version based BWC and set correct minCompatVersion #24732
  • Fix _field_caps serialization in order to support cross cluster search #24722
  • Avoid race when shutting down controller processes #24579
Mapping
  • Fix parsing of ip range queries. #25768 (issue: #25636)
  • Disable date field mapping changing #25285 (issue: #25271)
  • Correctly enable _all for older 5.x indices #25087 (issue: #25068)
  • token_count datatype should handle null value #25046 (issue: #24928)
  • keep _parent field while updating child type mapping #24407 (issue: #23381)
More Like This
  • Pass over _routing value with more_like_this items to be retrieved #24679 (issue: #23699)
Nested Docs
  • In case of a single type the _id field should be added to the nested document instead of _uid field #25149
Network
  • Ensure pending transport handlers are invoked for all channel failures #25150
  • Notify onConnectionClosed rather than onNodeDisconnect to prune transport handlers #24639 (issues: #24557, #24575, #24632)
Packaging
  • Exit Windows scripts promptly on failure #25959
  • Pass config path as a system property #25943
  • ES_HOME needs to be made absolute before attempt at traversal #25865
  • Fix elasticsearch-keystore handling of path.conf #25811
  • Stop disabling explicit GC #25759
  • Avoid failing install if system-sysctl is masked #25657 (issue: #24234)
  • Get short path name for native controllers #25344
  • When stopping via systemd only kill the JVM, not its control group #25195
  • remove remaining references to scripts directory #24771
  • Handle parentheses in batch file path #24731 (issue: #24712)
Parent/Child
  • The default _parent field should not try to load global ordinals #25851 (issue: #25849)
Percolator
  • Fix range queries with date range based on current time in percolator queries. #24666 (issue: #23921)
Plugin Lang Painless
  • Painless: allow doubles to be casted to longs. #25936
Plugin Repository Azure
  • Make calls to CloudBlobContainer#exists privileged #25937 (issue: #25931)
Plugin Repository GCS
  • Ensure that gcs client creation is privileged #25938 (issue: #25932)
Plugin Repository HDFS
  • Upgrading HDFS Repository Plugin to use HDFS 2.8.1 Client #25497 (issue: #25450)
Plugin Repository S3
  • Avoid SecurityException in repository-S3 on DefaultS3OutputStream.flush() #25254 (issue: #25192)
Plugins
  • X-Pack plugin download fails on Windows desktop #24570
Query DSL
  • SpanNearQueryBuilder should return the inner clause when a single clause is provided #25856 (issue: #25630)
  • Refactor field expansion for match, multi_match and query_string query #25726 (issues: #25551, #25556)
  • WrapperQueryBuilder should also rewrite the parsed query #25480
REST
  • Fix handling of invalid error trace parameter #25785 (issue: #25774)
  • Fix handling of exceptions thrown on HEAD requests #25172 (issue: #21125)
  • Fixed NPEs caused by requests without content. #23497 (issue: #24701)
  • Fix get mappings HEAD requests #23192 (issue: #21125)
Recovery
  • Close translog view after primary-replica resync #25862 (issue: #24841)
Reindex API
  • Reindex: don’t duplicate _source parameter #24629 (issue: #24628)
  • Add qa module that tests reindex-from-remote against pre-5.0 versions of Elasticsearch #24561 (issues: #23828, #24520)
Search
  • Caching a MinDocQuery can lead to wrong results. #25909
  • Fix random score generation when no seed is provided. #25908
  • Merge FunctionScoreQuery and FiltersFunctionScoreQuery #25889 (issues: #15709, #23628)
  • Respect cluster alias in _index aggs and queries #25885 (issue: #25606)
  • First increment shard stats before notifying and potentially sending response #25818
  • Remove assertion about deviation when casting to a float. #25806 (issue: #25330)
  • Prevent skipping shards if a suggest builder is present #25739 (issue: #25658)
  • Ensure remote cluster alias is preserved in inner hits aggs #25627 (issue: #25606)
  • Do not search locally if remote index pattern resolves to no indices #25436 (issue: #25426)
  • Adds check for negative search request size #25397 (issue: #22530)
  • Make sure range queries are correctly profiled. #25108
  • Fix RangeFieldMapper rangeQuery to properly handle relations #24808 (issue: #24744)
  • Fix ExpandSearchPhase when response contains no hits #24688 (issue: #24672)
Sequence IDs
  • Fix pre-6.0 response to unknown replication actions #25744 (issue: #10708)
  • Track local checkpoint on primary immediately #25434 (issues: #10708, #25355, #25415)
  • Initialize max unsafe auto ID timestamp on shrink #25356 (issues: #10708, #25355)
  • Use correct primary term for replicating NOOPs #25128
  • Handle already closed while filling gaps #25021 (issue: #24925)
Settings
  • Fix settings serialization to not serialize secure settings or not take the total size into account #25323
  • Keystore CLI should use the AddFileKeyStoreCommand for files #25298
  • Allow resetting settings that use an IP validator #24713 (issue: #24709)
Snapshot/Restore
  • Snapshot/Restore: Ensure that shard failure reasons are correctly stored in CS #25941 (issue: #25878)
  • Output all empty snapshot info fields if in verbose mode #25455 (issue: #24477)
  • Remove redundant and broken MD5 checksum from repository-s3 #25270 (issue: #25269)
  • Consolidates the logic for cleaning up snapshots on master election #24894 (issue: #24605)
  • Removes completed snapshot from cluster state on master change #24605 (issue: #24452)
Stats
  • _nodes/stats should not fail due to concurrent AlreadyClosedException #25016 (issue: #23099)
Suggesters
  • Context suggester should filter doc values field #25858 (issue: #25404)

Regressions

edit
Highlighting
  • Fix Fast Vector Highlighter NPE on match phrase prefix #25116 (issue: #25088)
Search
  • Always use DisjunctionMaxQuery to build cross fields disjunction #25115 (issue: #23966)

Upgrades

edit
Network
Upgrade API
  • Improve stability and logging of TemplateUpgradeServiceIT tests #25386 (issue: #25382)
« Elasticsearch version 6.0.0-beta2 Elasticsearch version 6.0.0-alpha2 »