IMPORTANT: No additional bug fixes or documentation updates
will be released for this version. For the latest information, see the
current release documentation.
Elasticsearch version 7.0.0
editElasticsearch version 7.0.0
editThese release notes include all changes made in the alpha, beta, and RC releases of 7.0.0.
Also see Breaking changes in 7.0.
Known issues
edit- Applying deletes or updates on an index after it has been shrunk may corrupt the index. In order to prevent this issue, it is recommended to stop shrinking read-write indices. For read-only indices, it is recommended to force-merge indices after shrinking, which significantly reduces the likeliness of this corruption in the case that deletes/updates would be applied by mistake. This bug is fixed in Elasticsearch 7.7 and later versions. More details can be found on the corresponding issue.
-
Indices created in 6.x with
date
anddate_nanos
fields using formats that are incompatible with java.time patterns will cause parsing errors, incorrect date calculations or wrong search results. https://github.com/elastic/elasticsearch/pull/52555 This is fixed in Elasticsearch 7.7 and later versions.
Breaking changes
edit- Aggregations
- Analysis
- Audit
- Authentication
- Authorization
- Circuit Breakers
- CRUD
- Distributed
- Features/Features
- Features/Indices APIs
- Features/Ingest
- Features/Java Low Level REST Client
- Features/Watcher
- Features/Stats
- Geo
- Index APIs
- Infra/Circuit Breakers
- Infra/Core
-
- Default node.name to the hostname #33677
- Remove bulk fallback for write thread pool #29609
- CCS: Drop http address from remote cluster info #29568 (issue: #29207)
- Remove the index thread pool #29556
- Main response should not have status 503 when okay #29045 (issue: #8902)
- Automatically prepare indices for splitting #27451
-
Don’t refresh on
_flush
_force_merge
and_upgrade
#27000 (issue: #26972)
- Infra/Logging
- Infra/Packaging
- Infra/REST API
- Infra/Scripting
- Infra/Settings
-
- Remove config prompting for secrets and text #27216
- Machine Learning
- Mapping
-
- Match phrase queries against non-indexed fields should throw an exception #31060
- Remove legacy mapping code. #29224
-
Reject updates to the
_default_
mapping. #29165 (issues: #15613, #28248) -
Remove the
update_all_types
option. #28288 -
Remove the
_default_
mapping. #28248 -
Reject the
index_options
parameter for numeric fields #26668 (issue: #21475) - Update the default for include_type_name to false. #37285
- Support include_type_name in RestGetIndicesAction #37149
- Network
- Percolator
-
- Remove deprecated percolator map_unmapped_fields_as_string setting #28060
- Ranking
-
- Add minimal sanity checks to custom/scripted similarities. #33564 (issue: #33309)
- Scroll queries asking for rescore are considered invalid #32918 (issue: #31775)
- Forbid negative scores in function_score query #35709 (issue: #33309)
- Forbid negative field boosts in analyzed queries #37930 (issue: #33309)
- Scripting
- Search
-
-
Remove deprecated url parameters
_source_include
and_source_exclude
#35097 (issues: #22792, #33475) - Disallow negative query boost #34486 (issue: #33309)
-
Forbid negative
weight
in Function Score Query #33390 (issue: #31927) - In the field capabilities API, remove support for providing fields in the request body. #30185
- Remove deprecated options for query_string #29203 (issue: #25551)
- Fix Laplace scorer to multiply by alpha (and not add) #27125
- Remove _primary and _replica shard preferences #26791 (issue: #26335)
- Limit the number of expanded fields it query_string and simple_query_string #26541 (issue: #25105)
- Make purely negative queries return scores of 0. #26015 (issue: #23449)
- Remove the deprecated _termvector endpoint. #36131 (issues: #36098, #8484)
- Remove deprecated Graph endpoints #35956
-
Validate metadata on
_msearch
#35938 (issue: #35869) - Make hits.total an object in the search response #35849 (issue: #33028)
- Remove the distinction between query and filter context in QueryBuilders #35354 (issue: #35293)
- Throw a parsing exception when boost is set in span_or query (#28390) #34112 (issue: #28390)
- Track total hits up to 10,000 by default #37466 (issue: #33028)
- Use mappings to format doc-value fields by default. #30831 (issues: #26948, #29639)
-
Remove deprecated url parameters
- Security
- Snapshot/Restore
- Store
- Suggesters
- ZenDiscovery
Breaking Java changes
editDeprecations
edit- Aggregations
- Analysis
-
- Replace parameter unicodeSetFilter with unicode_set_filter #29215 (issue: #22823)
- Replace delimited_payload_filter by delimited_payload #26625 (issue: #21978)
- Deprecate Standard Html Strip Analyzer in master #26719 (issue: #4704)
-
Remove
nGram
andedgeNGram
token filter names (#38911) #39070 (issues: #30209, #38911)
- Audit
- Core
- Cluster Coordination
- Features/Indices APIs
- Features/Ingest
- Features/Java High Level REST Client
- Features/Watcher
- Graph
-
-
Deprecate types in
_graph/explore
calls. #40466
-
Deprecate types in
- Infra/Core
- Infra/Packaging
-
- Deprecate fallback to java on PATH #37990
- Infra/Scripting
- Infra/Transport API
-
- Deprecate the transport client in favour of the high-level REST client #27085
- Machine Learning
-
- Deprecate X-Pack centric ML endpoints #36315 (issue: #35958)
- Adding ml_settings entry to HLRC and Docs for deprecation_info #38118
- Datafeed deprecation checks #38026 (issue: #37932)
- Remove "8" prefixes from file structure finder timestamp formats #38016
- Adjust structure finder for Joda to Java time migration #37306
- Resolve 7.0.0 TODOs in ML code #36842 (issue: #29963)
- Mapping
-
- Deprecate type exists requests. #34663
- Deprecate types in index API #36575 (issues: #35190, #35790)
- Deprecate uses of _type as a field name in queries #36503 (issue: #35190)
- Deprecate types in update_by_query and delete_by_query #36365 (issue: #35190)
- For msearch templates, make sure to use the right name for deprecation logging. #36344
- Deprecate types in termvector and mtermvector requests. #36182
- Deprecate types in update requests. #36181
- Deprecate types in document delete requests. #36087
- Deprecate types in get, exists, and multi get. #35930
- Deprecate types in search and multi search templates. #35669
- Deprecate types in explain requests. #35611
- Deprecate types in validate query requests. #35575
- Deprecate types in count and msearch. #35421 (issue: #34041)
- Deprecate types in rollover index API #38039 (issue: #35190)
- Deprecate types in get field mapping API #37667 (issue: #35190)
- Deprecate types in the put mapping API. #37280 (issues: #29453, #37285)
- Support include_type_name in the field mapping and index template APIs. #37210
- Deprecate types in create index requests. #37134 (issues: #29453, #37285)
- Deprecate use of the _type field in aggregations. #37131 (issue: #36802)
- Deprecate reference to _type in lookup queries #37016 (issue: #35190)
- Deprecate the document create endpoint. #36863
- Deprecate types in index API #36575 (issues: #35190, #35790)
- Deprecate types in update APIs #36225
- Migration
- Monitoring
- Rollup
- Scripting
- Search
- Security
- SQL
- Watcher
New features
edit- Allocation
- Analysis
- Authentication
- Authorization
- CCR
- Distributed
- Features/ILM
- Features/Ingest
- Features/Java High Level REST Client
-
- GraphClient for the high level REST client and associated tests #32366
- Features/Monitoring
- Geo
-
-
Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default
geo_shape
indexing approach #36751 (issue: #35320) -
Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default
geo_shape
indexing approach #35320 (issue: #32039) - geotile_grid implementation #37842 (issue: #30240)
- Fork Lucene’s LatLonShape Classes to local lucene package #36794
-
Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default
geo_shape
indexing approach #36751 (issue: #35320) -
Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default
geo_shape
indexing approach #35320 (issue: #32039)
-
Integrate Lucene’s LatLonShape (BKD Backed GeoShapes) as default
- Infra/Core
-
-
Skip shard refreshes if shard is
search idle
#27500
-
Skip shard refreshes if shard is
- Infra/Logging
-
- Logging: Unify log rotation for index/search slow log #27298
- Infra/Plugins
- Infra/REST API
- Java High Level REST Client
- Java Low Level REST Client
- Machine Learning
- Mapping
-
-
Add a
feature_vector
field. #31102 (issue: #27552) - Expose Lucene’s FeatureField. #30618
-
Make typeless APIs usable with indices whose type name is different from
_doc
#35790 (issue: #35190) - Give precedence to index creation when mixing typed templates with typeless index creation and vice-versa. #37871 (issue: #37773)
- Add nanosecond field mapper #37755 (issues: #27330, #32601)
-
Add a
- Ranking
- Recovery
- SQL
-
- Add basic support for ST_AsWKT geo function #34205
- Add support for SYS GEOMETRY_COLUMNS #30496 (issue: #29872)
- Introduce HISTOGRAM grouping function #36510 (issue: #36509)
- DATABASE() and USER() system functions #35946 (issue: #35863)
- Introduce INTERVAL support #35521 (issue: #29990)
- Allow sorting of groups by aggregates #38042 (issue: #35118)
- Implement FIRST/LAST aggregate functions #37936 (issue: #35639)
- Introduce SQL DATE data type #37693 (issue: #37340)
- Search
-
- Add “took” timing info to response for _msearch/template API #30961 (issue: #30957)
- Add allow_partial_search_results flag to search requests with default setting true #28440 (issue: #27435)
- Enable adaptive replica selection by default #26522 (issue: #24915)
- Add intervals query #36135 (issues: #29636, #32406)
- Added soft limit to open scroll contexts #25244 #36009 (issue: #25244)
- Introduce ability to minimize round-trips in CCS #37828 (issues: #32125, #37566)
- Add script filter to intervals #36776
- Add the ability to set the number of hits to track accurately #36357 (issue: #33028)
- Add a maximum search request size. #26423
- Make IntervalQuery available via the Query DSL #36135 (issue: #29636)
- Security
- Suggesters
Enhancements
edit- Aggregations
-
- Uses MergingDigest instead of AVLDigest in percentiles agg #28702 (issue: #19528)
- Added keyed response to pipeline percentile aggregations 22302 #36392 (issue: #22302)
- Enforce max_buckets limit only in the final reduction phase #36152 (issues: #32125, #35921)
- Histogram aggs: add empty buckets only in the final reduce step #35921
- Handles exists query in composite aggs #35758
- Added parent validation for auto date histogram #35670
- Add Composite to AggregationBuilders #38207 (issue: #38020)
- Allow nested fields in the composite aggregation #37178 (issue: #28611)
- Remove single shard optimization when suggesting shard_size #37041 (issue: #32125)
- Use List instead of priority queue for stable sorting in bucket sort aggregator #36748 (issue: #36322)
- Keys are compared in BucketSortPipelineAggregation so making key type… #36407
- Allocation
- Analysis
- Audit
- Authentication
-
- Invalidate Token API enhancements - HLRC #36362 (issue: #35388)
- Add DEBUG/TRACE logs for LDAP bind #36028
- Add Tests for findSamlRealm #35905
- Add realm information for Authenticate API #35648
- Formal support for "password_hash" in Put User #35242 (issue: #34729)
- Propagate auth result to listeners #36900 (issue: #30794)
- Reorder realms based on last success #36878
- Improve error message for 6.x style realm settings #36876 (issues: #30241, #36026)
- Change missing authn message to not mention tokens #36750
- Invalidate Token API enhancements - HLRC #36362 (issue: #35388)
- Enhance Invalidate Token API #35388 (issues: #34556, #35115)
- Authorization
-
- Improve exact index matching performance #36017
-
manage_token
privilege forkibana_system
#35751 - Grant .tasks access to kibana_system role #35573
- Add apm_user reserved role #38206
- Permission for restricted indices #37577 (issue: #34454)
- Remove kibana_user and kibana_dashboard_only_user index privileges #37441
- Create snapshot role #35820 (issue: #34454)
- Build
- CCR
-
- Add time since last auto follow fetch to auto follow stats #36542 (issues: #33007, #35895)
- Clean followed leader index UUIDs in auto follow metadata #36408 (issue: #33007)
- Change AutofollowCoordinator to use wait_for_metadata_version #36264 (issues: #33007, #35895)
- Refactor AutoFollowCoordinator to track leader indices per remote cluster #36031 (issues: #33007, #35895)
- Concurrent file chunk fetching for CCR restore #38495
- Tighten mapping syncing in ccr remote restore #38071 (issues: #36879, #37887)
- Do not allow put mapping on follower #37675 (issue: #30086)
- Added ccr to xpack usage infrastructure #37256 (issue: #37221)
- FollowingEngine should fail with 403 if operation has no seqno assigned #37213
- Added auto_follow_exception.timestamp field to auto follow stats #36947
- Add time since last auto follow fetch to auto follow stats #36542 (issues: #33007, #35895)
- Reduce retention lease sync intervals #40302
- Renew retention leases while following #39335 (issues: #37165, #38718)
- Reduce refresh when lookup term in FollowingEngine #39184
- Integrate retention leases to recovery from remote #38829 (issue: #37165)
- Enable removal of retention leases #38751 (issue: #37165)
- Introduce forget follower API #39718 (issue: #37165)
- Client
-
- Fixed required fields and paths list #39358
- Cluster Coordination
- Core
-
- Override the JVM DNS cache policy #36570
- Replace usages of AtomicBoolean based block of code by the RunOnce class #35553 (issue: #35489)
- Added wait_for_metadata_version parameter to cluster state api. #35535
- Extract RunOnce into a dedicated class #35489
- Introduce elasticsearch-core jar #28191 (issue: #27933)
- Rename core module to server #28180 (issue: #27933)
- CRUD
-
- Rename seq# powered optimistic concurrency control parameters to ifSeqNo/ifPrimaryTerm #36757 (issues: #10708, #36148)
- Expose Sequence Number based Optimistic Concurrency Control in the rest layer #36721 (issues: #10708, #36148)
- Add doc’s sequence number + primary term to GetResult and use it for updates #36680 (issues: #10708, #36148)
- Add seq no powered optimistic locking support to the index and delete transport actions #36619 (issues: #10708, #36148)
- Add Seq# based optimistic concurrency control to UpdateRequest #37872 (issues: #10708, #36148)
- Introduce ssl settings to reindex from remote #37527 (issues: #29755, #37287)
- Use Sequence number powered OCC for processing updates #37308 (issues: #10708, #36148)
- Document Seq No powered optimistic concurrency control #37284 (issues: #10708, #36148)
- Enable IPv6 URIs in reindex from remote #36874
- Rename seq# powered optimistic concurrency control parameters to ifSeqNo/ifPrimaryTerm #36757 (issues: #10708, #36148)
- Expose Sequence Number based Optimistic Concurrency Control in the rest layer #36721 (issues: #10708, #36148)
- Add doc’s sequence number + primary term to GetResult and use it for updates #36680 (issues: #10708, #36148)
- Add seq no powered optimistic locking support to the index and delete transport actions #36619 (issues: #10708, #36148)
- Set acking timeout to 0 on dynamic mapping update #31140 (issues: #30672, #30844)
- Discovery-Plugins
- Distributed
-
- [Close Index API] Mark shard copy as stale if needed during shard verification #36755
- [Close Index API] Refactor MetaDataIndexStateService #36354 (issue: #36249)
- [Close Index API] Add TransportShardCloseAction for pre-closing verifications #36249
- TransportResyncReplicationAction should not honour blocks #35795 (issues: #35332, #35597)
- Expose all permits acquisition in IndexShard and TransportReplicationAction #35540 (issue: #33888)
- [RCI] Check blocks while having index shard permit in TransportReplicationAction #35332 (issue: #33888)
- Recover retention leases during peer recovery #38435 (issue: #37165)
- Lift retention lease expiration to index shard #38380 (issues: #37165, #37963, #38070)
- Introduce retention lease background sync #38262 (issue: #37165)
- Allow shards of closed indices to be replicated as regular shards #38024 (issue: #33888)
- Expose retention leases in shard stats #37991 (issue: #37165)
- Introduce retention leases versioning #37951 (issue: #37165)
- Soft-deletes policy should always fetch latest leases #37940 (issues: #37165, #37375)
- Sync retention leases on expiration #37902 (issue: #37165)
- Ignore shard started requests when primary term does not match #37899 (issue: #33888)
- Move update and delete by query to use seq# for optimistic concurrency control #37857 (issues: #10708, #36148, #37639)
- Introduce retention lease serialization #37447 (issues: #37165, #37398)
- Add run under primary permit method #37440 (issue: #37398)
- Introduce retention lease syncing #37398 (issue: #37165)
- Introduce retention lease persistence #37375 (issue: #37165)
- Add validation for retention lease construction #37312 (issue: #37165)
- Introduce retention lease expiration #37195 (issue: #37165)
- Introduce shard history retention leases #37167 (issue: #37165)
- [Close Index API] Add unique UUID to ClusterBlock #36775
- [Close Index API] Mark shard copy as stale if needed during shard verification #36755
- [Close Index API] Propagate tasks ids between Freeze, Close and Verify Shard actions #36630
- Always initialize the global checkpoint #34381
- Introduce retention lease actions #38756 (issue: #37165)
- Add dedicated retention lease exceptions #38754 (issue: #37165)
- Copy retention leases when trim unsafe commits #37995 (issue: #37165)
- Allow retention lease operations under blocks #39089 (issues: #34648, #37165)
- Remove retention leases when unfollowing #39088 (issues: #34648, #37165)
- Introduce retention lease state file #39004 (issues: #37165, #38588, #39032)
- Enable soft-deletes by default for 7.0+ indices #38929 (issue: #36141)
- Engine
-
- Remove versionType from translog #31945
- Do retry if primary fails on AsyncAfterWriteAction #31857 (issues: #31716, #31755)
- handle AsyncAfterWriteAction exception before listener is registered #31755 (issue: #31716)
- Use IndexWriter#flushNextBuffer to free memory #27753
- Remove pre 6.0.0 support from InternalEngine #27720
- Add sequence numbers based optimistic concurrency control support to Engine #36467 (issues: #10708, #36148)
- Require soft-deletes when access changes snapshot #36446
- Use delCount of SegmentInfos to calculate numDocs #36323
- Always configure soft-deletes field of IndexWriterConfig #36196 (issue: #36141)
- Enable soft-deletes by default on 7.0.0 or later #36141
-
Always return false from
refreshNeeded
on ReadOnlyEngine #35837 (issue: #35785) -
Add a
_freeze
/_unfreeze
API #35592 (issue: #34352) - [RCI] Add IndexShardOperationPermits.asyncBlockOperations(ActionListener<Releasable>) #34902 (issue: #33888)
- Specialize pre-closing checks for engine implementations #38702
- Ensure that max seq # is equal to the global checkpoint when creating ReadOnlyEngines #37426
- Enable Bulk-Merge if all source remains #37269
- Rename setting to enable mmap #37070 (issue: #36668)
- Add hybridfs store type #36668
- Introduce time-based retention policy for soft-deletes #34943 (issue: #34908)
- Handle AsyncAfterWriteAction failure on primary in the same way as failures on replicas #31969 (issues: #31716, #31755)
- Explicitly advance max_seq_no before indexing #39473 (issue: #38879)
- Also mmap cfs files for hybridfs #38940 (issue: #36668)
- Features/CAT APIs
- Features/Features
- Features/ILM
- Features/Indices APIs
-
- Add cluster-wide shard limit #32856 (issue: #20705)
- Remove RestGetAllAliasesAction #31308 (issue: #31129)
- Add rollover-creation-date setting to rolled over index #31144 (issue: #30887)
- add is-write-index flag to aliases #30942
- Make index and bulk APIs work without types. #29479
- Simplify deprecation issue levels #36326
- New mapping signature and mapping string source fixed. #37401
- Features/Ingest
-
- Add ignore_missing property to foreach filter (#22147) #31578 (issue: #22147)
- Compile mustache template only if field includes {{' #37207 (issue: #37120)
- Move ingest-geoip default databases out of config #36949 (issue: #36898)
- Make the ingest-geoip databases even lazier to load #36679
- Updates the grok patterns to be consistent with the logstash #27181
- Features/Java High Level REST Client
- Features/Monitoring
- Features/Stats
- Features/Watcher
-
- Validate email adresses when storing a watch #34042 (issue: #33980)
- Move watcher to use seq# and primary term for concurrency control #37977 (issues: #10708, #37872)
- Use ILM for Watcher history deletion #37443 (issue: #32041)
- Add whitelist to HttpClient #36817 (issue: #29937)
- Remove the index type from internal watcher indexes #39761 (issue: #38637)
- Geo
- Index APIs
- Infra/Circuit Breakers
- Infra/Core
-
- Add RunOnce utility class that executes a Runnable exactly once #35484
- Improved IndexNotFoundException’s default error message #34649 (issue: #34628)
- fix a few versionAdded values in ElasticsearchExceptions #37877
- Add simple method to write collection of writeables #37448 (issue: #37398)
- Date/Time parsing: Use java time API instead of exception handling #37222
- [API] spelling: interruptible #37049 (issue: #37035)
- Enhancements to IndicesQueryCache. #39099 (issue: #37117)
- Change zone formatting for all printers #39568 (issue: #38471)
- Infra/Logging
-
- Trim the JSON source in indexing slow logs #38081 (issue: #38080)
- Optimize warning header de-duplication #37725 (issues: #35754, #37530, #37597, #37622)
- Remove warn-date from warning headers #37622 (issues: #35754, #37530, #37597)
- Add some deprecation optimizations #37597 (issues: #35754, #37530)
- Only update response headers if we have a new one #37590 (issues: #35754, #37530)
- Infra/Packaging
-
- Choose JVM options ergonomically #30684
- Add OS/architecture classifier to distributions #37881
- Change file descriptor limit to 65535 #37537 (issue: #35839)
- Exit batch files explictly using ERRORLEVEL #29583 (issue: #29582)
- Add no-jdk distributions #39882
- Allow AVX-512 on JDK 11+ #40828 (issue: #32138)
- Infra/REST API
- Infra/Scripting
- Infra/Packaging
- Infra/Settings
- Infra/Transport API
- Ingest
- Java High Level REST Client
-
- Add get users action #36332 (issue: #29827)
- Add delete template API #36320 (issue: #27205)
- Implement get-user-privileges API #36292
- Get Deprecation Info API #36279 (issue: #29827)
- Add support for Follow Stats API #36253 (issue: #33824)
- Add support for CCR Stats API #36213 (issue: #33824)
- Put Role #36209 (issue: #29827)
- Add index templates exist API #36132 (issue: #27205)
- Add support for CCR Get Auto Follow Pattern apis #36049 (issue: #33824)
- Add support for CCR Delete Auto Follow Pattern API #35981 (issue: #33824)
- Remove fromXContent from IndexUpgradeInfoResponse #35934
- Add delete expired data API #35906 (issue: #29827)
- Execute watch API #35868 (issue: #29827)
- Add ability to put user with a password hash #35844 (issue: #35242)
- Add ML find file structure API #35833 (issue: #29827)
- Add support for get roles API #35787 (issue: #29827)
- Added support for CCR Put Auto Follow Pattern API #35780 (issue: #33824)
- XPack ML info action #35777 (issue: #29827)
- ML Delete event from Calendar #35760 (issue: #29827)
- Add ML revert model snapshot API #35750 (issue: #29827)
- ML Get Calendar Events #35747 (issue: #29827)
-
Add high-level REST client API for
_freeze
and_unfreeze
#35723 (issue: #34352) - Fix issue in equals impl for GlobalOperationPrivileges #35721
- ML Delete job from calendar #35713 (issue: #29827)
- ML Add Event To Calendar API #35704 (issue: #29827)
- Add ML update model snapshot API (#35537) #35694 (issue: #29827)
- Add support for CCR Unfollow API #35693 (issue: #33824)
- Clean up PutLicenseResponse #35689 (issue: #35547)
- Clean up StartBasicResponse #35688 (issue: #35547)
- Add support for put privileges API #35679
- ML Add Job to Calendar API #35666 (issue: #29827)
- Add support for CCR Resume Follow API #35638 (issue: #33824)
- Add support for get application privileges API #35556 (issue: #29827)
- Clean up XPackInfoResponse class and related tests #35547
- Add parameters to stopRollupJob API #35545 (issue: #34811)
- Add ML delete model snapshot API #35537 (issue: #29827)
- Add get watch API #35531 (issue: #29827)
- Add ML Update Filter API #35522 (issue: #29827)
- Add ml get filters api #35502 (issue: #29827)
- Add ML get model snapshots API #35487 (issue: #29827)
- Add "_has_privileges" API to Security Client #35479 (issue: #29827)
- Add Delete Privileges API to HLRC #35454 (issue: #29827)
- Add support for CCR Put Follow API #35409
- Add ML delete filter action #35382 (issue: #29827)
- Add delete user action #35294 (issue: #29827)
- HLRC for _mtermvectors #35266 (issues: #27205, #33447)
- Reindex API with wait_for_completion false #35202 (issue: #27205)
- Add watcher stats API #35185 (issue: #29827)
- HLRC support for getTask #35166 (issue: #27205)
- Add GetRollupIndexCaps API #35102 (issue: #29827)
- HLRC: migration api - upgrade #34898 (issue: #29827)
- Add stop rollup job support to HL REST Client #34702 (issue: #29827)
- Bulk Api support for global parameters #34528 (issue: #26026)
- Add delete rollup job support to HL REST Client #34066 (issue: #29827)
- Add support for get license basic/trial status API #33176 (issue: #29827)
- Add machine learning open job #32860 (issue: #29827)
- Add ML HLRC wrapper and put_job API call #32726
- Add Get Snapshots High Level REST API #31537 (issue: #27205)
- Java Low Level REST Client
-
- On retry timeout add root exception #25576
- License
- Machine Learning
-
- Create the ML annotations index #36731 (issues: #26034, #33376)
- Split in batches and migrate all jobs and datafeeds #36716 (issue: #32905)
- Add cluster setting to enable/disable config migration #36700 (issue: #32905)
- Add audits when deprecation warnings occur with datafeed start #36233
- Add lazy parsing for DatafeedConfig:Aggs,Query #36117
- Add support for lazy nodes (#29991) #34538 (issue: #29991)
- Move ML Optimistic Concurrency Control to Seq No #38278 (issues: #10708, #36148)
- Add explanation so far to file structure finder exceptions #38191 (issue: #29821)
- Add reason field in JobTaskState #38029 (issue: #34431)
- Add _meta information to all ML indices #37964
- Add upgrade mode docs, hlrc, and fix bug #37942
- Tighten up use of aliases rather than concrete indices #37874
- Add support for single bucket aggs in Datafeeds #37544 (issue: #36838)
- Create the ML annotations index #36731 (issues: #26034, #33376)
- Merge the Jindex master feature branch #36702 (issue: #32905)
- Add cluster setting to enable/disable config migration #36700 (issue: #32905)
- Allow stop unassigned datafeed and relax unset upgrade mode wait #39034
- Mapping
- Network
-
- Add cors support to NioHttpServerTransport #30827 (issue: #28898)
- Reintroduce mandatory http pipelining support #30820
- Make http pipelining support mandatory #30695 (issues: #28898, #29500)
- Add nio http server transport #29587 (issue: #28898)
- Add class for serializing message to bytes #29384 (issue: #28898)
- Selectors operate on channel contexts #28468 (issue: #27260)
- Unify nio read / write channel contexts #28160 (issue: #27260)
- Create nio-transport plugin for NioTransport #27949 (issue: #27260)
- Add elasticsearch-nio jar for base nio classes #27801 (issue: #27802)
- Unify transport settings naming #36623
- Add sni name to SSLEngine in netty transport #33144 (issue: #32517)
- Add cors support to NioHttpServerTransport #30827 (issue: #28898)
- Reintroduce mandatory http pipelining support #30820
- Make http pipelining support mandatory #30695 (issues: #28898, #29500)
- Add nio http server transport #29587 (issue: #28898)
- Selectors operate on channel contexts #28468 (issue: #27260)
- Unify nio read / write channel contexts #28160 (issue: #27260)
- Create nio-transport plugin for NioTransport #27949 (issue: #27260)
- Add elasticsearch-nio jar for base nio classes #27801 (issue: #27802)
- Add NioGroup for use in different transports #27737 (issue: #27260)
- Add read timeouts to http module #27713
-
Implement byte array reusage in
NioTransport
#27696 (issue: #27563) - Introduce resizable inbound byte buffer #27551 (issue: #27563)
- Decouple nio constructs from the tcp transport #27484 (issue: #27260)
- Remove manual tracking of registered channels #27445 (issue: #27260)
- Remove tcp profile from low level nio channel #27441 (issue: #27260)
-
Decouple
ChannelFactory
from Tcp classes #27286 (issue: #27260) - Enable TLSv1.3 by default for JDKs with support #38103 (issue: #32276)
- Packaging
- Percolator
- Plugins
-
- Plugin install: don’t print download progress in batch mode #36361
- Ranking
- Recovery
-
- SyncedFlushService.getShardRoutingTable() should use metadata to check for index existence #37691 (issue: #33888)
- Make prepare engine step of recovery source non-blocking #37573 (issue: #37174)
- Make recovery source send operations non-blocking #37503 (issue: #37458)
- Prepare to make send translog of recovery non-blocking #37458 (issue: #37291)
- Make finalize step of recovery source non-blocking #37388 (issue: #37291)
- Make recovery source partially non-blocking #37291 (issue: #36195)
- Do not mutate RecoveryResponse #37204 (issue: #37174)
- Don’t block on peer recovery on the target side #37076 (issue: #36195)
- Reduce recovery time with compress or secure transport #36981 (issue: #33844)
- Translog corruption marker #33415 (issue: #31389)
- Do not wait for advancement of checkpoint in recovery #39006 (issues: #38949, #39000)
- Rollup
- Recovery
- Scripting
- Settings
- Search
-
- Make limit on number of expanded fields configurable #35284 (issues: #26541, #34778)
- Search: Simply SingleFieldsVisitor #34052
- Don’t count hits via the collector if the hit count can be computed from index stats. #33701
- Limit the number of concurrent requests per node #31206 (issue: #31192)
- Default max concurrent search req. numNodes * 5 #31171 (issues: #30783, #30994)
- Change ScriptException status to 400 (bad request) #30861 (issue: #12315)
- Change default value to true for transpositions parameter of fuzzy query #26901
-
Introducing "took" time (in ms) for
_msearch
#23767 (issue: #23131) - Add copy constructor to SearchRequest #36641 (issue: #32125)
- Add raw sort values to SearchSortValues transport serialization #36617 (issue: #32125)
- Add sort and collapse info to SearchHits transport serialization #36555 (issue: #32125)
- Add default methods to DocValueFormat #36480
- Respect indices options on _msearch #35887
- Allow efficient can_match phases on frozen indices #35431 (issues: #34352, #34357)
- Add a new query type - ScriptScoreQuery #34533 (issues: #23850, #27588, #30303)
- Tie break on cluster alias when merging shard search failures #38715 (issue: #38672)
- Add finalReduce flag to SearchRequest #38104 (issues: #37000, #37838)
- Streamline skip_unavailable handling #37672 (issue: #32125)
- Expose sequence number and primary terms in search responses #37639
- Add support for merging multiple search responses into one #37566 (issue: #32125)
- Allow field types to optimize phrase prefix queries #37436 (issue: #31921)
- Add support for providing absolute start time to SearchRequest #37142 (issue: #32125)
- Ensure that local cluster alias is never treated as remote #37121 (issues: #32125, #36997)
- [API] spelling: cacheable #37047 (issue: #37035)
- Add ability to suggest shard_size on coord node rewrite #37017 (issues: #32125, #36997, #37000)
- Skip final reduction if SearchRequest holds a cluster alias #37000 (issues: #32125, #36997)
- Add support for local cluster alias to SearchRequest #36997 (issue: #32125)
- Use SearchRequest copy constructor in ExpandSearchPhase #36772 (issue: #36641)
- Add raw sort values to SearchSortValues transport serialization #36617 (issue: #32125)
- Avoid BytesRef’s copying in ScriptDocValues’s Strings #29581 (issue: #29567)
- Security
-
- Make credentials mandatory when launching xpack/migrate #36197 (issues: #29847, #33972)
- Move CAS operations in TokenService to sequence numbers #38311 (issues: #10708, #37872)
- Cleanup construction of interceptors #38294
- Add passphrase support to elasticsearch-keystore #37472 (issue: #32691)
- Types removal security index template #39705 (issue: #38637)
- Types removal security index template #39542 (issue: #38637)
- Snapshot/Restore
-
- #31608 Add S3 Setting to Force Path Type Access #34721 (issue: #31608)
- Allow Parallel Restore Operations #36397
- Repo Creation out of ClusterStateTask #36157 (issue: #9488)
- Add read-only repository verification #35731 (issue: #35703)
- RestoreService should update primary terms when restoring shards of existing indices #38177 (issue: #33888)
- Allow open indices to be restored #37733
- Create specific exception for when snapshots are in progress #37550 (issue: #37541)
- Make Atomic Blob Writes Mandatory #37168 (issues: #37011, #37066)
- Speed up HDFS Repository Writes #37069
- Implement Atomic Blob Writes for HDFS Repository #37066 (issue: #37011)
- [API] spelling: repositories #37053 (issue: #37035)
- Use CancellableThreads to Abort #35901 (issue: #21759)
- S3 client encryption #30513 (issues: #11128, #16843)
- Mark Deleted Snapshot Directories with Tombstones #40228 (issue: #39852)
- Stats
- Store
- SQL
-
- Introduce support for NULL values #34573 (issue: #32079)
- Extend the ODBC metric by differentiating between 32 and 64bit platforms #36753 (issue: #36740)
- Fix wrong appliance of StackOverflow limit for IN #36724 (issue: #36592)
- Introduce NOW/CURRENT_TIMESTAMP function #36562 (issue: #36534)
- Move requests' parameters to requests JSON body #36149 (issue: #35992)
- Make INTERVAL millis optional #36043 (issue: #36032)
- Implement data type verification for conditionals #35916 (issue: #35907)
- Implement GREATEST and LEAST functions #35879 (issue: #35878)
-
Implement null safe equality operator
<=>
#35873 (issue: #35871) - SYS COLUMNS returns ODBC specific schema #35870 (issue: #35376)
- Polish grammar for intervals #35853
- Add filtering to SYS TYPES #35852 (issue: #35342)
- Implement NULLIF(expr1, expr2) function #35826 (issue: #35818)
- Lock down JDBC driver #35798 (issue: #35437)
- Implement NVL(expr1, expr2) #35794 (issue: #35782)
- Implement ISNULL(expr1, expr2) #35793 (issue: #35781)
- Implement IFNULL variant of COALESCE #35762 (issue: #35749)
- XPack FeatureSet functionality #35725 (issue: #34821)
- Perform lazy evaluation of mismatched mappings #35676 (issues: #35659, #35675)
- Improve validation of unsupported fields #35675 (issue: #35673)
- Move internals from Joda to java.time #35649 (issue: #35633)
- Allow look-ahead resolution of aliases for WHERE clause #38450 (issue: #29983)
- Implement CURRENT_DATE #38175 (issue: #38160)
- Generate relevant error message when grouping functions are not used in GROUP BY #38017 (issue: #37952)
- Skip the nested and object field types in case of an ODBC request #37948 (issue: #37801)
- Add protocol tests and remove jdbc_type from drivers response #37516 (issues: #36635, #36882)
- Remove slightly used meta commands #37506 (issue: #37409)
- Describe aliases as views #37496 (issue: #37422)
-
Make
FULL
non-reserved keyword in the grammar #37377 (issue: #37376) - Use declared source for error messages #37161
- Improve error message when unable to translate to ES query DSL #37129 (issue: #37040)
- [API] spelling: subtract #37055 (issue: #37035)
- [API] spelling: similar #37054 (issue: #37035)
- [API] spelling: input #37048 (issue: #37035)
- Enhance message for PERCENTILE[_RANK] with field as 2nd arg #36933 (issue: #36903)
- Preserve original source for each expression #36912 (issue: #36894)
- Extend the ODBC metric by differentiating between 32 and 64bit platforms #36753 (issue: #36740)
- Fix wrong appliance of StackOverflow limit for IN #36724 (issue: #36592)
- Enhance checks for inexact fields #39427 (issue: #38501)
- Change the default precision for CURRENT_TIMESTAMP function #39391 (issue: #39288)
- Add "fuzziness" option to QUERY and MATCH function predicates #40529 (issue: #40495)
- Add "validate.properties" property to JDBC’s allowed list of settings #39050 (issue: #38068)
- Suggesters
- Task Management
- ZenDiscovery
-
- Introduce vote withdrawal #35446
- Add basic Zen1 transport-level BWC #35443
- Add diff-based publishing #35290
- Introduce auto_shrink_voting_configuration setting #35217
- Introduce transport API for cluster bootstrapping #34961
- Reconfigure cluster as its membership changes #34592 (issue: #33924)
- Fail fast on disconnects #34503
- Add storage-layer disruptions to CoordinatorTests #34347
- Add low-level bootstrap implementation #34345
- Gather votes from all nodes #34335
- Add Cluster State Applier #34257
- Add safety phase to CoordinatorTests #34241
- Integrate FollowerChecker with Coordinator #34075
- Integrate LeaderChecker with Coordinator #34049
- Trigger join when active master detected #34008
- Update PeerFinder term on term bump #33992
- Calculate optimal cluster configuration #33924
- Introduce FollowersChecker #33917
- Integrate publication pipeline into Coordinator #33771
- Add DisruptableMockTransport #33713
- Implement basic cluster formation #33668
- Introduce LeaderChecker #33024
- Add leader-side join handling logic #33013
- Add PeerFinder#onFoundPeersUpdated #32939
- Introduce PreVoteCollector #32847
- Introduce ElectionScheduler #32846
- Introduce ElectionScheduler #32709
- Add HandshakingTransportAddressConnector #32643 (issue: #32246)
- Add UnicastConfiguredHostsResolver #32642 (issue: #32246)
- Cluster state publication pipeline #32584 (issue: #32006)
- Introduce gossip-like discovery of master nodes #32246
- Add core coordination algorithm for cluster state publishing #32171 (issue: #32006)
- Add term and config to cluster state #32100 (issue: #32006)
- Add discovery types to cluster stats #36442
-
Introduce
zen2
discovery type #36298 - Persist cluster states the old way on non-master-eligible nodes #36247 (issue: #3)
- Storage layer WriteStateException propagation #36052
- Implement Tombstone REST APIs #36007
- Update default for USE_ZEN2 to true #35998
- Add warning if cluster fails to form fast enough #35993
- Allow Setting a List of Bootstrap Nodes to Wait for #35847
- VotingTombstone class #35832
- PersistedState interface implementation #35819
- Support rolling upgrades from Zen1 #35737
- Add lag detector #35685
- Move ClusterState fields to be persisted to ClusterState.MetaData #35625
- Introduce ClusterBootstrapService #35488
- Introduce vote withdrawal #35446
- Add basic Zen1 transport-level BWC #35443
- Add elasticsearch-node detach-cluster tool #37979
- Deprecate minimum_master_nodes #37868
- Step down as master when configured out of voting configuration #37802 (issue: #37712)
- Enforce cluster UUIDs #37775
- Bubble exceptions up in ClusterApplierService #37729
- Use m_m_nodes from Zen1 master for Zen2 bootstrap #37701
- Add tool elasticsearch-node unsafe-bootstrap #37696
- Report terms and version if cluster does not form #37473
- Bootstrap a Zen2 cluster once quorum is discovered #37463
- Zen2: Add join validation #37203
- Publish cluster states in chunks #36973
Bug fixes
edit- Aggregations
-
- Fix InternalAutoDateHistogram reproducible failure #32723 (issue: #32215)
- fix MultiValuesSourceFieldConfig toXContent #36525 (issue: #36474)
- Cache the score of the parent document in the nested agg #36019 (issues: #34555, #35985)
- Correct implemented interface of ParsedReverseNested #35455 (issue: #35449)
- Handle IndexOrDocValuesQuery in composite aggregation #35392
-
Don’t load global ordinals with the
map
execution_hint #37833 (issue: #37705) - Issue #37303 - Invalid variance fix #37384 (issue: #37303)
- Skip sibling pipeline aggregators reduction during non-final reduce #40101 (issue: #40059)
- Extend nextDoc to delegate to the wrapped doc-value iterator for date_nanos #39176 (issue: #39107)
- Only create MatrixStatsResults on final reduction #38130 (issue: #37587)
- Allocation
- Analysis
- Audit
- Authorization
- Authentication
-
- Fix kerberos setting registration #35986 (issues: #30241, #35942)
- Add support for Kerberos V5 Oid #35764 (issue: #34763)
- Enhance parsing of StatusCode in SAML Responses #38628
- Limit token expiry to 1 hour maximum #38244
- Fix expired token message in Exception header #37196
- Fix NPE in CachingUsernamePasswordRealm #36953 (issue: #36951)
- Allow non super users to create API keys #40028 (issue: #40029)
- Use consistent view of realms for authentication #38815 (issue: #30301)
- Correct authenticate response for API key #39684
- Fix security index auto-create and state recovery race #39582
- Build
- CCR
-
- Fix follow stats API’s follower index filtering feature #36647
- AutoFollowCoordinator should tolerate that auto follow patterns may be removed #35945 (issue: #35937)
- Only auto follow indices when all primary shards have started #35814 (issue: #35480)
- Avoid NPE in follower stats when no tasks metadata #35802
- Fix the names of CCR stats endpoints in usage API #35438
- Prevent CCR recovery from missing documents #38237
- Fix file reading in ccr restore service #38117
- Correct argument names in update mapping/settings from leader #38063
- Ensure changes requests return the latest mapping version #37633
- Do not set fatal exception when shard follow task is stopped. #37603
- Add fatal_exception field for ccr stats in monitoring mapping #37563
- Do not add index event listener if CCR disabled #37432
- When removing an AutoFollower also mark it as removed. #37402 (issue: #36761)
- Make shard follow tasks more resilient for restarts #37239 (issue: #37231)
- Resume follow Api should not require a request body #37217 (issue: #37022)
- Report error if auto follower tries auto follow a leader index with soft deletes disabled #36886 (issue: #33007)
- Remote cluster license checker and no license info. #36837 (issue: #36815)
- Make CCR resilient against missing remote cluster connections #36682 (issues: #36255, #36667)
- AutoFollowCoordinator and follower index already created #36540 (issue: #33007)
- Safe publication of AutoFollowCoordinator #40153 (issue: #38560)
- Enable reading auto-follow patterns from x-content #40130 (issue: #40128)
- Stop auto-followers on shutdown #40124
- Protect against the leader index being removed #39351 (issue: #39308)
-
Handle the fact that
ShardStats
instance may have no commit or seqno stats #38782 (issue: #38779) - Fix LocalIndexFollowingIT#testRemoveRemoteConnection() test #38709 (issue: #38695)
- Fix shard follow task startup error handling #39053 (issue: #38779)
- Filter out upgraded version index settings when starting index following #38838 (issue: #38835)
- Circuit Breakers
- Core
- CRUD
- Cluster Coordination
-
- Fix node tool cleanup #39389
- Avoid serialising state if it was already serialised #39179
- Do not perform cleanup if Manifest write fails with dirty exception #40519 (issue: #39077)
- Cache compressed cluster state size #39827 (issue: #39806)
- Drop node if asymmetrically partitioned from master #39598
- Fixing the custom object serialization bug in diffable utils. #39544
- Clean GatewayAllocator when stepping down as master #38885
- Distributed
-
- Combine the execution of an exclusive replica operation with primary term update #36116 (issue: #35850)
- ActiveShardCount should not fail when closing the index #35936
- TransportVerifyShardBeforeCloseAction should force a flush #38401 (issues: #33888, #37961)
- Fix limit on retaining sequence number #37992 (issue: #37165)
- Close Index API should force a flush if a sync is needed #37961 (issues: #33888, #37426)
- Force Refresh Listeners when Acquiring all Operation Permits #36835
- Replaced the word shards with replicas in an error message. (#36234) #36275 (issue: #36234)
- Ignore waitForActiveShards when syncing leases #39224 (issue: #39089)
- Fix synchronization in LocalCheckpointTracker#contains #38755 (issues: #33871, #38633)
- Enforce retention leases require soft deletes #39922 (issue: #39914)
- Treat TransportService stopped error as node is closing #39800 (issue: #39584)
- Use cause to determine if node with primary is closing #39723 (issue: #39584)
- Don’t ack if unable to remove failing replica #39584 (issue: #39467)
- Fix NPE on Stale Index in IndicesService #38891 (issue: #38845)
- Engine
-
- Set Lucene version upon index creation. #36038 (issue: #33826)
- Wrap can_match reader with ElasticsearchDirectoryReader #35857
- Copy checkpoint atomically when rolling generation #35407
- Subclass NIOFSDirectory instead of using FileSwitchDirectory #37140 (issues: #36668, #37111)
- Bubble up exception when processing NoOp #39338 (issue: #38898)
- ReadOnlyEngine should update translog recovery state information #39238
- Advance max_seq_no before add operation to Lucene #38879 (issue: #31629)
- Features/Features
- Features/ILM
-
- Preserve ILM operation mode when creating new lifecycles #38134 (issues: #38229, #38230)
- Retry ILM steps that fail due to SnapshotInProgressException #37624 (issues: #37541, #37552)
-
Remove
indexing_complete
when removing policy #36620 - Handle failure to release retention leases in ILM #39281 (issue: #39181)
- Correct ILM metadata minimum compatibility version #40569 (issue: #40565)
- Handle null retention leases in WaitForNoFollowersStep #40477
- Allow ILM to stop if indices have nonexistent policies #40820 (issue: #40824)
- Features/Indices APIs
-
- Validate top-level keys for create index request (#23755) #23869 (issue: #23755)
- Reject delete index requests with a body #37501 (issue: #8217)
- Fix duplicate phrase in shrink/split error message #36734 (issue: #36729)
- Get Aliases with wildcard exclusion expression #34230 (issues: #33518, #33805, #34144)
- Features/Ingest
-
- Fix Deprecation Warning in Script Proc. #32407
- Support unknown fields in ingest pipeline map configuration #38352 (issue: #36938)
- Ingest node - user_agent, move device parsing to an object #38115 (issues: #37329, #38094)
- Fix on_failure with Drop processor #36686 (issue: #36151)
- Support default pipelines + bulk upserts #36618 (issue: #36219)
- Ingest ingest then create index #39607 (issues: #32758, #32786, #36545)
- Features/Java High Level REST Client
-
- Drop extra level from user parser #34932
- Update IndexTemplateMetaData to allow unknown fields #38448 (issue: #36938)
-
if_seq_no
andif_primary_term
parameters aren’t wired correctly in REST Client’s CRUD API #38411 - Update Rollup Caps to allow unknown fields #38339 (issue: #36938)
- Fix ILM explain response to allow unknown fields #38054 (issue: #36938)
- Fix ILM status to allow unknown fields #38043 (issue: #36938)
- Fix ILM Lifecycle Policy to allow unknown fields #38041 (issue: #36938)
- Update authenticate to allow unknown fields #37713 (issue: #36938)
- Update verify repository to allow unknown fields #37619 (issue: #36938)
- Update get users to allow unknown fields #37593 (issue: #36938)
- Update Execute Watch to allow unknown fields #37498 (issue: #36938)
- Update Put Watch to allow unknown fields #37494 (issue: #36938)
- Update Delete Watch to allow unknown fields #37435 (issue: #36938)
- Fix rest reindex test for IPv4 addresses #37310
- Fix weighted_avg parser not found for RestHighLevelClient #37027 (issue: #36861)
- Features/Java Low Level REST Client
- Features/Monitoring
- Features/Watcher
-
- Ignore system locale/timezone in croneval CLI tool #33215
- Support merge nested Map in list for JIRA configurations #37634 (issue: #30068)
- Watcher accounts constructed lazily #36656
- Ensures watch definitions are valid json #30692 (issue: #29746)
- Use non-ILM template setting up watch history template & ILM disabled #39325 (issue: #38805)
- Only flush Watcher’s bulk processor if Watcher is enabled #38803 (issue: #38798)
- Fix Watcher stats class cast exception #39821 (issue: #39780)
- Use any index specified by .watches for Watcher #39541 (issue: #39478)
- Resolve concurrency with watcher trigger service #39092 (issue: #39087)
- Metric on watcher stats is a list not an enum #39114
- Geo
-
-
Test
GeoShapeQueryTests#testPointsOnly
fails #27454 - More robust handling of ignore_malformed in geoshape parsing #35603 (issues: #34047, #34498)
- Better handling of malformed geo_points #35554 (issue: #35419)
- Enables coerce support in WKT polygon parser #35414 (issue: #35059)
- Fix GeoHash PrefixTree BWC #38584 (issue: #38494)
- Do not normalize the longitude with value -180 for Lucene shapes #37299 (issue: #37297)
- Geo Point parse error fix #40447 (issue: #17617)
-
Test
- Highlighting
- Infra/Core
-
- Ensure shard is refreshed once it’s inactive #27559 (issue: #27500)
- Bubble-up exceptions from scheduler #38317 (issue: #38014)
- Revert back to joda’s multi date formatters #36814 (issues: #36447, #36602)
- Propagate Errors in executors to uncaught exception handler #36137 (issue: #28667)
- Correct name of basic_date_time_no_millis #39367
- Allow single digit milliseconds in strict date parsing #40676 (issue: #40403)
- Parse composite patterns using ClassicFormat.parseObject #40100 (issue: #39916)
- Bat scripts to work with JAVA_HOME with parantheses #39712 (issues: #30606, #33405, #38578, #38624)
- Change licence expiration date pattern #39681 (issue: #39136)
- Fix DateFormatters.parseMillis when no timezone is given #39100 (issue: #39067)
- Don’t close caches while there might still be in-flight requests. #38958 (issue: #37117)
- Allow single digit milliseconds in strict date parsing #40676 (issue: #40403)
- Infra/Packaging
-
- Remove NOREPLACE for /etc/elasticsearch in rpm and deb #37839
- Packaging: Update marker used to allow ELASTIC_PASSWORD #37243 (issue: #37240)
- Remove permission editing in postinst #37242 (issue: #37143)
- Some elasticsearch-cli tools could not be run not from ES_HOME #39937
- Obsolete pre 7.0 noarch package in rpm #39472 (issue: #39414)
-
Suppress error message when
/proc/sys/vm/max_map_count
is not exists. #35933 - Use TAR instead of DOCKER build type before 6.7.0 #40723 (issues: #39378, #40511)
- Source additional files correctly in elasticsearch-cli #40890 (issue: #40889)
- Infra/REST API
-
- Reject all requests that have an unconsumed body #37504 (issues: #30792, #37501, #8217)
- Fix #38623 remove xpack namespace REST API #38625
- Remove the "xpack" namespace from the REST API #38623
- Update spec files that erroneously documented parts as optional #39122
- ilm.explain_lifecycle documents human again #39113
- Index on rollup.rollup_search.json is a list #39097
- Infra/Scripting
- Infra/Settings
- Infra/Transport API
- Index APIs
- Ingest
- License
- Machine Learning
-
- Interrupt Grok in file structure finder timeout #36588
- Prevent stack overflow while copying ML jobs and datafeeds #36370 (issue: #36360)
- Adjust file structure finder parser config #35935
- Fix find_file_structure NPE with should_trim_fields #35465 (issue: #35462)
- Prevent notifications being created on deletion of a non existent job #35337 (issues: #34058, #35336)
- Clear Job#finished_time when it is opened (#32605) #32755
- Fix thread leak when waiting for job flush (#32196) #32541 (issue: #32196)
- Fix CPoissonMeanConjugate sampling error. #335
- Report index unavailable instead of waiting for lazy node #38423
- Fix error race condition on stop _all datafeeds and close _all jobs #38113 (issue: #37959)
- Update ML results mappings on process start #37706 (issue: #37607)
- Prevent submit after autodetect worker is stopped #37700 (issue: #37108)
- Fix ML datafeed CCS with wildcarded cluster name #37470 (issue: #36228)
- Update error message for process update #37363
- Wait for autodetect to be ready in the datafeed #37349 (issues: #36810, #37227)
- Stop datafeeds running when their jobs are stale #37227 (issue: #36810)
- Order GET job stats response by job id #36841 (issue: #36683)
- Make GetJobStats work with arbitrary wildcards and groups #36683 (issue: #34745)
- Fix datafeed skipping first bucket after lookback when aggs are … #39859 (issue: #39842)
- Refactoring lazy query and agg parsing #39776 (issue: #39528)
- Stop the ML memory tracker before closing node #39111 (issue: #37117)
- Scrolling datafeed should clear scroll contexts on error #40773 (issue: #40772)
- Mapping
-
- Ensure that field aliases cannot be used in multi-fields. #32219
-
Treat put-mapping calls with
_doc
as a top-level key as typed calls. #38032 - Correct deprec log in RestGetFieldMappingAction #37843 (issue: #37667)
- Restore a noop _all metadata field for 6x indices #37808 (issue: #37429)
- Make sure PutMappingRequest accepts content types other than JSON. #37720
- Make sure to use the resolved type in DocumentMapperService#extractMappings. #37451 (issue: #36811)
- Improve Precision for scaled_float #37169 (issue: #32570)
- Make sure to accept empty unnested mappings in create index requests. #37089
- Stop automatically nesting mappings in index creation requests. #36924
- Rewrite SourceToParse with resolved docType #36921 (issues: #35790, #36769)
-
Optimise rejection of out-of-range
long
values #40325 (issues: #26137, #40323) - Make sure to reject mappings with type _doc when include_type_name is false. #38270 (issue: #38266)
- Network
-
- Adjust SSLDriver behavior for JDK11 changes #32145 (issues: #32122, #32144)
- Netty4SizeHeaderFrameDecoder error #31057
- Fix memory leak in http pipelining #30815 (issue: #30801)
- Fix issue with finishing handshake in ssl driver #30580
- Do not resolve addresses in remote connection info #36671 (issue: #35658)
- Always compress based on the settings #36522 (issue: #36399)
- http.publish_host Should Contain CNAME #32806 (issue: #22029)
- Adjust SSLDriver behavior for JDK11 changes #32145 (issues: #32122, #32144)
- Add TRACE, CONNECT, and PATCH http methods #31035 (issue: #31017)
- Transport client: Don’t validate node in handshake #30737 (issue: #30141)
- Fix issue with finishing handshake in ssl driver #30580
- Remove potential nio selector leak #27825
- Fix issue where the incorrect buffers are written #27695 (issue: #27551)
- Do not set SO_LINGER on server channels #26997
- Do not set SO_LINGER to 0 when not shutting down #26871 (issue: #26764)
- Release pipelined http responses on close #26226
- Reload SSL context on file change for LDAP #36937 (issues: #30509, #36923)
- Do not resolve addresses in remote connection info #36671 (issue: #35658)
- Packaging
- Plugins
- Ranking
-
- QueryRescorer should keep the window size when rewriting #36836
- Recovery
-
- Register ResyncTask.Status as a NamedWriteable #36610
- RecoveryMonitor#lastSeenAccessTime should be volatile #36781
- Create retention leases file during recovery #39359 (issue: #37165)
- Recover peers from translog, ignoring soft deletes #38904 (issue: #37165)
- Retain history for peer recovery using leases #38855
- Resync should not send operations without sequence number #40433
- Rollup
- Scripting
- Settings
- Search
-
-
Ensure realtime
_get
and_termvectors
don’t run on the network thread #33814 (issue: #27500) - [bug] fuzziness custom auto #33462 (issue: #33454)
- Fix inner hits retrieval when stored fields are disabled (none) #33018 (issue: #32941)
- Set maxScore for empty TopDocs to Nan rather than 0 #32938
- Handle leniency for cross_fields type in multi_match query #27045 (issue: #23210)
- Raise IllegalArgumentException instead if query validation failed #26811 (issue: #26799)
- Inner hits fail to propagate doc-value format. #36310
- Fix custom AUTO issue with Fuzziness#toXContent #35807 (issue: #33462)
- Fix analyzed prefix query in query_string #35756 (issue: #31702)
- Fix problem with MatchNoDocsQuery in disjunction queries #35726 (issue: #34708)
- Fix phrase_slop in query_string query #35533 (issue: #35125)
- Add a More Like This query routing requirement check (#29678) #33974
- Look up connection using the right cluster alias when releasing contexts #38570
- Fix fetch source option in expand search phase #37908 (issue: #23829)
-
Change
rational
tosaturation
in script_score #37766 (issue: #37714) - Throw if two inner_hits have the same name #37645 (issue: #37584)
- Ensure either success or failure path for SearchOperationListener is called #37467 (issue: #37185)
-
query_string
should use indexed prefixes #36895 - Avoid duplicate types deprecation messages in search-related APIs. #36802
- Serialize top-level pipeline aggs as part of InternalAggregations #40177 (issues: #40059, #40101)
- CCS: Skip empty search hits when minimizing round-trips #40098 (issues: #32125, #40067)
- CCS: Disable minimizing round-trips when dfs is requested #40044 (issue: #32125)
- Fix Fuzziness#asDistance(String) #39643 (issue: #39614)
- Fix alias resolution runtime complexity. #40263 (issue: #40248)
-
Ensure realtime
- Security
-
- Handle 6.4.0+ BWC for Application Privileges #32929
- Remove license state listeners on closeables #36308 (issues: #33328, #35627, #35628)
- Fix exit code for Security CLI tools #37956 (issue: #37841)
- Fix potential NPE in UsersTool #37660
- Remove dynamic objects from security index #40499 (issue: #35460)
- Fix libs:ssl-config project setup #39074
- Do not create the missing index when invoking getRole #39039
- Snapshot/Restore
-
- Upgrade GCS Dependencies to 1.55.0 #36634 (issues: #35229, #35459)
- Improve Resilience SnapshotShardService #36113 (issue: #32265)
- Keep SnapshotsInProgress State in Sync with Routing Table #35710
- Ensure that gcs client creation is privileged #25938 (issue: #25932)
- Make calls to CloudBlobContainer#exists privileged #25937 (issue: #25931)
- Fix Concurrent Snapshot Ending And Stabilize Snapshot Finalization #38368 (issue: #38226)
- Fix Two Races that Lead to Stuck Snapshots #37686 (issues: #32265, #32348)
- Fix Race in Concurrent Snapshot Delete and Create #37612 (issue: #37581)
- Streamline S3 Repository- and Client-Settings #37393
- Blob store compression fix #39073
- SQL
-
- Fix translation of LIKE/RLIKE keywords #36672 (issues: #36039, #36584)
- Scripting support for casting functions CAST and CONVERT #36640 (issue: #36061)
- Fix translation to painless for conditionals #36636 (issue: #36631)
- Concat should be always not nullable #36601 (issue: #36169)
- Fix MOD() for long and integer arguments #36599 (issue: #36364)
- Fix issue with complex HAVING and GROUP BY ordinal #36594 (issue: #36059)
- Be lenient for tests involving comparison to H2 but strict for csv spec tests #36498 (issue: #36483)
- Non ISO 8601 versions of DAY_OF_WEEK and WEEK_OF_YEAR functions #36358 (issue: #36263)
- Do not ignore all fields whose names start with underscore #36214 (issue: #36206)
- Fix issue with wrong data type for scripted Grouping keys #35969 (issue: #35662)
- Fix translation of math functions to painless #35910 (issue: #35654)
- Fix jdbc jar to include deps #35602
- Fix query translation for scripted queries #35408 (issue: #35232)
- Clear the cursor if nested inner hits are enough to fulfill the query required limits #35398 (issue: #35176)
- Introduce IsNull node to simplify expressions #35206 (issues: #34876, #35171)
- The SSL default configuration shouldn’t override the https protocol if used #34635 (issue: #33817)
- Minor fix for javadoc #32573 (issue: #32553)
- Prevent grouping over grouping functions #38649 (issue: #38308)
- Relax StackOverflow circuit breaker for constants #38572 (issue: #38571)
- Fix issue with IN not resolving to underlying keyword field #38440 (issue: #38424)
- Change the Intervals milliseconds precision to 3 digits #38297 (issue: #37423)
- Fix esType for DATETIME/DATE and INTERVALS #38179 (issue: #38051)
- Added SSL configuration options tests #37875 (issue: #37711)
- Fix casting from date to numeric type to use millis #37869 (issue: #37655)
- Fix BasicFormatter NPE #37804
- Return Intervals in SQL format for CLI #37602 (issues: #29970, #36186, #36432)
- Fix object extraction from sources #37502 (issue: #37364)
- Fix issue with field names containing "." #37364 (issue: #37128)
- Fix bug regarding alias fields with dots #37279 (issue: #37224)
- Proper handling of COUNT(field_name) and COUNT(DISTINCT field_name) #37254 (issue: #30285)
- Fix COUNT DISTINCT filtering #37176 (issue: #37086)
- Fix issue with wrong NULL optimization #37124 (issue: #35872)
- Fix issue with complex expression as args of PERCENTILE/_RANK #37102 (issue: #37099)
- Handle the bwc Joda ZonedDateTime scripting class in Painless #37024 (issue: #37023)
- Fix bug regarding histograms usage in scripting #36866
- Fix issue with always false filter involving functions #36830 (issue: #35980)
- Protocol returns ISO 8601 String formatted dates instead of Long for JDBC/ODBC requests #36800 (issue: #36756)
- Enhance Verifier to prevent aggregate or grouping functions from #36799 (issue: #36798)
- Fix translation of LIKE/RLIKE keywords #36672 (issues: #36039, #36584)
- Scripting support for casting functions CAST and CONVERT #36640 (issue: #36061)
- Concat should be always not nullable #36601 (issue: #36169)
- Fix issue with complex HAVING and GROUP BY ordinal #36594 (issue: #36059)
- Add missing handling of IP field in JDBC #40384 (issue: #40358)
- Fix metric aggs on date/time to not return double #40377 (issues: #39492, #40376)
- CAST supports both SQL and ES types #40365 (issue: #40282)
- Fix RLIKE bug and improve testing for RLIKE statement #40354 (issues: #34609, #39931)
- Unwrap the first value in an array in case of array leniency #40318 (issue: #40296)
- Preserve original source for cast/convert function #40271 (issue: #40239)
- Fix LIKE function equality by considering its pattern as well #40260 (issue: #39931)
- Fix issue with optimization on queries with ORDER BY/LIMIT #40256 (issue: #40211)
- Rewrite ROUND and TRUNCATE functions with a different optional parameter handling method #40242 (issue: #40001)
- Fix issue with getting DATE type in JDBC #40207
- Fix issue with date columns returned always in UTC #40163 (issue: #40152)
- Add multi_value_field_leniency inside FieldHitExtractor #40113 (issue: #39700)
- Fix incorrect ordering of groupings (GROUP BY) based on orderings (ORDER BY) #40087 (issue: #39956)
- Fix bug with JDBC timezone setting and DATE type #39978 (issue: #39915)
- Use underlying exact field for LIKE/RLIKE #39443 (issue: #39442)
- Fix display size for DATE/DATETIME #40669
- Have LIKE/RLIKE use wildcard and regexp queries #40628 (issue: #40557)
- Fix getTime() methods in JDBC #40484
- SYS TABLES: enumerate tables of requested types #40535 (issue: #40348)
- Passing an input to the CLI "freezes" the CLI after displaying an error message #40164 (issue: #40557)
- Wrap ZonedDateTime parameters inside scripts #39911 (issue: #39877)
- ConstantProcessor can now handle NamedWriteable #39876 (issue: #39875)
- Extend the multi dot field notation extraction to lists of values #39823 (issue: #39738)
- Values in datetime script aggs should be treated as long #39773 (issue: #37042)
- Don’t allow inexact fields for MIN/MAX #39563 (issue: #39427)
- Fix merging of incompatible multi-fields #39560 (issue: #39547)
- Fix COUNT DISTINCT column name #39537 (issue: #39511)
- Enable accurate hit tracking on demand #39527 (issue: #37971)
- Ignore UNSUPPORTED fields for JDBC and ODBC modes in SYS COLUMNS #39518 (issue: #39471)
- Enforce JDBC driver - ES server version parity #38972 (issue: #38775)
- Fall back to using the field name for column label #38842 (issue: #38831)
- Suggesters
- Task Management
-
- Un-assign persistent tasks as nodes exit the cluster #37656
- Watcher
- ZenDiscovery
-
- Remove duplicate discovered peers #35505
- Respect the no_master_block setting #36478
- Cancel GetDiscoveredNodesAction when bootstrapped #36423 (issues: #36380, #36381)
- Only elect master-eligible nodes #35996
- Remove duplicate discovered peers #35505
- Fix size of rolling-upgrade bootstrap config #38031
- Always return metadata version if metadata is requested #37674
- Elect freshest master in upgrade #37122 (issue: #40)
- Fix cluster state persistence for single-node discovery #36825
Regressions
edit- Infra/Core
- Mapping
- Scripting