Elasticsearch version 8.0.0-alpha1

edit

Elasticsearch version 8.0.0-alpha1

edit

Also see Breaking changes in 8.0.

Breaking changes

edit
Aggregations
  • Percentiles aggregation: disallow specifying same percentile values twice #52257 (issue: #51871)
  • Remove Adjacency_matrix setting #46327 (issues: #46257, #46324)
  • Remove MovingAverage pipeline aggregation #39328
  • Remove deprecated _time and _term sort orders #39450
  • Remove deprecated date histo interval #75000
Allocation
Analysis
Authentication
  • Always add file and native realms unless explicitly disabled #69096 (issue: #50892)
  • Do not set a NameID format in Policy by default #44090 (issue: #40353)
  • Make order setting mandatory for Realm config #51195 (issue: #37614)
CCR
  • Avoid auto following leader system indices in CCR #72815 (issue: #67686)
Cluster Coordination
Distributed
Engine
  • Force merge should reject requests with only_expunge_deletes and max_num_segments set #44761 (issue: #43102)
  • Remove per-type indexing stats #47203 (issue: #41059)
  • Remove translog retention settings #51697 (issue: #50775)
Features/CAT APIs
  • Remove the deprecated local parameter for _cat/indices #64868 (issue: #62198)
  • Remove the deprecated local parameter for _cat/shards #64867 (issue: #62197)
Features/Features
  • Remove deprecated ._tier allocation filtering settings #73074 (issue: #72835)
Features/ILM+SLM
Features/Indices APIs
  • Change prefer_v2_templates parameter to default to true #55489 (issues: #53101, #55411)
  • Remove deprecated _upgrade API #64732 (issue: #21337)
  • Remove local parameter for get field mapping request #55100 (issue: #55099)
  • Remove include_type_name parameter from REST layer #48632 (issue: #41059)
  • Remove the template field in index templates #49460 (issue: #21009)
Features/Watcher
  • Move watcher history to data stream #64252
Geo
  • Disallow creating geo_shape mappings with deprecated parameters #70850 (issue: #32039)
  • Remove bounding box query type parameter #74536
Infra/Circuit Breakers
  • Fixed synchronizing inflight breaker with internal variable #40878
Infra/Core
Infra/Logging
Infra/Plugins
  • Remove deprecated basic license feature enablement settings from 8.0 #56211 (issue: #54745)
Infra/REST API
  • Remove content type required setting #61043
  • Remove deprecated endpoints containing _xpack #48170 (issue: #35958)
  • Remove deprecated endpoints of hot threads API #55109 (issue: #52640)
  • Allow parsing Content-Type and Accept headers with version #61427
Infra/Resiliency
Infra/Scripting
Infra/Settings
  • Change default value of action.destructive_requires_name to true #66908 (issue: #61074)
  • Forbid settings without a namespace #45947 (issues: #45905, #45940)
Machine Learning
Mapping
  • Remove boost mapping parameter #62639 (issue: #62623)
  • Remove support for chained multi-fields #42333 (issues: #41267, #41926)
  • Remove support for string in unmapped_type #45675
  • Removes typed URLs from mapping APIs #41676
Network
Packaging
Recovery
  • Remove dangling index auto import functionality #59698 (issue: #48366)
Reindex
Rollup
  • RollupStart endpoint should return OK if job already started #41502 (issues: #35928, #39845)
Search
  • Decouple shard allocation awareness from search and get requests #45735 (issue: #43453)
  • Fix range query on date fields for number inputs #63692 (issue: #63680)
  • Make fuzziness reject illegal values earlier #33511
  • Make remote cluster resolution stricter #40419 (issue: #37863)
  • Parse empty first line in msearch request body as action metadata #41011 (issue: #39841)
  • Remove CommonTermsQuery and cutoff_frequency param #42654 (issue: #37096)
  • Remove type query #47207 (issue: #41059)
  • Remove use_field_mapping format option for docvalue fields #55622
  • Remove deprecated SimpleQueryStringBuilder parameters #57200
  • Remove deprecated search.remote settings #42381 (issues: #33413, #38556)
  • Remove deprecated sort options: nested_path and nested_filter #42809 (issue: #27098)
  • Remove deprecated vector functions #48725 (issue: #48604)
  • Remove support for _type in searches #68564 (issues: #41059, #68311)
  • Remove support for sparse vectors #48781 (issue: #48368)
  • Remove the object format for indices_boost #55078
  • Removes type from TermVectors APIs #42198 (issue: #41059)
  • Removes typed endpoint from search and related APIs #41640
  • Set max allowed size for stored async response #74455 (issue: #67594)
  • indices.query.bool.max_clause_count now limits all query clauses #75297
Security
  • Remove obsolete security settings #40496
  • Remove support of creating CA on the fly when generating certificates #65590 (issue: #61884)
  • Remove the id field from the InvalidateApiKey API #66671 (issue: #66317)
  • Remove the migrate tool #42174
  • Compress audit logs #64472 (issue: #63843)
  • Remove insecure settings #46147 (issue: #45947)
Snapshot/Restore
  • Blob store compress default to true #40033
  • Get snapshots support for multiple repositories #42090 (issue: #41210)
  • Remove repository stats API #62309 (issue: #62297)
  • Remove frozen cache setting leniency #71013 (issue: #70341)
TLS
  • Reject misconfigured/ambiguous SSL server config #45892
  • Remove support for configurable PKCS#11 keystores #75404
  • Remove the client transport profile filter #43236

Breaking Java changes

edit
Authentication
  • Mandate x-pack REST handler installed #71061 (issue: #70523)
CCR
CRUD
  • Remove types from BulkRequest #46983 (issue: #41059)
  • Remove Client.prepareIndex(index, type, id) method #48443
  • Remove deprecated include-type methods from HLRC indices client #48471
Client
  • Remove SecurityClient from x-pack #42471
Features/ILM+SLM
Features/Monitoring
  • Remove MonitoringClient from x-pack #42770
Features/Watcher
  • Remove WatcherClient from x-pack #42815
Infra/Core
  • Remove XPackClient from x-pack #42729
  • Remove the transport client #42538
  • Remove transport client from x-pack #42202
Infra/REST API
  • Copy HTTP headers to ThreadContext strictly #45945
Machine Learning
  • Remove the MachineLearningClient #43108
Mapping
  • Remove type filter from GetMappings API #47364 (issue: #41059)
  • Remove type parameter from PutMappingRequest.buildFromSimplifiedDef() #50844 (issue: #41059)
  • Remove unused parameter from MetadataFieldMapper.TypeParser#getDefault() #51219
  • Remove type parameter from CIR.mapping(type, object...) #50739 (issue: #41059)
Search
  • Removes types from SearchRequest and QueryShardContext #42112
Snapshot/Restore
  • Remove deprecated repository methods #42359 (issue: #42213)

Enhancements

edit
Analysis
  • Move reload_analyzers endpoint to x-pack #43559
Authentication
EQL
  • Add option for returning results from the tail of the stream #64869 (issue: #58646)
  • Introduce case insensitive variant in~ #68176 (issue: #68172)
  • Optimize redundant toString #71070 (issue: #70681)
Engine
  • Always use soft-deletes in InternalEngine #50415
  • Remove translog retention policy #51417 (issue: #50775)
Features/CAT APIs
  • Remove size and add time params to _cat/threadpool #55736 (issue: #54478)
Features/Stats
Features/Watcher
  • Remove Watcher history clean up from monitoring #67154
Infra/Core
  • Remove aliases exist action #43430
  • Remove indices exists action #43164
  • Remove types exists action #43344
Infra/Logging
  • Make Elasticsearch JSON logs ECS compliant #47105 (issue: #46119)
Infra/REST API
  • Allow for field declaration for future compatible versions #69774 (issue: #51816)
  • Introduce stability description to the REST API specification #38413
  • Parsing: Validate that fields are not registered twice #70243
  • Support response content-type with versioned media type #65500 (issue: #51816)
  • [REST API Compatibility] Typed endpoints for index and get APIs #69131 (issue: #54160)
  • [REST API Compatibility] Typed endpoints for put and get mapping and get field mappings #71721 (issues: #51816, #54160)
  • [REST API Compatibility] Allow copy_settings flag for resize operations #75184 (issues: #38514, #51816)
  • [REST API Compatibility] Allow for type in geo shape query #74553 (issues: #51816, #54160)
  • [REST API Compatibility] Always return adjust_pure_negative value #75182 (issues: #49543, #51816)
  • [REST API Compatibility] Clean up x-pack/plugin rest compat tests #74701 (issue: #51816)
  • [REST API Compatibility] Do not return _doc for empty mappings in template #75448 (issues: #51816, #54160, #70966, #74544)
  • [REST API Compatibility] Dummy REST action for indices.upgrade API #75136 (issue: #51816)
  • [REST API Compatibility] REST Terms vector typed response #73117
  • [REST API Compatibility] Rename BulkItemResponse.Failure type field #74937 (issue: #51816)
  • [REST API Compatibility] Type metadata for docs used in simulate request #74222 (issues: #51816, #54160)
  • [REST API Compatibility] Typed TermLookups #74544 (issues: #46943, #51816, #54160)
  • [REST API Compatibility] Typed and x-pack graph explore API #74185 (issues: #46935, #51816, #54160)
  • [REST API Compatibility] Typed endpoint for bulk API #73571 (issue: #51816)
  • [REST API Compatibility] Typed endpoint for multi-get API #73878 (issue: #51816)
  • [REST API Compatibility] Typed endpoints for RestUpdateAction and RestDeleteAction #73115 (issues: #51816, #54160)
  • [REST API Compatibility] Typed endpoints for get_source API #73957 (issues: #46587, #46931, #51816)
  • [REST API Compatibility] Typed endpoints for explain API #73901 (issue: #51816)
  • [REST API Compatibility] Typed endpoints for search _count API #73958 (issues: #42112, #51816)
  • [REST API Compatibility] Typed indexing stats #74181 (issues: #47203, #51816, #54160)
  • [REST API Compatibility] Types for percolate query API #74698 (issues: #46985, #51816, #54160, #74689)
  • [REST API Compatibility] Validate query typed API #74171 (issues: #46927, #51816, #54160)
  • [REST API Compatibility] Voting config exclusion exception message #75406 (issues: #51816, #55291)
  • [REST API Compatibility] MoreLikeThisQuery with types #75123 (issues: #42198, #51816, #54160)
  • [REST API Compatibility] Update and delete by query using size field #69606
  • [REST API Compatibility] Indicies boost in object format #74422 (issues: #51816, #55078)
  • [REST API Compatibility] Typed endpoints for search and related endpoints #72155 (issues: #51816, #54160)
  • [REST API Compatibility] Allow to use size -1 #75342 (issues: #51816, #69548, #70209)
  • [REST API Compatibility] Ignore use_field_mapping option for docvalue #74435 (issue: #55622)
  • [REST API Compatibility] _time and _term sort orders #74919 (issues: #39450, #51816)
  • [REST API Compatability] template parameter and field on PUT index template #71238 (issues: #49460, #51816, #68905)
  • [REST API Compatibility] Make query registration easier #75722 (issue: #51816)
  • [REST API Compatibility] Typed query #75453 (issues: #47207, #51816, #54160)
  • [REST API Compatibility] Deprecate the use of synced flush #75372 (issues: #50882, #51816)
  • [REST API Compatibility] Licence accept_enterprise and response changes #75479 (issues: #50067, #50735, #51816, #58217)
Infra/Scripting
  • Update DeprecationMap to DynamicMap #56149 (issue: #52103)
Infra/Settings
  • Fixed inconsistent Setting.exist() #46603 (issue: #41830)
  • Remove index.optimize_auto_generated_id setting (#27583) #27600 (issue: #27583)
License
  • Add deprecated accept_enterprise param to /_xpack #58220 (issue: #58217)
  • Support accept_enterprise param in get license API #50067 (issue: #49474)
Machine Learning
  • The Windows build platform for the machine learning C++ code now uses Visual Studio 2019 #1352
  • The macOS build platform for the machine learning C++ code is now Mojave running Xcode 11.3.1, or Ubuntu 20.04 running clang 8 for cross compilation #1429
  • The Linux build platform for the machine learning C++ code is now CentOS 7 running gcc 9.3 #1170
  • Add a new application for evaluating PyTorch models. The app depends on LibTorch - the C++ front end to PyTorch - and performs inference on models stored in the TorchScript format #1902
Mapping
  • Sparse vector to throw exception consistently #62646
Packaging
  • Make the Docker build more re-usable in Cloud #50277 (issues: #46166, #49926)
  • Update docker-compose.yml to fix bootstrap check error #47650
Recovery
  • Use Lucene index in peer recovery and resync #51189 (issue: #50775)
Reindex
Rollup
  • Adds support for date_nanos in Rollup Metric and DateHistogram Configs #59349 (issue: #44505)
SQL
  • Add text formatting support for multivalue #68606
  • Add xDBC and CLI support. QA CSV specs #68966
  • Export array values through result sets #69512
  • Improve alias resolution in sub-queries #67216 (issue: #56713)
  • Improve the optimization of null conditionals #71192
  • Push WHERE clause inside subqueries #71362
  • Use Java String methods for LTRIM/RTRIM #57594
  • QL: Make canonical form take into account children #71266
  • QL: Polish optimizer expression rule declaration #71396
  • QL: Propagate nullability constraints across conjunctions #71187 (issue: #70683)
Search
  • Completely disallow setting negative size in search #70209 (issue: #69548)
  • Make 0 as invalid value for min_children in has_child query #41347
  • Return error when remote indices are locally resolved #74556 (issue: #26247)
Security
  • Add a tool for creating enrollment tokens #74890
  • Add the Enroll Kibana API #72207
  • Change default hashing algorithm for FIPS 140 #55544
  • Create enrollment token #73573 (issues: #71438, #72129)
  • Enroll node API #72129
  • Not encoding the Api Key in Enrollment token #74510 (issue: #73573)
  • Configure security for the initial node CLI #74868
Snapshot/Restore
  • Introduce searchable snapshots index setting for cascade deletion of snapshots #74977
  • Unify blob store compress setting #39346 (issue: #39073)
  • Add recovery state tracking for searchable snapshots #60505
TLS
  • Add ChaCha20 TLS ciphers on Java 12+ #42155
  • Add support for KeyStore filters to ssl-config #75407
  • Update TLS ciphers and protocols for JDK 11 #41808 (issues: #38646, #41385)

Bug fixes

edit
Aggregations
  • Fix BWC issues for x_pack/usage #55181 (issue: #54847)
  • Fix DoubleBounds null serialization #59475
  • Fix TopHitsAggregationBuilder adding duplicate _score sort clauses #42179 (issue: #42154)
  • Fix t_test usage stats #54753 (issue: #54744)
  • Throw exception if legacy interval cannot be parsed in DateIntervalWrapper #41972 (issue: #41970)
CCR
Cluster Coordination
  • Apply cluster states in system context #53785 (issue: #53751)
Distributed
  • Introduce ?wait_for_active_shards=index-setting #67158 (issue: #66419)
  • Respect CloseIndexRequest#waitForActiveShards in HLRC #67374 (issues: #67158, #67246)
  • Fixes to task result index mapping #50359 (issue: #50248)
Features/CAT APIs
  • Fix cat recovery display of bytes fields #40379 (issue: #40335)
Features/Java High Level REST Client
Geo
  • Preprocess polygon rings before processing it for decomposition #59501 (issues: #54441, #59386)
Infra/Core
  • Add searchable snapshot cache folder to NodeEnvironment #66297 (issue: #65725)
  • CLI tools: Write errors to stderr instead of stdout #45586 (issue: #43260)
  • Precompute ParsedMediaType for XContentType #67409
Infra/Logging
  • Fix NPE when logging null values in JSON #53715 (issue: #46702)
  • Fix stats in slow logs to be a escaped JSON #44642
  • Populate data stream fields when xOpaqueId not provided #62156
Infra/REST API
Infra/Scripting
  • Change compound assignment structure to support string concatenation #61825
  • Fixes casting in constant folding #61508
  • Several minor Painless fixes #61594
Machine Learning
  • Handle null value of FieldCapabilitiesResponse #64327
Mapping
  • Remove assertions that mappings have one top-level key #58779 (issue: #58521)
Packaging
  • Suppress illegal access in plugin install #41620 (issue: #41478)
SQL
  • Introduce dedicated node for HAVING declaration #71279 (issue: #69758)
  • Make RestSqlQueryAction thread-safe #69901
Search
  • Check for negative from values in search request body #54953 (issue: #54897)
  • Fix VectorsFeatureSetUsage serialization in BWC mode #55399 (issue: #55378)
  • Handle total hits equal to track_total_hits #37907 (issue: #37897)
  • Improve error msg for CCS request on node without remote cluster role #60351 (issue: #59683)
Snapshot/Restore
  • Fix GET /_snapshot/_all/_all if there are no repos #43558 (issue: #43547)

Upgrades

edit
Lucene
  • Upgrade to Lucene 8.9.0 #74729