Elasticsearch version 8.0.0-beta1

edit

Elasticsearch version 8.0.0-beta1

edit

Also see Breaking changes in 8.0.

Known issues

edit
  • If you’re using machine learning, it’s not safe to upgrade to 8.0.0-beta1 if the cluster you’re upgrading was first used prior to 7.7.0. If you attempt such an upgrade the filtered aliases against the machine learning results indices will lose their filters, causing machine learning anomaly detection results for different jobs to be mixed. Wait for 8.0.0-rc1 before attempting to test upgrading a cluster where machine learning was used prior to version 7.7.0.
  • System indices are included in wildcard queries when using the * pattern. Do not rely on this behavior, because system indices will be hidden in the 8.0 release. If you need to access system indices, specify the index name, use a more specific wildcard, or use the expand_wildcards parameter in your query.
  • By default, system indices are included in snapshots of all indices, and are restored when no indices are specified. This behavior results from including system indices in the implicit default wildcard (*) for snapshot and restore operations, which was intended for backwards compatibility in 7.x versions. In 8.0, all system indices should be accessed through the feature_states parameter of the snapshot or restore request.

Breaking changes

edit
ILM+SLM
  • Always enforce default tier preference #79751 (issue: #76147)
  • Validate that snapshot repository exists for ILM policies at creation/update time #78468 (issues: #72957, #77657)
  • Default ENFORCE_DEFAULT_TIER_PREFERENCE to true #79275 (issues: #76147, #79210)
Indices APIs
Infra/Core
  • Remove Joda dependency #79007
  • Remove Joda support from date formatters #78990
Ingest
  • Remove default maxmind GeoIP databases from distribution #78362 (issue: #68920)
License
  • Enforce license expiration #79671
Machine Learning
  • Remove allow_no_datafeeds and allow_no_jobs parameters from APIs #80048 (issue: #60732)
Packaging
  • Require Java 17 for running Elasticsearch #79873

Deprecations

edit
Authentication
  • Deprecate setup-passwords tool #76902
CRUD
  • Remove indices_segments verbose parameter #78451 (issue: #75955)
Monitoring
  • Add deprecation info API entries for deprecated monitoring settings #78799
  • Automatically install monitoring templates at plugin initialization #78350
  • Remove Monitoring ingest pipelines #77459 (issue: #50770)

New features

edit
Security

Enhancements

edit
Authentication
  • New CLI tool to reset password for built-in users #79709
  • Upgrade to UnboundID LDAP SDK v6.0.2 #79332
  • Auto-configure the elastic user password #78306
Cluster Coordination
Data streams
  • Data stream support read and write with custom routing and partition size #74394 (issue: #74390)
ILM+SLM
  • Inject migrate action regardless of allocate action #79090 (issue: #76147)
Infra/Core
  • Check whether stdout is a real console #79882
  • Share int, long, float, double, and byte pages #75053
  • Revert "Deprecate resolution loss on date field (#78921)" #79914 (issue: #78921)
  • Add two missing entries to the deprecation information API #80290 (issue: #80233)
Infra/Scripting
Ingest
  • Remove binary field after attachment processor execution #79172
  • Improving cache lookup to reduce recomputing / searches #77259
License
  • Enforce Transport TLS check on all licenses #79602 (issue: #75292)
Machine Learning
  • Add inference time configuration overrides #78441 (issue: #77799)
  • Optimize source extraction for categorize_text aggregation #79099
  • The Linux build platform for the machine learning C++ code is now CentOS 7 running gcc 10.3. #2028
  • Make ML indices hidden when the node becomes master #77416 (issue: #53674)
Mapping
  • Add support for configuring HNSW parameters #79193 (issue: #78473)
  • Extend dense_vector to support indexing vectors #78491 (issue: #78473)
Monitoring
  • Add previously removed Monitoring settings back for 8.0 #78784
  • Change Monitoring plugin cluster alerts to not install by default #79657
Packaging
  • Allow total memory to be overridden #78750 (issue: #65905)
Search
  • Node level can match action #78765
  • TSDB: Add time series information to field caps #78790 (issue: #74660)
  • Add new kNN search endpoint #79013 (issue: #78473)
  • Disallow kNN searches on nested vector fields #79403 (issue: #78473)
  • Ensure kNN search respects authorization #79693 (issue: #78473)
  • Load kNN vectors format with mmapfs #78724 (issue: #78473)
  • Support cosine similarity in kNN search #79500
  • Node level can match action #78765
Security
  • Add v7 restCompat for invalidating API key with the id field #78664 (issue: #66671)
  • Print enrollment token on startup #78293
  • Startup check for security implicit behavior change #76879
  • Update auto-generated credentials output #79755 (issue: #79312)
  • CLI tool to reconfigure nodes to enroll #79690 (issue: #7718)
  • Security auto-configuration for packaged installations #75144 (issue: #78306)
  • Update to OpenSAML 4 #77012 (issue: #71983)
Snapshot/Restore
  • Allow listing older repositories #78244
  • Optimize SLM Policy Queries #79341 (issue: #79321)
  • Upgrade repository-hdfs plugin to Hadoop 3 #76897
Transform
  • Prevent old beta transforms from starting #79712
TSDB
  • Automatically add timestamp mapper #79136
  • Create a coordinating node level reader for tsdb #79197
  • Fix TSDB shrink test in multi-version cluster #79940 (issue: #79936)
  • Do not allow shadowing metrics or dimensions #79757

Bug fixes

edit
Infra/Core
  • Prevent stack overflow in rounding #80450
Infra/Settings
  • Stricter UpdateSettingsRequest parsing on the REST layer #79227 (issue: #29268)
  • Set Auto expand replica on deprecation log data stream #79226 (issue: #78991)
Machine Learning
  • Add timeout parameter for delete trained models API #79739 (issue: #77070)
  • Fix MlMetadata backwards compatibility bug with 7.13 through 7.16 #80041
  • Tone down ML unassigned job notifications #79578 (issue: #79270)
  • Use a new annotations index for future annotations #79006 (issue: #78439)
Search
  • Remove unsafe assertion in wildcard field #78966
Snapshot/Restore
  • Don’t fill stack traces in SnapshotShardFailure #80009 (issue: #79718)

Regressions

edit
Search
  • Disable numeric sort optimization conditionally #78103

Upgrades

edit
Search