IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Elasticsearch version 7.8.1 Elasticsearch version 7.7.1 »
Elastic Docs ›Elasticsearch Guide [7.16] ›Release notes

Elasticsearch version 7.8.0

edit

Elasticsearch version 7.8.0

edit

Also see Breaking changes in 7.8.

Known issues

edit

  • SQL: If a WHERE clause contains at least two relational operators joined by AND, of which one is a comparison (<=, <, >=, >) and another one is an inequality (!=, <>), both against literals or foldable expressions, the inequality will be ignored. The workaround is to substitute the inequality with a NOT IN operator.

    We have fixed this issue in Elasticsearch 7.10.1 and later versions. For more details, see #65488.

Breaking changes

edit
Aggregations
  • value_count aggregation optimization #54854
Features/Indices APIs
  • Add auto create action #55858
Mapping
  • Disallow changing enabled on the root mapper #54463 (issue: #33933)
  • Fix updating include_in_parent/include_in_root of nested field #54386 (issue: #53792)

Deprecations

edit
Authentication
  • Deprecate the kibana reserved user; introduce kibana_system user #54967
Cluster Coordination
  • Voting config exclusions should work with absent nodes #50836 (issue: #47990)
Features/Features
  • Add node local storage deprecation check #54383 (issue: #54374)
Features/Indices APIs
  • Deprecate local parameter for get field mapping request #55014
Infra/Core
  • Deprecate node local storage setting #54374
Infra/Plugins

New features

edit
Aggregations
  • Add Student’s t-test aggregation support #54469 (issue: #53692)
  • Add support for filters to t-test aggregation #54980 (issue: #53692)
  • Histogram field type support for Sum aggregation #55681 (issue: #53285)
  • Histogram field type support for ValueCount and Avg aggregations #55933 (issue: #53285)
Features/Indices APIs
  • Add simulate template composition API _index_template/_simulate_index/{name} #55686 (issue: #53101)
Geo
  • Add geo_bounds aggregation support for geo_shape #55328
  • Add geo_shape support for geotile_grid and geohash_grid #55966
  • Add geo_shape support for the geo_centroid aggregation #55602
  • Add new point field #53804
SQL

Enhancements

edit
Aggregations
  • Aggs must specify a field or script (or both) #52226
  • Expose aggregation usage in Feature Usage API #55732 (issue: #53746)
  • Reduce memory for big aggregations run against many shards #54758
  • Save memory in on aggs in async search #55683
Allocation
  • Disk decider respect watermarks for single data node #55805
  • Improve same-shard allocation explanations #56010
Analysis
  • Add preserve_original setting in ngram token filter #55432
  • Add preserve_original setting in edge ngram token filter #55766 (issue: #55767)
  • Add pre-configured “lowercase” normalizer #53882 (issue: #53872)
Audit
  • Update the audit logfile list of system users #55578 (issue: #37924)
Authentication
  • Let realms gracefully terminate the authN chain #55623
Authorization
  • Add reserved_ml_user and reserved_ml_admin kibana privileges #54713
Autoscaling
  • Rollover: refactor out cluster state update #53965
CRUD
  • Avoid holding onto bulk items until all completed #54407
Cluster Coordination
  • Add voting config exclusion add and clear API spec and integration test cases #55760 (issue: #48131)
Features/CAT APIs
  • Add support for V2 index templates to /_cat/templates #55829 (issue: #53101)
Features/Indices APIs
  • Add HLRC support for simulate index template api #55936 (issue: #53101)
  • Add prefer_v2_templates flag and index setting #55411 (issue: #53101)
  • Add warnings/errors when V2 templates would match same indices as V1 #54367 (issue: #53101)
  • Disallow merging existing mapping field definitions in templates #57701 (issues: #55607, #55982, #57393)
  • Emit deprecation warning if multiple v1 templates match with a new index #55558 (issue: #53101)
  • Guard adding the index.prefer_v2_templates settings for pre-7.8 nodes #55546 (issues: #53101, #55411, #55539)
  • Handle merging dotted object names when merging V2 template mappings #55982 (issue: #53101)
  • Throw exception on duplicate mappings metadata fields when merging templates #57835 (issue: #57701)
  • Update template v2 api rest spec #55948 (issue: #53101)
  • Use V2 index templates during index creation #54669 (issue: #53101)
  • Use V2 templates when reading duplicate aliases and ingest pipelines #54902 (issue: #53101)
  • Validate V2 templates more strictly #56170 (issues: #43737, #46045, #53101, #53970)
Features/Java High Level REST Client
  • Enable support for decompression of compressed response within RestHighLevelClient #53533
Features/Stats
  • Fix available / total disk cluster stats #32480 (issue: #32478)
Features/Watcher
Geo
  • Add geo_shape mapper supporting doc-values in Spatial Plugin #55037 (issue: #53562)
Infra/Core
  • Decouple Environment from DiscoveryNode #54373
  • Ensure that the output of node roles are sorted #54376 (issue: #54370)
  • Reintroduce system index APIs for Kibana #54858 (issues: #52385, #53912)
  • Schedule commands in current thread context #54187 (issue: #17143)
  • Start resource watcher service early #54993 (issue: #54867)
Infra/Packaging
  • Make Windows JAVA_HOME handling consistent with Linux #55261 (issue: #55134)
Infra/REST API
  • Add validation to the usage service #54617
Infra/Scripting
  • Scripting: stats per context in nodes stats #54008 (issue: #50152)
Machine Learning
  • Add effective max model memory limit to ML info #55529 (issue: #63942)
  • Add loss_function to regression #56118
  • Add new inference_config field to trained model config #54421
  • Add failed_category_count to model_size_stats #55716 (issue: #1130)
  • Add prediction_field_type to inference config #55128
  • Allow a certain number of ill-formatted rows when delimited format is specified #55735 (issue: #38890)
  • Apply default timeout in StopDataFrameAnalyticsAction.Request #55512
  • Create an annotation when a model snapshot is stored #53783 (issue: #52149)
  • Do not execute ML CRUD actions when upgrade mode is enabled #54437 (issue: #54326)
  • Make find_file_structure recognize Kibana CSV report timestamps #55609 (issue: #55586)
  • More advanced model snapshot retention options #56125 (issue: #52150)
  • Return assigned node in start/open job/datafeed response #55473 (issue: #54067)
  • Skip daily maintenance activity if upgrade mode is enabled #54565 (issue: #54326)
  • Start gathering and storing inference stats #53429
  • Unassign data frame analytics tasks in SetUpgradeModeAction #54523 (issue: #54326)
  • Speed up anomaly detection for the lat_long function #1102
  • Reduce CPU scheduling priority of native analysis processes to favor the ES JVM when CPU is constrained. This change is implemented only for Linux and macOS, not for Windows #1109
  • Take training_percent into account when estimating memory usage for classification and regression #1111
  • Support maximize minimum recall when assigning class labels for multiclass classification #1113
  • Improve robustness of anomaly detection to bad input data #1114
  • Add new num_matches and preferred_to_categories fields to category output #1062
  • Add mean squared logarithmic error (MSLE) for regression #1101
  • Add pseudo-Huber loss for regression #1168
  • Reduce peak memory usage and memory estimates for classification and regression #1125.)
  • Reduce variability of classification and regression results across our target operating systems #1127
  • Switch data frame analytics model memory estimates from kilobytes to megabytes #1126 (issue: #54506)
  • Add a machine learning native code build for Linux on AArch64 #1132, #1135
  • Improve data frame analytics runtime by optimising memory alignment for intrinsic operations #1142
  • Fix spurious anomalies for count and sum functions after no data are received for long periods of time #1158
  • Improve false positive rates from periodicity test for time series anomaly detection #1177
  • Break progress reporting of data frame analyses into multiple phases #1179
  • Really centre the data before training for classification and regression begins. This means we can choose more optimal smoothing bias and should reduce the number of trees #1192
Mapping
  • Merge V2 index/component template mappings in specific manner #55607 (issue: #53101)
Recovery
  • Avoid copying file chunks in peer recovery #56072 (issue: #55353)
  • Retry failed peer recovery due to transient errors #55353
SQL
  • Add BigDecimal support to JDBC #56015 (issue: #43806)
  • Drop BASE TABLE type in favour for just TABLE #54836
  • Relax version lock between server and clients #56148
Search
  • Consolidate DelayableWriteable #55932
  • Exists queries to MatchNoneQueryBuilder when the field is unmapped #54857
  • Rewrite wrapper queries to match_none if possible #55271
  • SearchService#canMatch takes into consideration the alias filter #55120 (issue: #55090)
Snapshot/Restore
  • Add GCS support for searchable snapshots #55403
  • Allocate searchable snapshots with the balancer #54889 (issues: #50999, #54729)
  • Allow bulk snapshot deletes to abort #56009 (issue: #55773)
  • Allow deleting multiple snapshots at once #55474
  • Allow searching of snapshot taken while indexing #55511 (issue: #50999)
  • Allow to prewarm the cache for searchable snapshot shards #55322
  • Enable prewarming by default for searchable snapshots #56201 (issue: #55952)
  • Permit searches to be concurrent to prewarming #55795
  • Reduce contention in CacheFile.fileLock() method #55662
  • Require soft deletes for searchable snapshots #55453
  • Searchable Snapshots should respect max_restore_bytes_per_sec #55952
  • Update the HDFS version used by HDFS Repo #53693
  • Use streaming reads for GCS #55506 (issue: #55505)
  • Use workers to warm cache parts #55793 (issue: #55322)
Task Management
  • Add indexName in update-settings task name #55714
  • Add scroll info to search task description #54606
  • Broadcast cancellation to only nodes have outstanding child tasks #54312 (issues: #50990, #51157)
  • Support hierarchical task cancellation #54757 (issue: #50990)
Transform

Bug fixes

edit
Aggregations
  • Add analytics plugin usage stats to _xpack/usage #54911 (issue: #54847)
  • Aggregation support for Value Scripts that change types #54830 (issue: #54655)
  • Allow terms agg to default to depth first #54845
  • Clean up how pipeline aggs check for multi-bucket #54161 (issue: #53215)
  • Fix auto_date_histogram serialization bug #54447 (issues: #54382, #54429)
  • Fix error massage for unknown value type #55821 (issue: #55727)
  • Fix scripted metric in CCS #54776 (issue: #54758)
  • Use Decimal formatter for Numeric ValuesSourceTypes #54366 (issue: #54365)
Allocation
  • Fix Broken ExistingStoreRecoverySource Deserialization #55657 (issue: #55513)
Features/ILM+SLM
  • ILM stop step execution if writeIndex is false #54805
Features/Indices APIs
  • Fix NPE in MetadataIndexTemplateService#findV2Template #54945
  • Fix creating filtered alias using now in a date_nanos range query failed #54785 (issue: #54315)
  • Fix simulating index templates without specified index #56295 (issues: #53101, #56255)
  • Validate non-negative priorities for V2 index templates #56139 (issue: #53101)
Features/Watcher
  • Ensure watcher email action message ids are always unique #56574
Infra/Core
Machine Learning
  • Fix GET _ml/inference so size param is respected #57303 (issue: #57298)
  • Fix file structure finder multiline merge max for delimited formats #56023
  • Validate at least one feature is available for DF analytics #55876 (issue: #55593)
  • Trap and fail if insufficient features are supplied to data frame analyses. Otherwise, classification and regression got stuck at zero analyzing progress #1160 (issue: #55593)
  • Make categorization respect the model_memory_limit #1167 (issue: #1130)
  • Respect user overrides for max_trees for classification and regression #1185
  • Reset memory status from soft_limit to ok when pruning is no longer required #1193 (issue: #1131)
  • Fix restore from training state for classification and regression #1197
  • Improve the initialization of seasonal components for anomaly detection #1201 (issue: #1178)
Network
  • Fix issue with pipeline releasing bytes early #54458
  • Handle TLS file updates during startup #54999 (issue: #54867)
SQL
  • Fix DATETIME_PARSE behaviour regarding timezones #56158 (issue: #54960)
Search
  • Don’t expand default_field in query_string before required #55158 (issue: #53789)
  • Fix time_zone for query_string and date fields #55881 (issue: #55813)
Security
  • Fix certutil http for empty password with JDK 11 and lower #55437 (issue: #55386)
Transform
  • Fix count when matching exact ids #56544 (issue: #56196)
  • Fix http status code when bad scripts are provided #56117 (issue: #55994)

Regressions

edit
Infra/Scripting
  • Don’t double-wrap expression values #54432 (issue: #53661)
« Elasticsearch version 7.8.1 Elasticsearch version 7.7.1 »