Elasticsearch version 8.13.0
editElasticsearch version 8.13.0
editAlso see Breaking changes in 8.13.
Known issues
edit-
Searches involving nodes upgraded to 8.13.0 and a coordinator node that is running on version
8.12 or earlier can produce duplicate buckets when running
date_histogram
orhistogram
aggregations. This can happen during a rolling upgrade to 8.13 or while running cross-cluster searches. (issue: #108181). - Due to a bug in the bundled JDK 22 nodes might crash abruptly under high memory pressure. We recommend downgrading to JDK 21.0.2 asap to mitigate the issue.
-
Nodes upgraded to 8.13.0 fail to load downsampling persistent tasks. This prevents them from joining the cluster, blocking its upgrade (issue: #106880)
This affects clusters running version 8.10 or later, with an active downsampling configuration or a configuration that was activated at some point since upgrading to version 8.10 or later.
- When upgrading clusters from version 8.11.4 or earlier, if your cluster contains non-master-eligible nodes, information about the new functionality of these upgraded nodes may not be registered properly with the master node. This can lead to some new functionality added since 8.12.0 not being accessible on the upgraded cluster. If your cluster is running on ECK 2.12.1 and above, this may cause problems with finalizing the upgrade. To resolve this issue, perform a rolling restart on the non-master-eligible nodes once all Elasticsearch nodes are upgraded. This issue is fixed in 8.15.0.
-
The
pytorch_inference
process used to run Machine Learning models can consume large amounts of memory. In environments where the available memory is limited, the OS Out of Memory Killer will kill thepytorch_inference
process to reclaim memory. This can cause inference requests to fail. Elasticsearch will automatically restart thepytorch_inference
process after it is killed up to four times in 24 hours. (issue: #110530)
Breaking changes
edit- ES|QL
- TSDB
Bug fixes
edit- Aggregations
- Allocation
- Application
-
- Fix Search Applications bug where deleting an alias before deleting an application intermittently caused errors #106329
- Use search to determine if cluster contains data #103920
- [Connector API] Bugfix: support list type in filtering advenced snippet value #105633
-
[Connector API] Fix default ordering in
SyncJob
list endpoint #105945 - [Connector API] Fix serialisation of script params in connector index service #106060
- Authentication
- Authorization
-
- Adjust interception of requests for specific shard IDs #101656
- Client
-
-
Validate settings in
ReloadSecureSettings
API #103176
-
Validate settings in
- Data streams
-
-
Apm-data: fix
@custom
component templates #104182 -
Avoid false-positive matches on intermediate objects in
ecs@mappings
#105440 (issue: #102794) - Execute lazy rollover with an internal dedicated user #104732 #104905 (issue: #104732)
- Fix write index resolution when an alias is pointing to a TSDS #104440 (issue: #104189)
-
x-pack/plugin/core: add
match_mapping_type
toecs@mappings
dynamic templates #103035
-
Apm-data: fix
- Distributed
- ES|QL
-
- ESQL: Add single value checks on LIKE/RLIKE pushdown #103807 (issue: #103806)
- ESQL: Correct out-of-range filter pushdowns #99961 (issue: #99960)
- ESQL: Fix Analyzer to not interpret escaped * as a pattern #105325 (issue: #104955)
- ESQL: Fix a bug loading unindexed text fields #104553
- ESQL: Fix bug in grammar that allowed spaces inside id pattern #105476 (issue: #105441)
- ESQL: Fix replacement of nested expressions in aggs with multiple parameters #104718 (issue: #104706)
- ESQL: Fix wrong attribute shadowing in pushdown rules #105650 (issue: #105434)
- ESQL: Improve pushdown of certain filters #103538 (issue: #103536)
-
ESQL: allow
null
in date math #103610 (issue: #103085) -
ESQL: make
cidr_match
foldable #105403 (issue: #105376) - ES|QL: Disable optimizations that rely on Expression.nullable() #105691
- ES|QL: Improve type validation in aggs for UNSIGNED_LONG better support for VERSION #104911 (issue: #102961)
- ES|QL: better management of exact subfields for TEXT fields #103510 (issue: #99899)
-
Fix error on sorting unsortable
geo_point
andcartesian_point
#106351 (issue: #106007) -
For empty mappings use a
LocalRelation
#105081 (issue: #104809) - Resume driver when failing to fetch pages #106392 (issue: #106262)
- Review KEEP logic to prevent duplicate column names #103316
-
ProjectOperator
should not retain references to released blocks #105848
- Engine
- Health
- ILM+SLM
- Indices APIs
- Infra/CLI
-
- Fix server cli to always pass through exit code #104943
- Infra/Core
- Infra/Node Lifecycle
- Ingest Node
- Machine Learning
-
- Allow GET inference models by user a with read only permission #105346
-
Avoid computing
currentInferenceProcessors
on every cluster state #106057 - Catch all the potential exceptions in the ingest processor code #105391
- Changed system auditor to use levels #105429
- During ML maintenance, reset jobs in the reset state without a corresponding task #106062
-
Fix
categorize_text
aggregation nested under empty buckets #105987 (issue: #105836) - Fix resetting a job if the original reset task no longer exists. #106020
- Retry updates to model snapshot ID on job config #104077
- The OpenAI model parameter should be in service settings not task settings. Move the configuration field to service settings #105458
- Undeploy elser when inference model deleted #104230
- Mapping
-
- Fix parsing of flattened fields within subobjects: false #105373
- Network
-
- Fix use-after-free at event-loop shutdown #105486
- Search
-
- Correct profiled rewrite time for knn with a pre-filter #104150
-
Force execution of
SearchService.Reaper
#106544 (issue: #106543) -
Move
TransportTermsEnumAction
coordination off transport threads #104408 -
Remove
SearchException
usages without a proper status code #105150 -
Require the name field for
inner_hits
for collapse #104666 - add validation on _id field when upsert new doc #103399 (issue: #102981)
- Security
-
-
Revert "Validate settings in
ReloadSecureSettings
API" #103310
-
Revert "Validate settings in
- Snapshot/Restore
- TLS
-
- Respect --pass option in certutil csr mode #106105
- Transform
-
-
Fix
_reset
API when called withforce=true
on a failed transform #106574 (issue: #106573) - Fix a bug where destination index aliases are not set up for an unattended transform #105499
- Remove duplicate checkpoint audits #105164 (issue: #105106)
- Return results in order #105089 (issue: #104847)
-
Use deduced mappings for determining proper fields' format even if
deduce_mappings==false
#103682 (issue: #103115)
-
Fix
- Vector Search
-
- Fix bug when nested knn pre-filter might match nested docs #105994
- Watcher
Deprecations
editEnhancements
edit- Aggregations
- Allocation
- Application
-
- Add serverless scopes for Connector APIs #104063
- [Connector API] Change required privileges to indices:data/read(write) #105289
-
[Connector API] Implement update
index_name
action #104648 - [Connector API] Support filtering by name, index name in list action #105131
- [Connector API] Support filtering connectors by service type and a query #105178
- [Connector API] Support updating configuration values only #105249
-
[Connectors API] Add new field
api_key_secret_id
to Connector #104982 - [Connectors API] Implement connector status update action #104750
- [Connectors API] Implement update native action endpoint #104654
- [Connectors API] Implement update service type action #104643
- [Connectors API] Relax strict response parsing for get/list operations #104909
- [Profiling] Extract properties faster from source #104356
- [Profiling] Mark all templates as managed #103783
- [Profiling] Speed up processing of stacktraces #104674
- [Profiling] Support downsampling of generic events #104730
- [Profiling] Use shard request cache consistently #103643
- Authentication
- Authorization
-
-
[Security Solution] Allow write permission for
kibana_system
role on endpoint response index #103555
-
[Security Solution] Allow write permission for
- CRUD
- Client
-
- Add rest spec for Query User API #104529
- Cluster Coordination
- Data streams
-
- Introduce lazy rollover for mapping updates in data streams #103309 (issue: #89346)
-
Use new
ignore_dynamic_beyond_limit
in logs and metric data streams #105180 - X-pack/plugin/apm-data: add dynamic setting for enabling template registry #104386 (issue: #104385)
-
X-pack/plugin/core: rename
double_metrics
template #103033 - x-pack/plugin/apm-data: Add a new field transaction.profiler_stack_trace_ids to traces-apm@mappings.yaml #105223
- x-pack/plugin/apm-data: Map some APM fields as flattened and fix error.grouping_name script #103032
- x-pack/plugin/core: make automatic rollovers lazy #105273 (issue: #104083)
- Discovery-Plugins
- Downsampling
-
- Support patch transport version from 8.12 #104406
- ES|QL
-
- Add ES|QL async delete API #103628
- Avoid humongous blocks #103340
- ESQL: Add TO_UPPER and TO_LOWER functions #104309
- ESQL: Add option to drop null fields #102428
- ESQL: Add plan consistency verification after each optimizer #105371
-
ESQL: Check field exists before load from
_source
#103632 - ESQL: Delay finding field load infrastructure #103821
- ESQL: Expand shallow copy with vecs #103681 (issue: #100528)
- ESQL: Extend STATS command to support aggregate expressions #104958
- ESQL: Infer not null for aggregated fields #103673 (issue: #102787)
- ESQL: Nested expressions inside stats command #104387 (issue: #99828)
- ESQL: Pre-allocate rows in TopNOperator #104796
- ESQL: Referencing expressions that contain backticks requires escaping those backticks. #100740 (issue: #100312)
- ESQL: Simpify IS NULL/IS NOT NULL evaluation #103099 (issue: #103097)
- ESQL: Speed up reading many nulls #105088
- ESQL: Support loading shapes from source into WKB blocks #104269
-
ESQL: Track the rest of
DocVector
#103727 -
ESQL:
MV_FIRST
andMV_LAST
#103928 -
ESQL: add
date_diff
function #104118 (issue: #101942) - ESQL: push down "[text_field] is not null" #105593
- ES|QL Async Query API #103398
- Prepare enrich plan to support multi clusters #104355
- Reading points from source to reduce precision loss #103698
- Remove deprecated Block APIs #103592
- Reserve bytes before serializing page #105269
- Support ST_CENTROID over spatial points #104218 (issue: #104656)
- Support cross clusters query in ESQL #101640
- Support enrich ANY mode in cross clusters query #104840
- Support enrich coordinator mode #104936
- Support enrich remote mode #104993
- Geo
- Health
- ILM+SLM
- Infra/CLI
-
- Add replay diagnostic dir to system jvm options #103535
- Infra/Circuit Breakers
-
- Lower G1 minimum full GC interval #105259
- Infra/Core
- Infra/Metrics
-
- Modify name of threadpool metric for rejected #105015
- Infra/Node Lifecycle
-
- Wait for async searches to finish when shutting down #101487
- Infra/Transport API
-
-
Make
ParentTaskAssigningClient.getRemoteClusterClient
method also returnParentTaskAssigningClient
#100813
-
Make
- Ingest Node
-
-
Adding
ActionRequestLazyBuilder
implementation ofRequestBuilder
#104927 -
Adding a
RequestBuilder
interface #104778 - Adding a custom exception for problems with the graph of pipelines to be applied to a document #105196
- Improving the performance of the ingest simulate verbose API #105265
- Ingest geoip processor cache no results from the database #104092
- Limiting the number of nested pipelines that can be executed #105428
- Modifying request builders #104636
-
Adding
- Java Low Level REST Client
-
- Set thread name used by REST client #103160
- Machine Learning
-
- Add optional pruning configuration (weighted terms scoring) to text expansion query #102862
- Add text_embedding inference service with multilingual-e5 and custom eland models #104949
-
Add 3 automatic restarts for
pytorch_inference
processes that stop unexpectedly #104433 - Add support for Cohere inference service #104559
- Always test for spikes and dips as well as changes in the change point aggregation #103922
- Apply windowing and chunking to long documents #104363
-
Automatically download the ELSER model when PUT in
_inference
#104334 - Better handling of number of allocations in pytorch_inference in the case that hardware_concurrency fails #2607
- Change detection aggregation improvements #102824
- Conditionally send the dimensions field as part of the openai requests #105299 (issue: #105005)
- Endpoint to find positions of Grok pattern matches #104394
- Ensure unique IDs between inference models and trained model deployments #103996
- Expose some ML metrics via APM #102584
-
Make
task_type
optional in_inference
APIs #104483 -
Update
missingTrainedModel
message to include: you may need to create it #104155 - Upgrade MKL to version 2024.0 on Linux x86_64 #2619
- Upgrade PyTorch to version 2.1.2. #2588
- Upgrade zlib to version 1.2.13 on Windows #2588
- Use Boost.JSON for JSON processing #2614
- Validate inference model ids #103669
- Mapping
- Network
- Percolator
- Query Languages
-
- Introduce Alias.unwrap method #104575
- Search
-
- Dyamically adjust node metrics cache expire #104460
- Enhancement: Metrics for Search Took Times using Action Listeners #104996
- Field caps performance pt2 #105941
- Field-caps field has value lookup use map instead of looping array #105770
-
Flag in
_field_caps
to return only fields with values in index #103651 -
Include better output in profiling &
toString
for automaton based queries #105468 - Metrics for search latencies #102557
- Ref count search response bytes #103763 (issue: #102657)
- Remove leniency in msearch parsing #103232
- Resolve Cluster API #102726
- Reuse number field mapper tests in other modules #99142 (issue: #92947)
- S3 first byte latency metric #102435
- Update s3 latency metric to use micros #103633
- Upgrade to Lucene 9.10.0 #105578
- Security
-
- Add Query Users API #104033
-
Add
ApiKey
expiration time to audit log #103959 - Add expiration time to update api key api #103453
- Add stricter validation for api key expiration time #103973
-
Add support for the
simple_query_string
to the Query API Key API #104132 -
Add support for the
type
parameter, for sorting, to the Query API Key API #104625 - Aggs support for Query API Key Information API #104895
- Hot-reloadable remote cluster credentials #102798
- Snapshot/Restore
-
-
Add s3
HeadObject
request to request stats #105105 -
Expose
OperationPurpose
in S3 access logs using a custom query-string parameter #105044 - Fix blob cache race, decay, time dependency #104784
- Pause shard snapshots on graceful shutdown #101717
- Retry indefinitely for s3 indices blob read errors #103300
-
Add s3
- Store
-
- List hidden shard stores by default #103710
- TLS
-
- elasticsearch-certutil cert now verifies the issuing chain of the generated certificate #103948
- TSDB
- Transform
- Vector Search