New

The executive guide to generative AI

Read more
About usPartnersSupport|Login
This documentation contains work-in-progress information for future Elastic Stack and Cloud releases. Use the version selector to view supported release docs. It also contains some Elastic Cloud serverless information. Check out our serverless docs for more details.
« Release notes Elasticsearch version 8.17.4 »
Elastic Docs ›Elasticsearch Guide [8.18] ›Release notes

Elasticsearch version 8.18.0

edit

Elasticsearch version 8.18.0

edit

Coming in 8.18.0.

Also see Breaking changes in 8.18.

Breaking changes

edit
EQL
  • Set allow_partial_search_results=true by default #120267
Machine Learning
  • Disable machine learning on macOS x86_64 #104125
Search
  • Change Semantic Text To Act Like A Normal Text Field #120813
TLS
  • Drop TLS_RSA cipher support for JDK 24 #123600

Bug fixes

edit
Analysis
  • Adjust exception thrown when unable to load hunspell dict #123743
  • Non existing synonyms sets do not fail shard recovery for indices #125659 (issue: #125603)
CAT APIs
  • Fix cat_component_templates documentation #120487
Data streams
  • Avoid updating settings version in MetadataMigrateToDataStreamService when settings have not changed #118704
  • Ensure removal of index blocks does not leave key with null value #122246
  • Match dot prefix of migrated DS backing index with the source index #120042
  • Refresh source index before reindexing data stream index #120752 (issue: #120314)
  • Updating TransportRolloverAction.checkBlock so that non-write-index blocks do not prevent data stream rollover #122905
  • ReindexDataStreamIndex bug in assertion caused by reference equality #121325
Downsampling
  • Copy metrics and default_metric properties when downsampling aggregate_metric_double #121727 (issues: #119696, #96076)
  • Improve downsample performance by avoiding to read unnecessary dimension values when downsampling. #124451
EQL
  • Fix EQL double invoking listener #124918
ES|QL
  • Avoid over collecting in Limit or Lucene Operator #123296
  • Correct line and column numbers of missing named parameters #120852
  • Drop null columns in text formats #117643 (issue: #116848)
  • ES|QL - date nanos range bug? #125345 (issue: #125439)
  • ES|QL - Fix lucene push down behavior when a range contains nanos and millis #125595
  • Fix ROUND() with unsigned longs throwing in some edge cases #119536
  • Fix TopN row size estimate #119476 (issue: #106956)
  • Fix AbstractShapeGeometryFieldMapperTests #119265 (issue: #119201)
  • Fix ReplaceMissingFieldsWithNull #125764 (issues: #126036, #121754, #126030)
  • Fix a bug in TOP #121552
  • Fix async stop sometimes not properly collecting result #121843 (issue: #121249)
  • Fix attribute set equals #118823
  • Fix double lookup failure on ES|QL #115616 (issue: #111398)
  • Fix queries with document level security on lookup indexes #120617 (issue: #120509)
  • Fix writing for LOOKUP status #119296 (issue: #119086)
  • Implicit numeric casting for CASE/GREATEST/LEAST #122601 (issue: #121890)
  • Lazy collection copying during node transform #124424
  • Limit memory usage of fold #118602
  • Limit size of query #117898
  • Make numberOfChannels consistent with layout map by removing duplicated ChannelSet #125636
  • Reduce iteration complexity for plan traversal #123427
  • Remove redundant sorts from execution plan #121156
  • Use a must boolean statement when pushing down to Lucene when scoring is also needed #124001 (issue: #123967)
Health
  • Do not recommend increasing max_shards_per_node #120458
ILM+SLM
  • Add missing timeouts to rest-api-spec SLM APIs #119447
Indices APIs
  • Fix NPE in rolling over unknown target and return 404 #125352
  • Include hidden indices in DeprecationInfoAction #118035 (issue: #118020)
  • Preventing ConcurrentModificationException when updating settings for more than one index #126077
  • Updates the deprecation info API to not warn about system indices and data streams #122951
  • Avoid hoarding cluster state references during rollover #124266
Inference
  • [Inference API] Put back legacy EIS URL setting #121207
Infra/Core
  • Epoch Millis Rounding Down and Not Up 2 #118353
  • Fix system data streams to be restorable from a snapshot #124651 (issue: #89261)
  • Have create index return a bad request on poor formatting #123761
  • Include data streams when converting an existing resource to a system resource #121392
  • System Index Migration Failure Results in a Non-Recoverable State #122326
  • System data streams are not being upgraded in the feature migration API #123926
  • Wrap jackson exception on malformed json string #114445 (issue: #114142)
Infra/Logging
  • Move SlowLogFieldProvider instantiation to node construction #117949
Infra/Plugins
  • Remove unnecessary entitlement #120959
  • Restrict agent entitlements to the system classloader unnamed module #120546
Ingest Node
  • Fix geoip databases index access after system feature migration #121196
  • Fix geoip databases index access after system feature migration (again) #122938
  • Fix geoip databases index access after system feature migration (take 3) #124604
Machine Learning
  • Add ElasticInferenceServiceCompletionServiceSettings #123155
  • Add enterprise license check to inference action for semantic text fields #122293
  • Avoid potentially throwing calls to Task#getDescription in model download #124527
  • Change format for Unified Chat #121396
  • Fix AlibabaCloudSearchCompletionAction not accepting ChatCompletionInputs #125023
  • Fix get all inference endponts not returning multiple endpoints sharing model deployment #121821
  • Fix serialising the inference update request #122278
  • Fixing bedrock event executor terminated cache issue #118177 (issue: #117916)
  • Fixing bug setting index when parsing Google Vertex AI results #117287
  • Retry on streaming errors #123076
  • Set Connect Timeout to 5s #123272
  • Set default similarity for Cohere model to cosine #125370 (issue: #122878)
  • Updating Inference Update API documentation to have the correct PUT method #121048
  • [Inference API] Fix output stream ordering in InferenceActionProxy #124225
Mapping
  • Avoid serializing empty _source fields in mappings #122606
  • Fix realtime get of nested fields with synthetic source #119575 (issue: #119553)
  • Merge field mappers when updating mappings with [subobjects:false] #120370 (issue: #120216)
  • Merge template mappings properly during validation #124784 (issue: #123372)
  • Tweak copy_to handling in synthetic _source to account for nested objects #120974 (issue: #120831)
Ranking
  • Fix LTR query feature with phrases (and two-phase) queries #125103
Search
  • Catch and handle disconnect exceptions in search #115836
  • Fix leak in DfsQueryPhase and introduce search disconnect stress test #116060 (issue: #115056)
  • Handle long overflow in dates #124048 (issue: #112483)
  • Handle search timeout in SuggestPhase #122357 (issue: #122186)
  • In this pr, a 400 error is returned when _source / _seq_no / _feature / _nested_path / _field_names is requested, rather a 5xx #117229
  • Load FieldInfos from store if not yet initialised through a refresh on IndexShard #125650 (issue: #125483)
  • Re-enable parallel collection for field sorted top hits #125916
  • Skip fetching _inference_fields field in legacy semantic_text format #121720
  • Support indices created in ESv6 and updated in ESV7 using different LuceneCodecs as archive in current version. #125389
  • Test/107515 RestoreTemplateWithMatchOnlyTextMapperIT #120898
  • Fix/SearchStatesIt_failures #117729
  • CrossClusterIT testCancel failure #117750 (issue: #108061)
Snapshot/Restore
  • Fork post-snapshot-delete cleanup off master thread #122731
  • This PR fixes a bug whereby partial snapshots of system datastreams could be used to restore system features. #124931
  • Use the system index descriptor in the snapshot blob cache cleanup task #120937 (issue: #120518)
Suggesters
  • Return an empty suggestion when suggest phase times out #122575 (issue: #122548)
Transform
  • If the Transform is configured to write to an alias as its destination index, when the delete_dest_index parameter is set to true, then the Delete API will now delete the write index backing the alias #122074 (issue: #121913)
Vector Search
  • Apply default k for knn query eagerly #118774
  • Fix bbq_hnsw merge file cleanup on random IO exceptions #119691 (issue: #119392)
  • Knn vector rescoring to sort score docs #122653 (issue: #119711)
  • Return appropriate error on null dims update instead of npe #125716
Watcher
  • Watcher history index has too many indexed fields - #117701 (issue: #71479)

Deprecations

edit
Indices APIs
  • Increase the frozen indices deprecation level to CRITICAL #119879
Infra/Core
  • Add deprecation warning to TransportHandshaker #123188
Infra/REST API
  • Enhancement/v7 critical deprecation logging #118298

Enhancements

edit
Authentication
  • Allow SSHA-256 for API key credential hash #120997
Authorization
  • Allow kibana_system user to manage .reindexed-v8-internal.alerts indices #118959
  • Do not fetch reserved roles from native store when Get Role API is called #121971
  • Make reserved built-in roles queryable #117581
  • [Security Solution] allows kibana_system user to manage .reindexed-v8-* Security Solution indices #119054
CCS
  • Resolve/cluster allows querying for cluster info only (no index expression required) #119898
Data streams
  • Add action to create index from a source index #118890
  • Add index and reindex request settings to speed up reindex #119780
  • Add rest endpoint for create_from_source_index #119250
  • Add sanity check to ReindexDatastreamIndexAction #120231
  • Adding a migration reindex cancel API #118291
  • Adding get migration reindex status #118267
  • Consistent mapping for OTel log and event bodies #120547
  • Filter deprecated settings when making dest index #120163
  • Ignore closed indices for reindex #120244
  • Improve how reindex data stream index action handles api blocks #120084
  • Initial work on ReindexDatastreamIndexAction #116996
  • Make requests_per_second configurable to throttle reindexing #120207
  • Optimized index sorting for OTel logs #119504
  • Reindex data stream indices on different nodes #125171
  • Report Deprecated Indices That Are Flagged To Ignore Migration Reindex As A Warning #120629
  • Retry ILM async action after reindexing data stream #124149
  • Update data stream deprecations warnings to new format and filter sea… #119097
Distributed
  • Metrics for incremental bulk splits #116765
Downsampling
  • Improve downsample performance by buffering docids and do bulk processing #124477
  • Improve rolling up metrics #124739
EQL
  • Add support for partial shard results #116388
  • Optional named arguments for function in map #118619
ES|QL
  • Add ES|QL cross-cluster query telemetry collection #119474
  • Add a LicenseAware interface for licensed Nodes #118931 (issue: #117405)
  • Add a PostAnalysisAware, distribute verification #119798
  • Add a standard deviation aggregating function: STD_DEV #116531
  • Add cluster level reduction #117731
  • Add nulls support to Categorize #117655
  • Async search responses have CCS metadata while searches are running #117265
  • Backport Term query for ES|QL to 8.x #118135
  • Backport scoring support in ES|QL to 8.x branch #117747
  • Check for early termination in Driver #118188
  • Do not serialize EsIndex in plan #119580
  • ES|QL - Remove restrictions for disjunctions in full text functions #118544
  • ES|QL - enabling scoring with METADATA _score #113120
  • ES|QL Add ES|QL hash function #117989
  • ES|QL Support IN operator for Date nanos #119772 (issue: #118578)
  • ES|QL: CATEGORIZE as a BlockHash #114317
  • ES|QL: Enterprise license enforcement for CCS #118102
  • ES|QL: Partial result on demand for async queries #118122
  • Enable KQL function as a tech preview #119730
  • Enable LOOKUP JOIN in non-snapshot builds #121193 (issue: #121185)
  • Enable node-level reduction by default #119621
  • Enable physical plan verification #118114
  • ES|QL - Support date nanos in date extract function #120727 (issue: #110000)
  • ES|QL - support date nanos in date format function #120143 (issue: #109994)
  • ES|QL Support date nanos on date diff function #120645 (issue: #109999)
  • ES|QL bucket function for date nanos #118474 (issue: #118031)
  • ES|QL compare nanos and millis #118027 (issue: #116281)
  • ES|QL implicit casting for date nanos #118697 (issue: #118476)
  • Extend TranslationAware to all pushable expressions #120192
  • Hash functions #118938
  • Implement a MetricsAware interface #121074
  • LOOKUP JOIN using field-caps for field mapping #117246
  • Lookup join on multiple join fields not yet supported #118858
  • Move scoring in ES|QL out of snapshot #120354
  • Optimize ST_EXTENT_AGG for geo_shape and cartesian_shape #119889
  • Push down StartsWith and EndsWith functions to Lucene #123381 (issue: #123067)
  • Push down filter passed lookup join #118410
  • Resume Driver on cancelled or early finished #120020
  • Reuse child outputSet inside the plan where possible #124611
  • Rewrite TO_UPPER/TO_LOWER comparisons #118870 (issue: #118304)
  • ST_EXTENT_AGG optimize envelope extraction from doc-values for cartesian_shape #118802
  • Smarter field caps with subscribable listener #116755
  • Support some stats on aggregate_metric_double #120343 (issue: #110649)
  • Take named parameters for identifier and pattern out of snapshot #121850
  • Term query for ES|QL #117359
  • Update grammar to rely on indexPattern instead of identifier in join target #120494
  • _score should not be a reserved attribute in ES|QL #118435 (issue: #118460)
Engine
  • Enhance add-block API to flush and add verified metadata #119743
Experiences
  • Integrate IBM watsonx to Inference API for re-ranking task #117176
Geo
  • Optimize indexing points with index and doc values set to true #120271
ILM+SLM
  • Add a replicate_for option to the ILM searchable_snapshot action #119003
Indices APIs
  • Add remove_index_block arg to _create_from api #120548
  • Remove index blocks by default in create_from #120643
  • introduce new categories for deprecated resources in deprecation API #120505
Inference
  • [Inference API] Rename model_id prop to model in EIS sparse inference request body #122398
  • Add version prefix to Inference Service API path #117696
  • Update sparse text embeddings API route for Inference Service #118369
  • [Elastic Inference Service] Add ElasticInferenceService Unified ChatCompletions Integration #118871
Infra/CLI
  • Ignore _JAVA_OPTIONS #124843
  • Strengthen encryption for elasticsearch-keystore tool to AES 256 #119749
Infra/Core
  • Improve size limiting string message #122427
  • Return unique deprecation for old indices with incompatible date formats #124597
Infra/REST API
  • A new query parameter ?include_source_on_error was added for create / index, update and bulk REST APIs to control if to include the document source in the error response in case of parsing errors. The default value is true. #120725
Ingest Node
  • Optimize IngestCtxMap construction #120833
  • Optimize IngestDocMetadata isAvailable #120753
  • Optimize IngestDocument FieldPath allocation #120573
  • Optimize some per-document hot paths in the geoip processor #120824
  • Returning ignored fields in the simulate ingest API #117214
Logs
  • Add LogsDB option to route on sort fields #116687
  • Add a new index setting to skip recovery source when synthetic source is enabled #114618
  • Configure index sorting through index settings for logsdb #118968 (issue: #118686)
  • Optimize loading mappings when determining synthetic source usage and whether host.name can be sorted on. #120055
Machine Learning
  • Add Inference Unified API for chat completions for OpenAI #117589
  • Add Jina AI API to do inference for Embedding and Rerank models #118652
  • Add enterprise license check for Inference API actions #119893
  • Adding chunking settings to IbmWatsonxService #114914
  • Adding default endpoint for Elastic Rerank #117939
  • Adding endpoint creation validation for all task types to remaining services #115020
  • Check for presence of error object when validating streaming responses from integrations in the inference API #118375
  • Ignore failures from renormalizing buckets in read-only index #118674
  • Inference duration and error metrics #115876
  • Migrate stream to core error parsing #120722
  • Remove all mentions of eis and gateway and deprecate flags that do #116692
  • Remove deprecated sort from reindex operation within dataframe analytics procedure #117606
  • Retry on ClusterBlockException on transform destination index #118194
Mapping
  • Add Optional Source Filtering to Source Loaders #113827
Network
  • Allow http unsafe buffers by default #116115
  • Http stream activity tracker and exceptions handling #119564
  • Remove HTTP content copies #117303
  • ConnectTransportException returns retryable BAD_GATEWAY #118681 (issue: #118320)
Ranking
  • Set default reranker for text similarity reranker to Elastic reranker #120551
Search
  • Add match support for semantic_text fields #117839
  • Add support for sparse_vector queries against semantic_text fields #118617
  • Add support for knn vector queries on semantic_text fields #119011
  • Adding linear retriever to support weighted sums of sub-retrievers #120222
  • Feat: add a user-configurable timeout parameter to the _resolve/cluster API #120542
  • Make semantic text part of the text family #119792
  • Only aggregations require at least one shard request #115314
  • Prevent data nodes from sending stack traces to coordinator when error_trace=false #118266
  • Propagate status codes from shard failures appropriately #118016 (issue: #118482)
Snapshot/Restore
Store
  • Abort pending deletion on IndicesService close #123569
TSDB
  • Increase field limit for OTel metrics to 10 000 #120591
Transform
  • Add support for extended_stats #120340
  • Auto-migrate max_page_search_size #119348
  • Create upgrade mode #117858
  • Wait while index is blocked #119542
  • [Deprecation] Add transform_ids to outdated index #120821
Vector Search
  • Even better(er) binary quantization #117994
  • Speed up bit compared with floats or bytes script operations #117199

New features

edit
CRUD
  • Metrics for indexing failures due to version conflicts #119067
ES|QL
  • ES|QL - Add Match function options #120360
  • ES|QL - Allow full text functions disjunctions for non-full text functions #120291
  • ES|QL: Enable async get to support formatting #111104 (issue: #110926)
  • Expand type compatibility for match function and operator #117555
  • ST_EXTENT aggregation #117451 (issue: #104659)
  • Support ST_ENVELOPE and related (ST_XMIN, ST_XMAX, ST_YMIN, ST_YMAX) functions #116964 (issue: #104875)
Highlighting
  • Add Highlighter for Semantic Text Fields #118064
Infra/Core
  • Infrastructure for assuming cluster features in the next major version #118143
Machine Learning
  • ES|QL categorize with multiple groupings #118173
  • Support mTLS for the Elastic Inference Service integration inside the inference API #119679
  • [Inference API] Add node-local rate limiting for the inference API #120400
Mapping
  • Add option to store sparse_vector outside _source #117917
  • Release semantic_text as a GA feature #124670
Ranking
  • Add a generic rescorer retriever based on the search request’s rescore functionality #118585 (issue: #118327)
Relevance
  • Add Multi-Field Support for Semantic Text Fields #120128
Vector Search
  • Add new experimental rank_vectors mapping for late-interaction second order ranking #118804
  • KNN vector rescoring for quantized vectors #116663
  • Mark bbq indices as GA and add rolling upgrade integration tests #121105
  • Add new experimental rank_vectors mapping for late-interaction second order ranking #119601

Upgrades

edit
Infra/Core
  • Bump major version for feature migration system indices #117243
  • Permanently switch from Java SecurityManager to Entitlements. The Java SecurityManager has been deprecated since Java 17, and it is now completely disabled in Java 24. In order to retain an similar level of protection, Elasticsearch implemented its own protection mechanism, Entitlements. Starting with this version, Entitlements will permanently replace the Java SecurityManager. #125073
  • Update ASM 9.7 → 9.7.1 to support JDK 24 #118094
Machine Learning
  • Automatically rollover legacy .ml-anomalies indices #120885
  • Automatically rollover legacy ml indices #120405
  • Change the auditor to write via an alias #120064
  • Check if the anomaly results index has been rolled over #125404
  • Update minimum supported snapshot version for Machine Learning jobs to 8.3.0 #118166
Packaging
  • Update bundled JDK to Java 24 #125159
Search
Watcher
  • Script for migrating .watches and .triggered_watches indices #120371
« Release notes Elasticsearch version 8.17.4 »

On this page

Was this helpful?
Feedback