The API accepts 3 different authentication methods:
Elasticsearch APIs support key-based authentication. You must create an API key and use the encoded value in the request header. For example:
curl -X GET "${ES_URL}/_cat/indices?v=true" \
-H "Authorization: ApiKey ${API_KEY}"
To get API keys, use the /_security/api_key APIs.
Basic auth tokens are constructed with the Basic keyword, followed by a space, followed by a base64-encoded string of your username:password
(separated by a : colon).
Example: send a Authorization: Basic aGVsbG86aGVsbG8=
HTTP header with your requests to authenticate with the API.
Elasticsearch APIs support the use of bearer tokens in the Authorization HTTP header to authenticate with the API.
For examples, refer to Token-based authentication services
The name of the behavioral analytics collection.
The analytics event type.
Values are page_view, search, or search_click.
Whether the response type has to include more details
curl \
--request POST 'http://api.example.com/_application/analytics/{collection_name}/event/{event_type}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"session\": {\n \"id\": \"1797ca95-91c9-4e2e-b1bd-9c38e6f386a9\"\n },\n \"user\": {\n \"id\": \"5f26f01a-bbee-4202-9298-81261067abbd\"\n },\n \"search\":{\n \"query\": \"search term\",\n \"results\": {\n \"items\": [\n {\n \"document\": {\n \"id\": \"123\",\n \"index\": \"products\"\n }\n }\n ],\n \"total_results\": 10\n },\n \"sort\": {\n \"name\": \"relevance\"\n },\n \"search_application\": \"website\"\n },\n \"document\":{\n \"id\": \"123\",\n \"index\": \"products\"\n }\n}"'{
"session": {
"id": "1797ca95-91c9-4e2e-b1bd-9c38e6f386a9"
},
"user": {
"id": "5f26f01a-bbee-4202-9298-81261067abbd"
},
"search":{
"query": "search term",
"results": {
"items": [
{
"document": {
"id": "123",
"index": "products"
}
}
],
"total_results": 10
},
"sort": {
"name": "relevance"
},
"search_application": "website"
},
"document":{
"id": "123",
"index": "products"
}
}{
"accepted": true,
"event": {}
}Get information about component templates in a cluster. Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.
IMPORTANT: CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the get component template API.
The name of the component template. It accepts wildcard expressions. If it is omitted, all component templates are returned.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
The period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/component_templates/{name}' \
--header "Authorization: $API_KEY"[
{
"name": "my-template-1",
"version": "null",
"alias_count": "0",
"mapping_count": "0",
"settings_count": "1",
"metadata_count": "0",
"included_in": "[my-index-template]"
},
{
"name": "my-template-2",
"version": null,
"alias_count": "0",
"mapping_count": "3",
"settings_count": "0",
"metadata_count": "0",
"included_in": "[my-index-template]"
}
]Get high-level information about indices in a cluster, including backing indices for data streams.
Use this request to get the following information for each index in a cluster:
These metrics are retrieved directly from Lucene, which Elasticsearch uses internally to power indexing and search. As a result, all document counts include hidden nested documents. To get an accurate count of Elasticsearch documents, use the cat count or count APIs.
CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use an index endpoint.
Comma-separated list of data streams, indices, and aliases used to limit the request.
Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
The type of index that wildcard patterns can match.
The health status used to limit returned indices. By default, the response includes indices of any health status.
Values are green, GREEN, yellow, YELLOW, red, or RED.
If true, the response includes information from segments that are not loaded into memory.
If true, the response only includes information from primary shards.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
Period to wait for a connection to the master node.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
curl \
--request GET 'http://api.example.com/_cat/indices/{index}' \
--header "Authorization: $API_KEY"[
{
"health": "yellow",
"status": "open",
"index": "my-index-000001",
"uuid": "u8FNjxh8Rfy_awN11oDKYQ",
"pri": "1",
"rep": "1",
"docs.count": "1200",
"docs.deleted": "0",
"store.size": "88.1kb",
"pri.store.size": "88.1kb",
"dataset.size": "88.1kb"
},
{
"health": "green",
"status": "open",
"index": "my-index-000002",
"uuid": "nYFWZEO7TUiOjLQXBaYJpA ",
"pri": "1",
"rep": "0",
"docs.count": "0",
"docs.deleted": "0",
"store.size": "260b",
"pri.store.size": "260b",
"dataset.size": "260b"
}
]Get configuration and usage information about inference trained models.
IMPORTANT: CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get trained models statistics API.
A unique identifier for the trained model.
Specifies what to do when the request: contains wildcard expressions and there are no models that match; contains the _all string or no identifiers and there are no matches; contains wildcard expressions and there are only partial matches.
If true, the API returns an empty array when there are no matches and the subset of results when there are partial matches.
If false, the API returns a 404 status code when there are no matches or only partial matches.
The unit used to display byte values.
Values are b, kb, mb, gb, tb, or pb.
A comma-separated list of column names to display.
A comma-separated list of column names or aliases used to sort the response.
Skips the specified number of transforms.
The maximum number of transforms to display.
Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/ml/trained_models/{model_id}' \
--header "Authorization: $API_KEY"[
{
"id": "ddddd-1580216177138",
"heap_size": "0b",
"operations": "196",
"create_time": "2025-03-25T00:01:38.662Z",
"type": "pytorch",
"ingest.pipelines": "0",
"data_frame.id": "__none__"
},
{
"id": "lang_ident_model_1",
"heap_size": "1mb",
"operations": "39629",
"create_time": "2019-12-05T12:28:34.594Z",
"type": "lang_ident",
"ingest.pipelines": "0",
"data_frame.id": "__none__"
}
]Get information about cluster-level changes that have not yet taken effect. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the pending cluster tasks API.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
Unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
curl \
--request GET 'http://api.example.com/_cat/pending_tasks' \
--header "Authorization: $API_KEY"[
{ "insertOrder": "1685", "timeInQueue": "855ms", "priority": "HIGH", "source": "update-mapping [foo][t]"},
{ "insertOrder": "1686", "timeInQueue": "843ms", "priority": "HIGH", "source": "update-mapping [foo][t]"},
{ "insertOrder": "1693", "timeInQueue": "753ms", "priority": "HIGH", "source": "refresh-mapping [foo][[t]]"},
{ "insertOrder": "1688", "timeInQueue": "816ms", "priority": "HIGH", "source": "update-mapping [foo][t]"},
{ "insertOrder": "1689", "timeInQueue": "802ms", "priority": "HIGH", "source": "update-mapping [foo][t]"},
{ "insertOrder": "1690", "timeInQueue": "787ms", "priority": "HIGH", "source": "update-mapping [foo][t]"},
{ "insertOrder": "1691", "timeInQueue": "773ms", "priority": "HIGH", "source": "update-mapping [foo][t]"}
]Get thread pool statistics for each node in a cluster. Returned information includes all built-in thread pools and custom thread pools. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the nodes info API.
List of columns to appear in the response. Supports simple wildcards.
List of columns that determine how the table should be sorted.
Sorting defaults to ascending and can be changed by setting :asc
or :desc as a suffix to the column name.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
If true, the request computes the list of selected nodes from the
local cluster state. If false the list of selected nodes are computed
from the cluster state of the master node. In both cases the coordinating
node will send requests for further information to each selected node.
Period to wait for a connection to the master node.
curl \
--request GET 'http://api.example.com/_cat/thread_pool' \
--header "Authorization: $API_KEY"[
{
"node_name": "node-0",
"name": "analyze",
"active": "0",
"queue": "0",
"rejected": "0"
},
{
"node_name": "node-0",
"name": "fetch_shard_started",
"active": "0",
"queue": "0",
"rejected": "0"
},
{
"node_name": "node-0",
"name": "fetch_shard_store",
"active": "0",
"queue": "0",
"rejected": "0"
},
{
"node_name": "node-0",
"name": "flush",
"active": "0",
"queue": "0",
"rejected": "0"
},
{
"node_name": "node-0",
"name": "write",
"active": "0",
"queue": "0",
"rejected": "0"
}
][
{
"id": "0EWUhXeBQtaVGlexUeVwMg",
"name": "generic",
"active": "0",
"rejected": "0",
"completed": "70"
}
]Get configuration and usage information about transforms.
CAT APIs are only intended for human consumption using the Kibana console or command line. They are not intended for use by applications. For application consumption, use the get transform statistics API.
Specifies what to do when the request: contains wildcard expressions and there are no transforms that match; contains the _all string or no identifiers and there are no matches; contains wildcard expressions and there are only partial matches.
If true, it returns an empty transforms array when there are no matches and the subset of results when there are partial matches.
If false, the request returns a 404 status code when there are no matches or only partial matches.
Skips the specified number of transforms.
Comma-separated list of column names to display.
Comma-separated list of column names or column aliases used to sort the response.
The unit used to display time values.
Values are nanos, micros, ms, s, m, h, or d.
The maximum number of transforms to obtain.
curl \
--request GET 'http://api.example.com/_cat/transforms' \
--header "Authorization: $API_KEY"[
{
"id" : "ecommerce_transform",
"state" : "started",
"checkpoint" : "1",
"documents_processed" : "705",
"checkpoint_progress" : "100.00",
"changes_last_detection_time" : null
}
]You can also use the API to get the health status of only specified data streams and indices. For data streams, the API retrieves the health status of the stream’s backing indices.
The cluster health status is: green, yellow or red. On the shard level, a red status indicates that the specific shard is not allocated in the cluster. Yellow means that the primary shard is allocated but replicas are not. Green means that all shards are allocated. The index level status is controlled by the worst shard status.
One of the main benefits of the API is the ability to wait until the cluster reaches a certain high watermark health level. The cluster status is controlled by the worst index status.
Whether to expand wildcard expression to concrete indices that are open, closed or both.
Can be one of cluster, indices or shards. Controls the details level of the health information returned.
Values are cluster, indices, or shards.
If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
A number controlling to how many active shards to wait for, all to wait for all shards in the cluster to be active, or 0 to not wait.
Can be one of immediate, urgent, high, normal, low, languid. Wait until all currently queued events with the given priority are processed.
Values are immediate, urgent, high, normal, low, or languid.
The request waits until the specified number N of nodes is available. It also accepts >=N, <=N, >N and <N. Alternatively, it is possible to use ge(N), le(N), gt(N) and lt(N) notation.
A boolean value which controls whether to wait (until the timeout provided) for the cluster to have no shard initializations. Defaults to false, which means it will not wait for initializing shards.
A boolean value which controls whether to wait (until the timeout provided) for the cluster to have no shard relocations. Defaults to false, which means it will not wait for relocating shards.
One of green, yellow or red. Will wait (until the timeout provided) until the status of the cluster changes to the one provided or better, i.e. green > yellow > red. By default, will not wait for any status.
Values are green, GREEN, yellow, YELLOW, red, or RED.
curl \
--request GET 'http://api.example.com/_cluster/health' \
--header "Authorization: $API_KEY"{
"cluster_name" : "testcluster",
"status" : "yellow",
"timed_out" : false,
"number_of_nodes" : 1,
"number_of_data_nodes" : 1,
"active_primary_shards" : 1,
"active_shards" : 1,
"relocating_shards" : 0,
"initializing_shards" : 0,
"unassigned_shards" : 1,
"delayed_unassigned_shards": 0,
"number_of_pending_tasks" : 0,
"number_of_in_flight_fetch": 0,
"task_max_waiting_in_queue_millis": 0,
"active_shards_percent_as_number": 50.0
}Limits the information returned to the specific target. Supports a comma-separated list, such as http,ingest.
curl \
--request GET 'http://api.example.com/_info/{target}' \
--header "Authorization: $API_KEY"{
"cluster_name": "string",
"http": {
"current_open": 42.0,
"total_opened": 42.0,
"clients": [
{
"id": 42.0,
"agent": "string",
"local_address": "string",
"remote_address": "string",
"last_uri": "string",
"opened_time_millis": 42.0,
"closed_time_millis": 42.0,
"last_request_time_millis": 42.0,
"request_count": 42.0,
"request_size_bytes": 42.0,
"x_opaque_id": "string"
}
],
"routes": {
"additionalProperty1": {
"requests": {
"count": 42.0,
"total_size_in_bytes": 42.0,
"size_histogram": [
{}
]
},
"responses": {
"count": 42.0,
"total_size_in_bytes": 42.0,
"handling_time_histogram": [
{}
],
"size_histogram": [
{}
]
}
},
"additionalProperty2": {
"requests": {
"count": 42.0,
"total_size_in_bytes": 42.0,
"size_histogram": [
{}
]
},
"responses": {
"count": 42.0,
"total_size_in_bytes": 42.0,
"handling_time_histogram": [
{}
],
"size_histogram": [
{}
]
}
}
}
},
"ingest": {
"pipelines": {
"additionalProperty1": {
"count": 42.0,
"current": 42.0,
"failed": 42.0,
"processors": [
{
"additionalProperty1": {},
"additionalProperty2": {}
}
],
"": 42.0,
"ingested_as_first_pipeline_in_bytes": 42.0,
"produced_as_first_pipeline_in_bytes": 42.0
},
"additionalProperty2": {
"count": 42.0,
"current": 42.0,
"failed": 42.0,
"processors": [
{
"additionalProperty1": {},
"additionalProperty2": {}
}
],
"": 42.0,
"ingested_as_first_pipeline_in_bytes": 42.0,
"produced_as_first_pipeline_in_bytes": 42.0
}
},
"total": {
"count": 42.0,
"current": 42.0,
"failed": 42.0,
"": 42.0
}
},
"thread_pool": {
"additionalProperty1": {
"active": 42.0,
"completed": 42.0,
"largest": 42.0,
"queue": 42.0,
"rejected": 42.0,
"threads": 42.0
},
"additionalProperty2": {
"active": 42.0,
"completed": 42.0,
"largest": 42.0,
"queue": 42.0,
"rejected": 42.0,
"threads": 42.0
}
},
"script": {
"cache_evictions": 42.0,
"compilations": 42.0,
"compilations_history": {
"additionalProperty1": 42.0,
"additionalProperty2": 42.0
},
"compilation_limit_triggered": 42.0,
"contexts": [
{
"context": "string",
"compilations": 42.0,
"cache_evictions": 42.0,
"compilation_limit_triggered": 42.0
}
]
}
}By default, the API returns all attributes and core settings for cluster nodes.
Comma-separated list of node IDs or names used to limit returned information.
If true, returns settings in flat format.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_nodes/{node_id}' \
--header "Authorization: $API_KEY"{
"_nodes": {},
"cluster_name": "elasticsearch",
"nodes": {
"USpTGYaBSIKbgSUJR2Z9lg": {
"name": "node-0",
"transport_address": "192.168.17:9300",
"host": "node-0.elastic.co",
"ip": "192.168.17",
"version": "{version}",
"transport_version": 100000298,
"index_version": 100000074,
"component_versions": {
"ml_config_version": 100000162,
"transform_config_version": 100000096
},
"build_flavor": "default",
"build_type": "{build_type}",
"build_hash": "587409e",
"roles": [
"master",
"data",
"ingest"
],
"attributes": {},
"plugins": [
{
"name": "analysis-icu",
"version": "{version}",
"description": "The ICU Analysis plugin integrates Lucene ICU
module into elasticsearch, adding ICU relates analysis components.",
"classname":
"org.elasticsearch.plugin.analysis.icu.AnalysisICUPlugin",
"has_native_controller": false
}
],
"modules": [
{
"name": "lang-painless",
"version": "{version}",
"description": "An easy, safe and fast scripting language for
Elasticsearch",
"classname": "org.elasticsearch.painless.PainlessPlugin",
"has_native_controller": false
}
]
}
}
}Get a report with the health status of an Elasticsearch cluster. The report contains a list of indicators that compose Elasticsearch functionality.
Each indicator has a health status of: green, unknown, yellow or red. The indicator will provide an explanation and metadata describing the reason for its current health status.
The cluster’s status is controlled by the worst indicator status.
In the event that an indicator’s status is non-green, a list of impacts may be present in the indicator result which detail the functionalities that are negatively affected by the health issue. Each impact carries with it a severity level, an area of the system that is affected, and a simple description of the impact on the system.
Some health indicators can determine the root cause of a health problem and prescribe a set of steps that can be performed in order to improve the health of the system. The root cause and remediation steps are encapsulated in a diagnosis. A diagnosis contains a cause detailing a root cause analysis, an action containing a brief description of the steps to take to fix the problem, the list of affected resources (if applicable), and a detailed step-by-step troubleshooting guide to fix the diagnosed problem.
NOTE: The health indicators perform root cause analysis of non-green health statuses. This can be computationally expensive when called frequently. When setting up automated polling of the API for health status, set verbose to false to disable the more expensive analysis logic.
A feature of the cluster, as returned by the top-level health report API.
curl \
--request GET 'http://api.example.com/_health_report/{feature}' \
--header "Authorization: $API_KEY"{
"cluster_name": "string",
"indicators": {
"master_is_stable": {
"status": "green",
"symptom": "string",
"impacts": [
{
"description": "string",
"id": "string",
"impact_areas": [
"search"
],
"severity": 42.0
}
],
"diagnosis": [
{
"id": "string",
"action": "string",
"affected_resources": {},
"cause": "string",
"help_url": "string"
}
],
"details": {
"current_master": {},
"recent_masters": [
{}
],
"exception_fetching_history": {
"message": "string",
"stack_trace": "string"
},
"cluster_formation": [
{}
]
}
},
"shards_availability": {
"status": "green",
"symptom": "string",
"impacts": [
{
"description": "string",
"id": "string",
"impact_areas": [
"search"
],
"severity": 42.0
}
],
"diagnosis": [
{
"id": "string",
"action": "string",
"affected_resources": {},
"cause": "string",
"help_url": "string"
}
],
"details": {
"creating_primaries": 42.0,
"creating_replicas": 42.0,
"initializing_primaries": 42.0,
"initializing_replicas": 42.0,
"restarting_primaries": 42.0,
"restarting_replicas": 42.0,
"started_primaries": 42.0,
"started_replicas": 42.0,
"unassigned_primaries": 42.0,
"unassigned_replicas": 42.0
}
},
"disk": {
"status": "green",
"symptom": "string",
"impacts": [
{
"description": "string",
"id": "string",
"impact_areas": [
"search"
],
"severity": 42.0
}
],
"diagnosis": [
{
"id": "string",
"action": "string",
"affected_resources": {},
"cause": "string",
"help_url": "string"
}
],
"details": {
"indices_with_readonly_block": 42.0,
"nodes_with_enough_disk_space": 42.0,
"nodes_over_high_watermark": 42.0,
"nodes_over_flood_stage_watermark": 42.0,
"nodes_with_unknown_disk_status": 42.0
}
},
"repository_integrity": {
"status": "green",
"symptom": "string",
"impacts": [
{
"description": "string",
"id": "string",
"impact_areas": [
"search"
],
"severity": 42.0
}
],
"diagnosis": [
{
"id": "string",
"action": "string",
"affected_resources": {},
"cause": "string",
"help_url": "string"
}
],
"details": {
"total_repositories": 42.0,
"corrupted_repositories": 42.0,
"corrupted": [
"string"
]
}
},
"data_stream_lifecycle": {
"status": "green",
"symptom": "string",
"impacts": [
{
"description": "string",
"id": "string",
"impact_areas": [
"search"
],
"severity": 42.0
}
],
"diagnosis": [
{
"id": "string",
"action": "string",
"affected_resources": {},
"cause": "string",
"help_url": "string"
}
],
"details": {
"stagnating_backing_indices_count": 42.0,
"total_backing_indices_in_error": 42.0,
"stagnating_backing_indices": [
{}
]
}
},
"ilm": {
"status": "green",
"symptom": "string",
"impacts": [
{
"description": "string",
"id": "string",
"impact_areas": [
"search"
],
"severity": 42.0
}
],
"diagnosis": [
{
"id": "string",
"action": "string",
"affected_resources": {},
"cause": "string",
"help_url": "string"
}
],
"details": {
"ilm_status": "RUNNING",
"policies": 42.0,
"stagnating_indices": 42.0
}
},
"slm": {
"status": "green",
"symptom": "string",
"impacts": [
{
"description": "string",
"id": "string",
"impact_areas": [
"search"
],
"severity": 42.0
}
],
"diagnosis": [
{
"id": "string",
"action": "string",
"affected_resources": {},
"cause": "string",
"help_url": "string"
}
],
"details": {
"slm_status": "RUNNING",
"policies": 42.0,
"unhealthy_policies": {
"count": 42.0,
"invocations_since_last_success": {}
}
}
},
"shards_capacity": {
"status": "green",
"symptom": "string",
"impacts": [
{
"description": "string",
"id": "string",
"impact_areas": [
"search"
],
"severity": 42.0
}
],
"diagnosis": [
{
"id": "string",
"action": "string",
"affected_resources": {},
"cause": "string",
"help_url": "string"
}
],
"details": {
"data": {
"max_shards_in_cluster": 42.0,
"current_used_shards": 42.0
},
"frozen": {
"max_shards_in_cluster": 42.0,
"current_used_shards": 42.0
}
}
},
"file_settings": {
"status": "green",
"symptom": "string",
"impacts": [
{
"description": "string",
"id": "string",
"impact_areas": [
"search"
],
"severity": 42.0
}
],
"diagnosis": [
{
"id": "string",
"action": "string",
"affected_resources": {},
"cause": "string",
"help_url": "string"
}
],
"details": {
"failure_streak": 42.0,
"most_recent_failure": "string"
}
}
},
"status": "green"
}The connector and sync jobs APIs provide a convenient way to create and manage Elastic connectors and sync jobs in an internal index.
Connectors are Elasticsearch integrations for syncing content from third-party data sources, which can be deployed on Elastic Cloud or hosted on your own infrastructure.
This API provides an alternative to relying solely on Kibana UI for connector and sync job management. The API comes with a set of validations and assertions to ensure that the state representation in the internal index remains valid.
This API requires the manage_connector privilege or, for read-only endpoints, the monitor_connector privilege.
Starting offset (default: 0)
Specifies a max number of results to get
A comma-separated list of connector index names to fetch connector documents for
A comma-separated list of connector names to fetch connector documents for
A comma-separated list of connector service types to fetch connector documents for
A flag to indicate if the desired connector should be fetched, even if it was soft-deleted.
A wildcard query string that filters connectors with matching name, description or index name
curl \
--request GET 'http://api.example.com/_connector' \
--header "Authorization: $API_KEY"{
"count": 42.0,
"results": [
{
"api_key_id": "string",
"api_key_secret_id": "string",
"configuration": {
"additionalProperty1": {
"category": "string",
"": 42.0,
"depends_on": [
{}
],
"display": "textbox",
"label": "string",
"options": [
{}
],
"order": 42.0,
"placeholder": "string",
"required": true,
"sensitive": true,
"tooltip": "string",
"type": "str",
"ui_restrictions": [
"string"
],
"validations": [
{}
],
"value": {}
},
"additionalProperty2": {
"category": "string",
"": 42.0,
"depends_on": [
{}
],
"display": "textbox",
"label": "string",
"options": [
{}
],
"order": 42.0,
"placeholder": "string",
"required": true,
"sensitive": true,
"tooltip": "string",
"type": "str",
"ui_restrictions": [
"string"
],
"validations": [
{}
],
"value": {}
}
},
"custom_scheduling": {
"additionalProperty1": {
"configuration_overrides": {
"max_crawl_depth": 42.0,
"sitemap_discovery_disabled": true,
"domain_allowlist": [
"string"
],
"sitemap_urls": [
"string"
],
"seed_urls": [
"string"
]
},
"enabled": true,
"interval": "string",
"": "string",
"name": "string"
},
"additionalProperty2": {
"configuration_overrides": {
"max_crawl_depth": 42.0,
"sitemap_discovery_disabled": true,
"domain_allowlist": [
"string"
],
"sitemap_urls": [
"string"
],
"seed_urls": [
"string"
]
},
"enabled": true,
"interval": "string",
"": "string",
"name": "string"
}
},
"deleted": true,
"description": "string",
"error": "string",
"features": {
"document_level_security": {
"enabled": true
},
"incremental_sync": {
"enabled": true
},
"native_connector_api_keys": {
"enabled": true
},
"sync_rules": {
"advanced": {
"enabled": true
},
"basic": {
"enabled": true
}
}
},
"filtering": [
{
"active": {
"advanced_snippet": {},
"rules": [
{}
],
"validation": {}
},
"domain": "string",
"draft": {
"advanced_snippet": {},
"rules": [
{}
],
"validation": {}
}
}
],
"id": "string",
"index_name": "string",
"is_native": true,
"language": "string",
"last_access_control_sync_error": "string",
"": "string",
"last_access_control_sync_status": "canceling",
"last_deleted_document_count": 42.0,
"last_indexed_document_count": 42.0,
"last_sync_error": "string",
"last_sync_status": "canceling",
"name": "string",
"pipeline": {
"extract_binary_content": true,
"name": "string",
"reduce_whitespace": true,
"run_ml_inference": true
},
"scheduling": {
"access_control": {
"enabled": true,
"interval": "string"
},
"full": {
"enabled": true,
"interval": "string"
},
"incremental": {
"enabled": true,
"interval": "string"
}
},
"service_type": "string",
"status": "created",
"sync_cursor": {},
"sync_now": true
}
]
}Connectors are Elasticsearch integrations that bring content from third-party data sources, which can be deployed on Elastic Cloud or hosted on your own infrastructure. Elastic managed connectors (Native connectors) are a managed service on Elastic Cloud. Self-managed connectors (Connector clients) are self-managed on your infrastructure.
curl \
--request POST 'http://api.example.com/_connector' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"description":"string","index_name":"string","is_native":true,"language":"string","name":"string","service_type":"string"}'{
"description": "string",
"index_name": "string",
"is_native": true,
"language": "string",
"name": "string",
"service_type": "string"
}{
"result": "created",
"id": "string"
}Check in a connector sync job and set the last_seen field to the current time before updating it in the internal index.
To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.
The unique identifier of the connector sync job to be checked in.
curl \
--request PUT 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}/_check_in' \
--header "Authorization: $API_KEY"{}Set the error field for a connector sync job and set its status to error.
To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.
The unique identifier for the connector sync job.
curl \
--request PUT 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}/_error' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"error\": \"some-error\"\n}"'{
"error": "some-error"
}{}Activates the valid draft filtering for a connector.
The unique identifier of the connector to be updated
curl \
--request PUT 'http://api.example.com/_connector/{connector_id}/_filtering/_activate' \
--header "Authorization: $API_KEY"{
"result": "created"
}Get cross-cluster replication auto-follow patterns.
The auto-follow pattern collection that you want to retrieve. If you do not specify a name, the API returns information for all collections.
The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request GET 'http://api.example.com/_ccr/auto_follow/{name}' \
--header "Authorization: $API_KEY"{
"patterns": [
{
"name": "my_auto_follow_pattern",
"pattern": {
"active": true,
"remote_cluster" : "remote_cluster",
"leader_index_patterns" :
[
"leader_index*"
],
"leader_index_exclusion_patterns":
[
"leader_index_001"
],
"follow_index_pattern" : "{{leader_index}}-follower"
}
}
]
}Resume a cross-cluster replication auto-follow pattern that was paused. The auto-follow pattern will resume configuring following indices for newly created indices that match its patterns on the remote cluster. Remote indices created while the pattern was paused will also be followed unless they have been deleted or closed in the interim.
The name of the auto-follow pattern to resume.
The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request POST 'http://api.example.com/_ccr/auto_follow/{name}/resume' \
--header "Authorization: $API_KEY"{
"acknowledged" : true
}Convert a cross-cluster replication follower index to a regular index. The API stops the following task associated with a follower index and removes index metadata and settings associated with cross-cluster replication. The follower index must be paused and closed before you call the unfollow API.
Currently cross-cluster replication does not support converting an existing regular index to a follower index. Converting a follower index to a regular index is an irreversible operation.
The name of the follower index.
The period to wait for a connection to the master node.
If the master node is not available before the timeout expires, the request fails and returns an error.
It can also be set to -1 to indicate that the request should never timeout.
curl \
--request POST 'http://api.example.com/{index}/_ccr/unfollow' \
--header "Authorization: $API_KEY"{
"acknowledged" : true
}Deletes one or more data streams and their backing indices.
Comma-separated list of data streams to delete. Wildcard (*) expressions are supported.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Type of data stream that wildcard patterns can match. Supports comma-separated values,such as open,hidden.
curl \
--request DELETE 'http://api.example.com/_data_stream/{name}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}Get statistics about the data streams that are managed by a data stream lifecycle.
curl \
--request GET 'http://api.example.com/_lifecycle/stats' \
--header "Authorization: $API_KEY"{
"last_run_duration_in_millis": 2,
"last_run_duration": "2ms",
"time_between_starts_in_millis": 9998,
"time_between_starts": "9.99s",
"data_streams_count": 2,
"data_streams": [
{
"name": "my-data-stream",
"backing_indices_in_total": 2,
"backing_indices_in_error": 0
},
{
"name": "my-other-stream",
"backing_indices_in_total": 2,
"backing_indices_in_error": 1
}
]
}Converts an index alias to a data stream.
You must have a matching index template that is data stream enabled.
The alias must meet the following criteria:
The alias must have a write index;
All indices for the alias must have a @timestamp field mapping of a date or date_nanos field type;
The alias must not have any filters;
The alias must not use custom routing.
If successful, the request removes the alias and creates a data stream with the same name.
The indices for the alias become hidden backing indices for the stream.
The write index for the alias becomes the write index for the stream.
Name of the index alias to convert to a data stream.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request POST 'http://api.example.com/_data_stream/_migrate/{name}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}Add a JSON document to the specified data stream or index and make it searchable. If the target is an index and the document already exists, the request updates the document and increments its version.
NOTE: You cannot use this API to send update requests for existing documents in a data stream.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
PUT /<target>/_doc/<_id> request format, you must have the create, index, or write index privilege.POST /<target>/_doc/ request format, you must have the create_doc, create, index, or write index privilege.auto_configure, create_index, or manage index privilege.Automatic data stream creation requires a matching index template with data stream enabled.
NOTE: Replica shards might not all be started when an indexing operation returns successfully.
By default, only the primary is required. Set wait_for_active_shards to change this default behavior.
Automatically create data streams and indices
If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.
If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.
NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.
If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.
Automatic index creation is controlled by the action.auto_create_index setting.
If it is true, any index can be created automatically.
You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely.
Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked.
When a list is specified, the default behaviour is to disallow.
NOTE: The action.auto_create_index setting affects the automatic creation of indices only.
It does not affect the creation of data streams.
Optimistic concurrency control
Index operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no and if_primary_term parameters.
If a mismatch is detected, the operation will result in a VersionConflictException and a status code of 409.
Routing
By default, shard placement — or routing — is controlled by using a hash of the document's ID value.
For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.
When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself.
This does come at the (very minimal) cost of an additional document parsing pass.
If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.
Distributed
The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.
Active shards
To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation.
If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs.
By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1).
This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards.
To alter this behavior per operation, use the wait_for_active_shards request parameter.
Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1).
Specifying a negative value or a number greater than the number of shard copies will throw an error.
For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes).
If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding.
This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data.
If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding.
This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard.
However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index.
The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.
It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts.
After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary.
The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.
No operation (noop) updates
When updating a document by using this API, a new version of the document is always created even if the document hasn't changed.
If this isn't acceptable use the _update API with detect_noop set to true.
The detect_noop option isn't available on this API because it doesn’t fetch the old source and isn't able to compare it against the new source.
There isn't a definitive rule for when noop updates aren't acceptable. It's a combination of lots of factors like how frequently your data source sends updates that are actually noops and how many queries per second Elasticsearch runs on the shard receiving the updates.
Versioning
Each indexed document is given a version number.
By default, internal versioning is used that starts at 1 and increments with each update, deletes included.
Optionally, the version number can be set to an external value (for example, if maintained in a database).
To enable this functionality, version_type should be set to external.
The value provided must be a numeric, long value greater than or equal to 0, and less than around 9.2e+18.
NOTE: Versioning is completely real time, and is not affected by the near real time aspects of search operations. If no version is provided, the operation runs without any version checks.
When using the external version type, the system checks to see if the version number passed to the index request is greater than the version of the currently stored document. If true, the document will be indexed and the new version number used. If the value provided is less than or equal to the stored document's version number, a version conflict will occur and the index operation will fail. For example:
PUT my-index-000001/_doc/1?version=2&version_type=external
{
"user": {
"id": "elkbee"
}
}
In this example, the operation will succeed since the supplied version of 2 is higher than the current document version of 1.
If the document was already updated and its version was set to 2 or higher, the indexing command will fail and result in a conflict (409 HTTP status code).
A nice side effect is that there is no need to maintain strict ordering of async indexing operations run as a result of changes to a source database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from a database is simplified if external versioning is used, as only the latest version will be used if the index operations arrive out of order.
The name of the data stream or index to target.
If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream.
If the target doesn't exist and doesn't match a data stream template, this request creates the index.
You can check for existing targets with the resolve index API.
A unique identifier for the document.
To automatically generate a document ID, use the POST /<target>/_doc/ request format and omit this parameter.
Only perform the operation if the document has this primary term.
Only perform the operation if the document has this sequence number.
True or false if to include the document source in the error message in case of parsing errors.
Set to create to only index the document if it does not already exist (put if absent).
If a document with the specified _id already exists, the indexing operation will fail.
The behavior is the same as using the <index>/_create endpoint.
If a document ID is specified, this paramater defaults to index.
Otherwise, it defaults to create.
If the request targets a data stream, an op_type of create is required.
Values are index or create.
The ID of the pipeline to use to preprocess incoming documents.
If the index has a default ingest pipeline specified, then setting the value to _none disables the default ingest pipeline for this request.
If a final pipeline is configured it will always run, regardless of the value of this parameter.
If true, Elasticsearch refreshes the affected shards to make this operation visible to search.
If wait_for, it waits for a refresh to make this operation visible to search.
If false, it does nothing with refreshes.
Values are true, false, or wait_for.
A custom value that is used to route operations to a specific shard.
The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards.
This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
An explicit version number for concurrency control. It must be a non-negative long number.
The version type.
Values are internal, external, external_gte, or force.
The number of shard copies that must be active before proceeding with the operation.
You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default value of 1 means it waits for each primary shard to be active.
If true, the destination must be an index alias.
curl \
--request POST 'http://api.example.com/{index}/_doc/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"@timestamp\": \"2099-11-15T13:12:00\",\n \"message\": \"GET /search HTTP/1.1 200 1070000\",\n \"user\": {\n \"id\": \"kimchy\"\n }\n}"'{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}{
"@timestamp": "2099-11-15T13:12:00",
"message": "GET /search HTTP/1.1 200 1070000",
"user": {
"id": "kimchy"
}
}{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "W0tpsmIBdwcYyG50zbta",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"result": "created"
}{
"_shards": {
"total": 2,
"failed": 0,
"successful": 2
},
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"_seq_no": 0,
"_primary_term": 1,
"result": "created"
}Verify that a document exists.
For example, check to see if a document with the _id 0 exists:
HEAD my-index-000001/_doc/0
If the document exists, the API returns a status code of 200 - OK.
If the document doesn’t exist, the API returns 404 - Not Found.
Versioning support
You can use the version parameter to check the document only if its current version is equal to the specified one.
Internally, Elasticsearch has marked the old document as deleted and added an entirely new document. The old version of the document doesn't disappear immediately, although you won't be able to access it. Elasticsearch cleans up deleted documents in the background as you continue to index more data.
The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.
If it is set to _local, the operation will prefer to be run on a local allocated shard when possible.
If it is set to a custom value, the value is used to guarantee that the same shards will be used for the same custom value.
This can help with "jumping values" when hitting different shards in different refresh states.
A sample value can be something like the web session ID or the user name.
If true, the request is real-time as opposed to near-real-time.
If true, the request refreshes the relevant shards before retrieving the document.
Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).
A custom value used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or lists the fields to return.
A comma-separated list of source fields to exclude from the response.
You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter.
If the _source parameter is false, this parameter is ignored.
A comma-separated list of source fields to include in the response.
If this parameter is specified, only these source fields are returned.
You can exclude fields from this subset using the _source_excludes query parameter.
If the _source parameter is false, this parameter is ignored.
A comma-separated list of stored fields to return as part of a hit.
If no fields are specified, no stored fields are included in the response.
If this field is specified, the _source parameter defaults to false.
Explicit version number for concurrency control. The specified version must match the current version of the document for the request to succeed.
The version type.
Values are internal, external, external_gte, or force.
HEAD my-index-000001/_doc/0
curl -I "localhost:9200/my-index-000001/_doc/0?pretty"
const response = await client.exists({
index: "my-index-000001",
id: 0,
});
console.log(response);
resp = client.exists(
index="my-index-000001",
id="0",
)
print(resp)
response = client.exists(
index: 'my-index-000001',
id: 0
)
puts response
Get the source of a document. For example:
GET my-index-000001/_source/1
You can use the source filtering parameters to control which parts of the _source are returned:
GET my-index-000001/_source/1/?_source_includes=*.id&_source_excludes=entities
The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.
If true, the request is real-time as opposed to near-real-time.
If true, the request refreshes the relevant shards before retrieving the document.
Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).
A custom value used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or lists the fields to return.
A comma-separated list of source fields to exclude in the response.
A comma-separated list of source fields to include in the response.
A comma-separated list of stored fields to return as part of a hit.
The version number for concurrency control. It must match the current version of the document for the request to succeed.
The version type.
Values are internal, external, external_gte, or force.
curl \
--request GET 'http://api.example.com/{index}/_source/{id}' \
--header "Authorization: $API_KEY"{}Check whether a document source exists in an index. For example:
HEAD my-index-000001/_source/1
A document's source is not available if it is disabled in the mapping.
The node or shard the operation should be performed on. By default, the operation is randomized between the shard replicas.
If true, the request is real-time as opposed to near-real-time.
If true, the request refreshes the relevant shards before retrieving the document.
Setting it to true should be done after careful thought and verification that this does not cause a heavy load on the system (and slow down indexing).
A custom value used to route operations to a specific shard.
Indicates whether to return the _source field (true or false) or lists the fields to return.
A comma-separated list of source fields to exclude in the response.
A comma-separated list of source fields to include in the response.
The version number for concurrency control. It must match the current version of the document for the request to succeed.
The version type.
Values are internal, external, external_gte, or force.
curl \
--request HEAD 'http://api.example.com/{index}/_source/{id}' \
--header "Authorization: $API_KEY"Copy documents from a source to a destination. You can copy all documents to the destination index or reindex a subset of the documents. The source can be any existing index, alias, or data stream. The destination must differ from the source. For example, you cannot reindex a data stream into itself.
IMPORTANT: Reindex requires _source to be enabled for all documents in the source.
The destination should be configured as wanted before calling the reindex API.
Reindex does not copy the settings from the source or its associated template.
Mappings, shard counts, and replicas, for example, must be configured ahead of time.
If the Elasticsearch security features are enabled, you must have the following security privileges:
read index privilege for the source data stream, index, or alias.write index privilege for the destination data stream, index, or index alias.auto_configure, create_index, or manage index privilege for the destination data stream, index, or alias.source.remote.user must have the monitor cluster privilege and the read index privilege for the source data stream, index, or alias.If reindexing from a remote cluster, you must explicitly allow the remote host in the reindex.remote.whitelist setting.
Automatic data stream creation requires a matching index template with data stream enabled.
The dest element can be configured like the index API to control optimistic concurrency control.
Omitting version_type or setting it to internal causes Elasticsearch to blindly dump documents into the destination, overwriting any that happen to have the same ID.
Setting version_type to external causes Elasticsearch to preserve the version from the source, create any documents that are missing, and update any documents that have an older version in the destination than they do in the source.
Setting op_type to create causes the reindex API to create only missing documents in the destination.
All existing documents will cause a version conflict.
IMPORTANT: Because data streams are append-only, any reindex request to a destination data stream must have an op_type of create.
A reindex can only add new documents to a destination data stream.
It cannot update existing documents in a destination data stream.
By default, version conflicts abort the reindex process.
To continue reindexing if there are conflicts, set the conflicts request body property to proceed.
In this case, the response includes a count of the version conflicts that were encountered.
Note that the handling of other error types is unaffected by the conflicts property.
Additionally, if you opt to count version conflicts, the operation could attempt to reindex more documents from the source than max_docs until it has successfully indexed max_docs documents into the target or it has gone through every document in the source query.
NOTE: The reindex API makes no effort to handle ID collisions. The last document written will "win" but the order isn't usually predictable so it is not a good idea to rely on this behavior. Instead, make sure that IDs are unique by using a script.
Running reindex asynchronously
If the request contains wait_for_completion=false, Elasticsearch performs some preflight checks, launches the request, and returns a task you can use to cancel or get the status of the task.
Elasticsearch creates a record of this task as a document at _tasks/<task_id>.
Reindex from multiple sources
If you have many sources to reindex it is generally better to reindex them one at a time rather than using a glob pattern to pick up multiple sources. That way you can resume the process if there are any errors by removing the partially completed source and starting over. It also makes parallelizing the process fairly simple: split the list of sources to reindex and run each list in parallel.
For example, you can use a bash script like this:
for index in i1 i2 i3 i4 i5; do
curl -HContent-Type:application/json -XPOST localhost:9200/_reindex?pretty -d'{
"source": {
"index": "'$index'"
},
"dest": {
"index": "'$index'-reindexed"
}
}'
done
Throttling
Set requests_per_second to any positive decimal number (1.4, 6, 1000, for example) to throttle the rate at which reindex issues batches of index operations.
Requests are throttled by padding each batch with a wait time.
To turn off throttling, set requests_per_second to -1.
The throttling is done by waiting between batches so that the scroll that reindex uses internally can be given a timeout that takes into account the padding.
The padding time is the difference between the batch size divided by the requests_per_second and the time spent writing.
By default the batch size is 1000, so if requests_per_second is set to 500:
target_time = 1000 / 500 per second = 2 seconds
wait_time = target_time - write_time = 2 seconds - .5 seconds = 1.5 seconds
Since the batch is issued as a single bulk request, large batch sizes cause Elasticsearch to create many requests and then wait for a while before starting the next set. This is "bursty" instead of "smooth".
Slicing
Reindex supports sliced scroll to parallelize the reindexing process. This parallelization can improve efficiency and provide a convenient way to break the request down into smaller parts.
NOTE: Reindexing from remote clusters does not support manual or automatic slicing.
You can slice a reindex request manually by providing a slice ID and total number of slices to each request.
You can also let reindex automatically parallelize by using sliced scroll to slice on _id.
The slices parameter specifies the number of slices to use.
Adding slices to the reindex request just automates the manual process, creating sub-requests which means it has some quirks:
slices only contains the status of completed slices.slices will rethrottle the unfinished sub-request proportionally.slices will cancel each sub-request.slices, each sub-request won't get a perfectly even portion of the documents. All documents will be addressed, but some slices may be larger than others. Expect larger slices to have a more even distribution.requests_per_second and max_docs on a request with slices are distributed proportionally to each sub-request. Combine that with the previous point about distribution being uneven and you should conclude that using max_docs with slices might not result in exactly max_docs documents being reindexed.If slicing automatically, setting slices to auto will choose a reasonable number for most indices.
If slicing manually or otherwise tuning automatic slicing, use the following guidelines.
Query performance is most efficient when the number of slices is equal to the number of shards in the index.
If that number is large (for example, 500), choose a lower number as too many slices will hurt performance.
Setting slices higher than the number of shards generally does not improve efficiency and adds overhead.
Indexing performance scales linearly across available resources with the number of slices.
Whether query or indexing performance dominates the runtime depends on the documents being reindexed and cluster resources.
Modify documents during reindexing
Like _update_by_query, reindex operations support a script that modifies the document.
Unlike _update_by_query, the script is allowed to modify the document's metadata.
Just as in _update_by_query, you can set ctx.op to change the operation that is run on the destination.
For example, set ctx.op to noop if your script decides that the document doesn’t have to be indexed in the destination. This "no operation" will be reported in the noop counter in the response body.
Set ctx.op to delete if your script decides that the document must be deleted from the destination.
The deletion will be reported in the deleted counter in the response body.
Setting ctx.op to anything else will return an error, as will setting any other field in ctx.
Think of the possibilities! Just be careful; you are able to change:
_id_index_version_routingSetting _version to null or clearing it from the ctx map is just like not sending the version in an indexing request.
It will cause the document to be overwritten in the destination regardless of the version on the target or the version type you use in the reindex API.
Reindex from remote
Reindex supports reindexing from a remote Elasticsearch cluster.
The host parameter must contain a scheme, host, port, and optional path.
The username and password parameters are optional and when they are present the reindex operation will connect to the remote Elasticsearch node using basic authentication.
Be sure to use HTTPS when using basic authentication or the password will be sent in plain text.
There are a range of settings available to configure the behavior of the HTTPS connection.
When using Elastic Cloud, it is also possible to authenticate against the remote cluster through the use of a valid API key.
Remote hosts must be explicitly allowed with the reindex.remote.whitelist setting.
It can be set to a comma delimited list of allowed remote host and port combinations.
Scheme is ignored; only the host and port are used.
For example:
reindex.remote.whitelist: [otherhost:9200, another:9200, 127.0.10.*:9200, localhost:*"]
The list of allowed hosts must be configured on any nodes that will coordinate the reindex. This feature should work with remote clusters of any version of Elasticsearch. This should enable you to upgrade from any version of Elasticsearch to the current version by reindexing from a cluster of the old version.
WARNING: Elasticsearch does not support forward compatibility across major versions. For example, you cannot reindex from a 7.x cluster into a 6.x cluster.
To enable queries sent to older versions of Elasticsearch, the query parameter is sent directly to the remote host without validation or modification.
NOTE: Reindexing from remote clusters does not support manual or automatic slicing.
Reindexing from a remote server uses an on-heap buffer that defaults to a maximum size of 100mb.
If the remote index includes very large documents you'll need to use a smaller batch size.
It is also possible to set the socket read timeout on the remote connection with the socket_timeout field and the connection timeout with the connect_timeout field.
Both default to 30 seconds.
Configuring SSL parameters
Reindex from remote supports configurable SSL settings.
These must be specified in the elasticsearch.yml file, with the exception of the secure settings, which you add in the Elasticsearch keystore.
It is not possible to configure SSL in the body of the reindex request.
If true, the request refreshes affected shards to make this operation visible to search.
The throttle for this request in sub-requests per second. By default, there is no throttle.
The period of time that a consistent view of the index should be maintained for scrolled search.
The number of slices this task should be divided into. It defaults to one slice, which means the task isn't sliced into subtasks.
Reindex supports sliced scroll to parallelize the reindexing process. This parallelization can improve efficiency and provide a convenient way to break the request down into smaller parts.
NOTE: Reindexing from remote clusters does not support manual or automatic slicing.
If set to auto, Elasticsearch chooses the number of slices to use.
This setting will use one slice per shard, up to a certain limit.
If there are multiple sources, it will choose the number of slices based on the index or backing index with the smallest number of shards.
The period each indexing waits for automatic index creation, dynamic mapping updates, and waiting for active shards. By default, Elasticsearch waits for at least one minute before failing. The actual wait time could be longer, particularly when multiple waits occur.
The number of shard copies that must be active before proceeding with the operation.
Set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
The default value is one, which means it waits for each primary shard to be active.
If true, the request blocks until the operation is complete.
If true, the destination must be an index alias.
Values are abort or proceed.
The maximum number of documents to reindex.
By default, all documents are reindexed.
If it is a value less then or equal to scroll_size, a scroll will not be used to retrieve the results for the operation.
If conflicts is set to proceed, the reindex operation could attempt to reindex more documents from the source than max_docs until it has successfully indexed max_docs documents into the target or it has gone through every document in the source query.
curl \
--request POST 'http://api.example.com/_reindex' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"source\": {\n \"index\": [\"my-index-000001\", \"my-index-000002\"]\n },\n \"dest\": {\n \"index\": \"my-new-index-000002\"\n }\n}"'{
"source": {
"index": ["my-index-000001", "my-index-000002"]
},
"dest": {
"index": "my-new-index-000002"
}
}{
"source": {
"index": "metricbeat-*"
},
"dest": {
"index": "metricbeat"
},
"script": {
"lang": "painless",
"source": "ctx._index = 'metricbeat-' + (ctx._index.substring('metricbeat-'.length(), ctx._index.length())) + '-1'"
}
}{
"max_docs": 10,
"source": {
"index": "my-index-000001",
"query": {
"function_score" : {
"random_score" : {},
"min_score" : 0.9
}
}
},
"dest": {
"index": "my-new-index-000001"
}
}{
"source": {
"index": "my-index-000001"
},
"dest": {
"index": "my-new-index-000001",
"version_type": "external"
},
"script": {
"source": "if (ctx._source.foo == 'bar') {ctx._version++; ctx._source.remove('foo')}",
"lang": "painless"
}
}{
"source": {
"remote": {
"host": "http://otherhost:9200",
"username": "user",
"password": "pass"
},
"index": "my-index-000001",
"query": {
"match": {
"test": "data"
}
}
},
"dest": {
"index": "my-new-index-000001"
}
}{
"source": {
"index": "my-index-000001",
"slice": {
"id": 0,
"max": 2
}
},
"dest": {
"index": "my-new-index-000001"
}
}{
"source": {
"index": "my-index-000001"
},
"dest": {
"index": "my-new-index-000001"
}
}{
"source": {
"index": "source",
"query": {
"match": {
"company": "cat"
}
}
},
"dest": {
"index": "dest",
"routing": "=cat"
}
}{
"source": {
"index": "source"
},
"dest": {
"index": "dest",
"pipeline": "some_ingest_pipeline"
}
}{
"source": {
"index": "my-index-000001",
"query": {
"term": {
"user.id": "kimchy"
}
}
},
"dest": {
"index": "my-new-index-000001"
}
}{
"max_docs": 1,
"source": {
"index": "my-index-000001"
},
"dest": {
"index": "my-new-index-000001"
}
}{
"source": {
"index": "my-index-000001",
"_source": ["user.id", "_doc"]
},
"dest": {
"index": "my-new-index-000001"
}
}{
"source": {
"index": "my-index-000001"
},
"dest": {
"index": "my-new-index-000001"
},
"script": {
"source": "ctx._source.tag = ctx._source.remove(\"flag\")"
}
}{
"batches": 42.0,
"created": 42.0,
"deleted": 42.0,
"failures": [
{
"cause": {
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
},
"id": "string",
"index": "string",
"status": 42.0
}
],
"noops": 42.0,
"retries": {
"bulk": 42.0,
"search": 42.0
},
"requests_per_second": 42.0,
"slice_id": 42.0,
"": 42.0,
"timed_out": true,
"total": 42.0,
"updated": 42.0,
"version_conflicts": 42.0
}Get information and statistics about terms in the fields of a particular document.
You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request.
You can specify the fields you are interested in through the fields parameter or by adding the fields to the request body.
For example:
GET /my-index-000001/_termvectors/1?fields=message
Fields can be specified using wildcards, similar to the multi match query.
Term vectors are real-time by default, not near real-time.
This can be changed by setting realtime parameter to false.
You can request three types of values: term information, term statistics, and field statistics. By default, all term information and field statistics are returned for all fields but term statistics are excluded.
Term information
positions: true)offsets: true)payloads: true), as base64 encoded bytesIf the requested information wasn't stored in the index, it will be computed on the fly if possible. Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.
Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.
Behaviour
The term and field statistics are not accurate.
Deleted documents are not taken into account.
The information is only retrieved for the shard the requested document resides in.
The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context.
By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected.
Use routing only to hit a particular shard.
A comma-separated list or wildcard expressions of fields to include in the statistics.
It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.
If true, the response includes:
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
The node or shard the operation should be performed on. It is random by default.
If true, the request is real-time as opposed to near-real-time.
A custom value that is used to route operations to a specific shard.
If true, the response includes:
By default these values are not returned since term statistics can have a serious performance impact.
If true, returns the document version as part of a hit.
The version type.
Values are internal, external, external_gte, or force.
An artificial document (a document not present in the index) for which you want to retrieve term vectors.
Override the default per-field analyzer. This is useful in order to generate term vectors in any fashion, especially when using artificial documents. When providing an analyzer for a field that already stores term vectors, the term vectors will be regenerated.
If true, the response includes:
If true, the response includes term offsets.
If true, the response includes term payloads.
If true, the response includes term positions.
If true, the response includes:
By default these values are not returned since term statistics can have a serious performance impact.
Values are internal, external, external_gte, or force.
curl \
--request GET 'http://api.example.com/{index}/_termvectors/{id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"fields\" : [\"text\"],\n \"offsets\" : true,\n \"payloads\" : true,\n \"positions\" : true,\n \"term_statistics\" : true,\n \"field_statistics\" : true\n}"'{
"fields" : ["text"],
"offsets" : true,
"payloads" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
},
"fields": ["fullname"],
"per_field_analyzer" : {
"fullname": "keyword"
}
}{
"doc": {
"plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
},
"term_statistics": true,
"field_statistics": true,
"positions": false,
"offsets": false,
"filter": {
"max_num_terms": 3,
"min_term_freq": 1,
"min_doc_freq": 1
}
}{
"fields" : ["text", "some_field_without_term_vectors"],
"offsets" : true,
"positions" : true,
"term_statistics" : true,
"field_statistics" : true
}{
"doc" : {
"fullname" : "John Doe",
"text" : "test test test"
}
}{
"_index": "my-index-000001",
"_id": "1",
"_version": 1,
"found": true,
"took": 6,
"term_vectors": {
"text": {
"field_statistics": {
"sum_doc_freq": 4,
"doc_count": 2,
"sum_ttf": 6
},
"terms": {
"test": {
"doc_freq": 2,
"ttf": 4,
"term_freq": 3,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 4,
"payload": "d29yZA=="
},
{
"position": 1,
"start_offset": 5,
"end_offset": 9,
"payload": "d29yZA=="
},
{
"position": 2,
"start_offset": 10,
"end_offset": 14,
"payload": "d29yZA=="
}
]
}
}
}
}
}{
"_index": "my-index-000001",
"_version": 0,
"found": true,
"took": 6,
"term_vectors": {
"fullname": {
"field_statistics": {
"sum_doc_freq": 2,
"doc_count": 4,
"sum_ttf": 4
},
"terms": {
"John Doe": {
"term_freq": 1,
"tokens": [
{
"position": 0,
"start_offset": 0,
"end_offset": 8
}
]
}
}
}
}
}{
"_index": "imdb",
"_version": 0,
"found": true,
"term_vectors": {
"plot": {
"field_statistics": {
"sum_doc_freq": 3384269,
"doc_count": 176214,
"sum_ttf": 3753460
},
"terms": {
"armored": {
"doc_freq": 27,
"ttf": 27,
"term_freq": 1,
"score": 9.74725
},
"industrialist": {
"doc_freq": 88,
"ttf": 88,
"term_freq": 1,
"score": 8.590818
},
"stark": {
"doc_freq": 44,
"ttf": 47,
"term_freq": 1,
"score": 9.272792
}
}
}
}
}Create the enrich index for an existing enrich policy.
Enrich policy to execute.
Period to wait for a connection to the master node.
If true, the request blocks other enrich policy execution requests until complete.
curl \
--request PUT 'http://api.example.com/_enrich/policy/{name}/_execute' \
--header "Authorization: $API_KEY"{
"status": {
"phase": "SCHEDULED",
"step": "string"
},
"": "string"
}Returns an object containing IDs and other information about the running ES|QL queries.
curl \
--request GET 'http://api.example.com/_query/queries' \
--header "Authorization: $API_KEY"{
"queries": {
"additionalProperty1": {
"id": 42.0,
"node": "string",
"start_time_millis": 42.0,
"running_time_nanos": 42.0,
"query": "string"
},
"additionalProperty2": {
"id": 42.0,
"node": "string",
"start_time_millis": 42.0,
"running_time_nanos": 42.0,
"query": "string"
}
}
}Get the current global checkpoints for an index. This API is designed for internal use by the Fleet server project.
A single index or index alias that resolves to a single index.
A boolean value which controls whether to wait (until the timeout) for the global checkpoints
to advance past the provided checkpoints.
A boolean value which controls whether to wait (until the timeout) for the target index to exist
and all primary shards be active. Can only be true when wait_for_advance is true.
A comma separated list of previous global checkpoints. When used in combination with wait_for_advance,
the API will only return once the global checkpoints advances past the checkpoints. Providing an empty list
will cause Elasticsearch to immediately return the current global checkpoints.
Period to wait for a global checkpoints to advance past checkpoints.
curl \
--request GET 'http://api.example.com/{index}/_fleet/global_checkpoints' \
--header "Authorization: $API_KEY"{
"global_checkpoints": [
42.0
],
"timed_out": true
}Component templates are building blocks for constructing index templates that specify index mappings, settings, and aliases.
An index template can be composed of multiple component templates.
To use a component template, specify it in an index template’s composed_of list.
Component templates are only applied to new data streams and indices as part of a matching index template.
Settings and mappings specified directly in the index template or the create index request override any settings or mappings specified in a component template.
Component templates are only used during index creation. For data streams, this includes data stream creation and the creation of a stream’s backing indices. Changes to component templates do not affect existing indices, including a stream’s backing indices.
You can use C-style /* *\/ block comments in component templates.
You can include comments anywhere in the request body except before the opening curly bracket.
Applying component templates
You cannot directly apply a component template to a data stream or index.
To be applied, a component template must be included in an index template's composed_of list.
Name of the component template to create.
Elasticsearch includes the following built-in component templates: logs-mappings; logs-settings; metrics-mappings; metrics-settings;synthetics-mapping; synthetics-settings.
Elastic Agent uses these templates to configure backing indices for its data streams.
If you use Elastic Agent and want to overwrite one of these templates, set the version for your replacement template higher than the current version.
If you don’t use Elastic Agent and want to disable all built-in component and index templates, set stack.templates.enabled to false using the cluster update settings API.
If true, this request cannot replace or update existing component templates.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.
curl \
--request POST 'http://api.example.com/_component_template/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"template\": null,\n \"settings\": {\n \"number_of_shards\": 1\n },\n \"mappings\": {\n \"_source\": {\n \"enabled\": false\n },\n \"properties\": {\n \"host_name\": {\n \"type\": \"keyword\"\n },\n \"created_at\": {\n \"type\": \"date\",\n \"format\": \"EEE MMM dd HH:mm:ss Z yyyy\"\n }\n }\n }\n}"'{
"template": null,
"settings": {
"number_of_shards": 1
},
"mappings": {
"_source": {
"enabled": false
},
"properties": {
"host_name": {
"type": "keyword"
},
"created_at": {
"type": "date",
"format": "EEE MMM dd HH:mm:ss Z yyyy"
}
}
}
}{
"template": null,
"settings": {
"number_of_shards": 1
},
"aliases": {
"alias1": {},
"alias2": {
"filter": {
"term": {
"user.id": "kimchy"
}
},
"routing": "shard-1"
},
"{index}-alias": {}
}
}{
"acknowledged": true
}Clear the cache of one or more indices. For data streams, the API clears the caches of the stream's backing indices.
By default, the clear cache API clears all caches.
To clear only specific caches, use the fielddata, query, or request parameters.
To clear the cache only of specific fields, use the fields parameter.
Comma-separated list of data streams, indices, and aliases used to limit the request.
Supports wildcards (*).
To target all data streams and indices, omit this parameter or use * or _all.
Comma-separated list of data streams, indices, and aliases used to limit the request.
Supports wildcards (*).
To target all data streams and indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
If true, clears the fields cache.
Use the fields parameter to clear the cache of specific fields only.
Comma-separated list of field names used to limit the fielddata parameter.
If true, clears the query cache.
If true, clears the request cache.
curl \
--request POST 'http://api.example.com/{index}/_cache/clear' \
--header "Authorization: $API_KEY"{
"_shards": {
"failed": 42.0,
"successful": 42.0,
"total": 42.0,
"failures": [
{
"index": "string",
"node": "string",
"reason": {
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
},
"shard": 42.0,
"status": "string"
}
],
"skipped": 42.0
}
}Clone an existing index into a new index. Each original primary shard is cloned into a new primary shard in the new index.
IMPORTANT: Elasticsearch does not apply index templates to the resulting index. The API also does not copy index metadata from the original index. Index metadata includes aliases, index lifecycle management phase definitions, and cross-cluster replication (CCR) follower information. For example, if you clone a CCR follower index, the resulting clone will not be a follower index.
The clone API copies most index settings from the source index to the resulting index, with the exception of index.number_of_replicas and index.auto_expand_replicas.
To set the number of replicas in the resulting index, configure these settings in the clone request.
Cloning works as follows:
IMPORTANT: Indices can only be cloned if they meet the following requirements:
The current write index on a data stream cannot be cloned. In order to clone the current write index, the data stream must first be rolled over so that a new write index is created and then the previous write index can be cloned.
NOTE: Mappings cannot be specified in the _clone request. The mappings of the source index will be used for the target index.
Monitor the cloning process
The cloning process can be monitored with the cat recovery API or the cluster health API can be used to wait until all primary shards have been allocated by setting the wait_for_status parameter to yellow.
The _clone API returns as soon as the target index has been added to the cluster state, before any shards have been allocated.
At this point, all shards are in the state unassigned.
If, for any reason, the target index can't be allocated, its primary shard will remain unassigned until it can be allocated on that node.
Once the primary shard is allocated, it moves to state initializing, and the clone process begins. When the clone operation completes, the shard will become active. At that point, Elasticsearch will try to allocate any replicas and may decide to relocate the primary shard to another node.
Wait for active shards
Because the clone operation creates a new index to clone the shards to, the wait for active shards setting on index creation applies to the clone index action as well.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
curl \
--request PUT 'http://api.example.com/{index}/_clone/{target}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"settings\": {\n \"index.number_of_shards\": 5\n },\n \"aliases\": {\n \"my_search_indices\": {}\n }\n}"'{
"settings": {
"index.number_of_shards": 5
},
"aliases": {
"my_search_indices": {}
}
}{
"acknowledged": true,
"index": "string",
"shards_acknowledged": true
}Index templates define settings, mappings, and aliases that can be applied automatically to new indices.
Elasticsearch applies templates to new indices based on an wildcard pattern that matches the index name. Index templates are applied during data stream or index creation. For data streams, these settings and mappings are applied when the stream's backing indices are created. Settings and mappings specified in a create index API request override any settings or mappings specified in an index template. Changes to index templates do not affect existing indices, including the existing backing indices of a data stream.
You can use C-style /* *\/ block comments in index templates.
You can include comments anywhere in the request body, except before the opening curly bracket.
Multiple matching templates
If multiple index templates match the name of a new index or data stream, the template with the highest priority is used.
Multiple templates with overlapping index patterns at the same priority are not allowed and an error will be thrown when attempting to create a template matching an existing index template at identical priorities.
Composing aliases, mappings, and settings
When multiple component templates are specified in the composed_of field for an index template, they are merged in the order specified, meaning that later component templates override earlier component templates.
Any mappings, settings, or aliases from the parent index template are merged in next.
Finally, any configuration on the index request itself is merged.
Mapping definitions are merged recursively, which means that later mapping components can introduce new field mappings and update the mapping configuration.
If a field mapping is already contained in an earlier component, its definition will be completely overwritten by the later one.
This recursive merging strategy applies not only to field mappings, but also root options like dynamic_templates and meta.
If an earlier component contains a dynamic_templates block, then by default new dynamic_templates entries are appended onto the end.
If an entry already exists with the same key, then it is overwritten by the new definition.
Index or template name
If true, this request cannot replace or update existing index templates.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
User defined reason for creating/updating the index template
An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.
Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.
This setting overrides the value of the action.auto_create_index cluster setting.
If set to true in a template, then indices can be automatically created using that template even if auto-creation of indices is disabled via actions.auto_create_index.
If set to false, then indices or data streams matching the template must always be explicitly created, and may never be automatically created.
The configuration option ignore_missing_component_templates can be used when an index template references a component template that might not exist
Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.
curl \
--request PUT 'http://api.example.com/_index_template/{name}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index_patterns\" : [\"template*\"],\n \"priority\" : 1,\n \"template\": {\n \"settings\" : {\n \"number_of_shards\" : 2\n }\n }\n}"'{
"index_patterns" : ["template*"],
"priority" : 1,
"template": {
"settings" : {
"number_of_shards" : 2
}
}
}{
"index_patterns": [
"template*"
],
"template": {
"settings": {
"number_of_shards": 1
},
"aliases": {
"alias1": {},
"alias2": {
"filter": {
"term": {
"user.id": "kimchy"
}
},
"routing": "shard-1"
},
"{index}-alias": {}
}
}
}{
"acknowledged": true
}Flushing a data stream or index is the process of making sure that any data that is currently only stored in the transaction log is also permanently stored in the Lucene index. When restarting, Elasticsearch replays any unflushed operations from the transaction log into the Lucene index to bring it back into the state that it was in before the restart. Elasticsearch automatically triggers flushes as needed, using heuristics that trade off the size of the unflushed transaction log against the cost of performing each flush.
After each operation has been flushed it is permanently stored in the Lucene index. This may mean that there is no need to maintain an additional copy of it in the transaction log. The transaction log is made up of multiple files, called generations, and Elasticsearch will delete any generation files when they are no longer needed, freeing up disk space.
It is also possible to trigger a flush on one or more indices using the flush API, although it is rare for users to need to call this API directly. If you call the flush API after indexing some documents then a successful response indicates that Elasticsearch has flushed all the documents that were indexed before the flush API was called.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
If true, the request forces a flush even if there are no changes to commit to the index.
If true, the flush operation blocks until execution when another flush operation is running.
If false, Elasticsearch returns an error if you request a flush when another flush operation is running.
curl \
--request GET 'http://api.example.com/_flush' \
--header "Authorization: $API_KEY"{
"_shards": {
"failed": 42.0,
"successful": 42.0,
"total": 42.0,
"failures": [
{
"index": "string",
"node": "string",
"reason": {
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
},
"shard": 42.0,
"status": "string"
}
],
"skipped": 42.0
}
}Changes dynamic index settings in real time. For data streams, index setting changes are applied to all backing indices by default.
To revert a setting to the default value, use a null value.
The list of per-index settings that can be updated dynamically on live indices can be found in index module documentation.
To preserve existing settings from being updated, set the preserve_existing parameter to true.
NOTE: You can only define new analyzers on closed indices. To add an analyzer, you must close the index, define the analyzer, and reopen the index. You cannot close the write index of a data stream. To update the analyzer for a data stream's write index and future backing indices, update the analyzer in the index template used by the stream. Then roll over the data stream to apply the new analyzer to the stream's write index and future backing indices. This affects searches and any new data added to the stream after the rollover. However, it does not affect the data stream's backing indices or their existing data. To change the analyzer for existing backing indices, you must create a new data stream and reindex your data into it.
Comma-separated list of data streams, indices, and aliases used to limit
the request. Supports wildcards (*). To target all data streams and
indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index
alias, or _all value targets only missing or closed indices. This
behavior applies even if the request targets other open indices. For
example, a request targeting foo*,bar* returns an error if an index
starts with foo but no index starts with bar.
Type of index that wildcard patterns can match. If the request can target
data streams, this argument determines whether wildcard expressions match
hidden data streams. Supports comma-separated values, such as
open,hidden.
If true, returns settings in flat format.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
If true, existing index settings remain unchanged.
Whether to close and reopen the index to apply non-dynamic settings.
If set to true the indices to which the settings are being applied
will be closed temporarily and then reopened in order to apply the changes.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Values are true, false, or checksum.
Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and
d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
Time unit for milliseconds
A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
Configure custom similarity settings to customize how search results are scored.
curl \
--request PUT 'http://api.example.com/{index}/_settings' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index\" : {\n \"number_of_replicas\" : 2\n }\n}"'{
"index" : {
"number_of_replicas" : 2
}
}{
"index" : {
"refresh_interval" : null
}
}{
"analysis" : {
"analyzer":{
"content":{
"type":"custom",
"tokenizer":"whitespace"
}
}
}
}
POST /my-index-000001/_open{
"acknowledged": true
}Get information about ongoing and completed shard recoveries for one or more indices. For data streams, the API returns information for the stream's backing indices.
All recoveries, whether ongoing or complete, are kept in the cluster state and may be reported on at any time.
Shard recovery is the process of initializing a shard copy, such as restoring a primary shard from a snapshot or creating a replica shard from a primary shard. When a shard recovery completes, the recovered shard is available for search and indexing.
Recovery automatically occurs during the following processes:
You can determine the cause of a shard recovery using the recovery or cat recovery APIs.
The index recovery API reports information about completed recoveries only for shard copies that currently exist in the cluster. It only reports the last recovery for each shard copy and does not report historical information about earlier recoveries, nor does it report information about the recoveries of shard copies that no longer exist. This means that if a shard copy completes a recovery and then Elasticsearch relocates it onto a different node then the information about the original recovery will not be shown in the recovery API.
Comma-separated list of data streams, indices, and aliases used to limit the request.
Supports wildcards (*).
To target all data streams and indices, omit this parameter or use * or _all.
If true, the response only includes ongoing shard recoveries.
If true, the response includes detailed information about shard recoveries.
curl \
--request GET 'http://api.example.com/{index}/_recovery' \
--header "Authorization: $API_KEY"{
"index1" : {
"shards" : [ {
"id" : 0,
"type" : "SNAPSHOT",
"stage" : "INDEX",
"primary" : true,
"start_time" : "2014-02-24T12:15:59.716",
"start_time_in_millis": 1393244159716,
"stop_time" : "0s",
"stop_time_in_millis" : 0,
"total_time" : "2.9m",
"total_time_in_millis" : 175576,
"source" : {
"repository" : "my_repository",
"snapshot" : "my_snapshot",
"index" : "index1",
"version" : "{version}",
"restoreUUID": "PDh1ZAOaRbiGIVtCvZOMww"
},
"target" : {
"id" : "ryqJ5lO5S4-lSFbGntkEkg",
"host" : "my.fqdn",
"transport_address" : "my.fqdn",
"ip" : "10.0.1.7",
"name" : "my_es_node"
},
"index" : {
"size" : {
"total" : "75.4mb",
"total_in_bytes" : 79063092,
"reused" : "0b",
"reused_in_bytes" : 0,
"recovered" : "65.7mb",
"recovered_in_bytes" : 68891939,
"recovered_from_snapshot" : "0b",
"recovered_from_snapshot_in_bytes" : 0,
"percent" : "87.1%"
},
"files" : {
"total" : 73,
"reused" : 0,
"recovered" : 69,
"percent" : "94.5%"
},
"total_time" : "0s",
"total_time_in_millis" : 0,
"source_throttle_time" : "0s",
"source_throttle_time_in_millis" : 0,
"target_throttle_time" : "0s",
"target_throttle_time_in_millis" : 0
},
"translog" : {
"recovered" : 0,
"total" : 0,
"percent" : "100.0%",
"total_on_start" : 0,
"total_time" : "0s",
"total_time_in_millis" : 0
},
"verify_index" : {
"check_index_time" : "0s",
"check_index_time_in_millis" : 0,
"total_time" : "0s",
"total_time_in_millis" : 0
}
} ]
}
}{
"index1" : {
"shards" : [ {
"id" : 0,
"type" : "EXISTING_STORE",
"stage" : "DONE",
"primary" : true,
"start_time" : "2014-02-24T12:38:06.349",
"start_time_in_millis" : "1393245486349",
"stop_time" : "2014-02-24T12:38:08.464",
"stop_time_in_millis" : "1393245488464",
"total_time" : "2.1s",
"total_time_in_millis" : 2115,
"source" : {
"id" : "RGMdRc-yQWWKIBM4DGvwqQ",
"host" : "my.fqdn",
"transport_address" : "my.fqdn",
"ip" : "10.0.1.7",
"name" : "my_es_node"
},
"target" : {
"id" : "RGMdRc-yQWWKIBM4DGvwqQ",
"host" : "my.fqdn",
"transport_address" : "my.fqdn",
"ip" : "10.0.1.7",
"name" : "my_es_node"
},
"index" : {
"size" : {
"total" : "24.7mb",
"total_in_bytes" : 26001617,
"reused" : "24.7mb",
"reused_in_bytes" : 26001617,
"recovered" : "0b",
"recovered_in_bytes" : 0,
"recovered_from_snapshot" : "0b",
"recovered_from_snapshot_in_bytes" : 0,
"percent" : "100.0%"
},
"files" : {
"total" : 26,
"reused" : 26,
"recovered" : 0,
"percent" : "100.0%",
"details" : [ {
"name" : "segments.gen",
"length" : 20,
"recovered" : 20
}, {
"name" : "_0.cfs",
"length" : 135306,
"recovered" : 135306,
"recovered_from_snapshot": 0
}, {
"name" : "segments_2",
"length" : 251,
"recovered" : 251,
"recovered_from_snapshot": 0
}
]
},
"total_time" : "2ms",
"total_time_in_millis" : 2,
"source_throttle_time" : "0s",
"source_throttle_time_in_millis" : 0,
"target_throttle_time" : "0s",
"target_throttle_time_in_millis" : 0
},
"translog" : {
"recovered" : 71,
"total" : 0,
"percent" : "100.0%",
"total_on_start" : 0,
"total_time" : "2.0s",
"total_time_in_millis" : 2025
},
"verify_index" : {
"check_index_time" : 0,
"check_index_time_in_millis" : 0,
"total_time" : "88ms",
"total_time_in_millis" : 88
}
} ]
}
}A refresh makes recent operations performed on one or more indices available for search. For data streams, the API runs the refresh operation on the stream’s backing indices.
By default, Elasticsearch periodically refreshes indices every second, but only on indices that have received one search request or more in the last 30 seconds.
You can change this default interval with the index.refresh_interval setting.
Refresh requests are synchronous and do not return a response until the refresh operation completes.
Refreshes are resource-intensive. To ensure good cluster performance, it's recommended to wait for Elasticsearch's periodic refresh rather than performing an explicit refresh when possible.
If your application workflow indexes documents and then runs a search to retrieve the indexed document, it's recommended to use the index API's refresh=wait_for query parameter option.
This option ensures the indexing operation waits for a periodic refresh before running the search.
Comma-separated list of data streams, indices, and aliases used to limit the request.
Supports wildcards (*).
To target all data streams and indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
curl \
--request GET 'http://api.example.com/{index}/_refresh' \
--header "Authorization: $API_KEY"{
"_shards": {
"failed": 42.0,
"successful": 42.0,
"total": 42.0,
"failures": [
{
"index": "string",
"node": "string",
"reason": {
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
},
"shard": 42.0,
"status": "string"
}
],
"skipped": 42.0
}
}Get store information about replica shards in one or more indices. For data streams, the API retrieves store information for the stream's backing indices.
The index shard stores API returns the following information:
By default, the API returns store information only for primary shards that are unassigned or have one or more unassigned replica shards.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.
Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
List of shard health statuses used to limit the request.
curl \
--request GET 'http://api.example.com/_shard_stores' \
--header "Authorization: $API_KEY"{
"indices": {
"my-index-000001": {
"shards": {
"0": {
"stores": [
{
"sPa3OgxLSYGvQ4oPs-Tajw": {
"name": "node_t0",
"ephemeral_id": "9NlXRFGCT1m8tkvYCMK-8A",
"transport_address": "local[1]",
"external_id": "node_t0",
"attributes": {},
"roles": [],
"version": "8.10.0",
"min_index_version": 7000099,
"max_index_version": 8100099
},
"allocation_id": "2iNySv_OQVePRX-yaRH_lQ",
"allocation": "primary",
"store_exception": {}
}
]
}
}
}
}
}Shrink an index into a new index with fewer primary shards.
Before you can shrink an index:
To make shard allocation easier, we recommend you also remove the index's replica shards. You can later re-add replica shards as part of the shrink operation.
The requested number of primary shards in the target index must be a factor of the number of shards in the source index. For example an index with 8 primary shards can be shrunk into 4, 2 or 1 primary shards or an index with 15 primary shards can be shrunk into 5, 3 or 1. If the number of shards in the index is a prime number it can only be shrunk into a single primary shard Before shrinking, a (primary or replica) copy of every shard in the index must be present on the same node.
The current write index on a data stream cannot be shrunk. In order to shrink the current write index, the data stream must first be rolled over so that a new write index is created and then the previous write index can be shrunk.
A shrink operation:
.routing.allocation.initial_recovery._id index setting.IMPORTANT: Indices can only be shrunk if they satisfy the following requirements:
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
The number of shard copies that must be active before proceeding with the operation.
Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).
curl \
--request POST 'http://api.example.com/{index}/_shrink/{target}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"settings\": {\n \"index.routing.allocation.require._name\": null,\n \"index.blocks.write\": null\n }\n}"'{
"settings": {
"index.routing.allocation.require._name": null,
"index.blocks.write": null
}
}{
"acknowledged": true,
"shards_acknowledged": true,
"index": "string"
}Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request POST 'http://api.example.com/_aliases' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"actions":[{"add":{"alias":"string","aliases":"string","filter":{},"index":"string","indices":"string","index_routing":"string","is_hidden":true,"is_write_index":true,"routing":"string","search_routing":"string","must_exist":true},"remove":{"alias":"string","aliases":"string","index":"string","indices":"string","must_exist":true},"remove_index":{"index":"string","indices":"string","must_exist":true}}]}'{
"actions": [
{
"add": {
"alias": "string",
"aliases": "string",
"filter": {},
"index": "string",
"indices": "string",
"index_routing": "string",
"is_hidden": true,
"is_write_index": true,
"routing": "string",
"search_routing": "string",
"must_exist": true
},
"remove": {
"alias": "string",
"aliases": "string",
"index": "string",
"indices": "string",
"must_exist": true
},
"remove_index": {
"index": "string",
"indices": "string",
"must_exist": true
}
}
]
}{
"acknowledged": true
}Comma-separated list of data streams, indices, and aliases to search.
Supports wildcards (*).
To search all data streams or indices, omit this parameter or use * or _all.
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
If true, the validation is executed on all shards instead of one random shard per index.
Analyzer to use for the query string.
This parameter can only be used when the q query string parameter is specified.
If true, wildcard and prefix queries are analyzed.
The default operator for query string query: AND or OR.
Values are and, AND, or, or OR.
Field to use as default where no field prefix is given in the query string.
This parameter can only be used when the q query string parameter is specified.
Type of index that wildcard patterns can match.
If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.
Supports comma-separated values, such as open,hidden.
Valid values are: all, open, closed, hidden, none.
If true, the response returns detailed information if an error has occurred.
If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.
If true, returns a more detailed explanation showing the actual Lucene query that will be executed.
Query in the Lucene query string syntax.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
curl \
--request POST 'http://api.example.com/{index}/_validate/query' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"query":{}}'{
"query": {}
}{
"explanations": [
{
"error": "string",
"explanation": "string",
"index": "string",
"valid": true
}
],
"_shards": {
"failed": 42.0,
"successful": 42.0,
"total": 42.0,
"failures": [
{
"index": "string",
"node": "string",
"reason": {
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
},
"shard": 42.0,
"status": "string"
}
],
"skipped": 42.0
},
"valid": true,
"error": "string"
}Manually move an index into a specific step in the lifecycle policy and run that step.
WARNING: This operation can result in the loss of data. Manually moving an index into a specific step runs that step even if it has already been performed. This is a potentially destructive action and this should be considered an expert level API.
You must specify both the current step and the step to be executed in the body of the request. The request will fail if the current step does not match the step currently running for the index This is to prevent the index from being moved from an unexpected step into the next step.
When specifying the target (next_step) to which the index will be moved, either the name or both the action and name fields are optional.
If only the phase is specified, the index will move to the first step of the first action in the target phase.
If the phase and action are specified, the index will move to the first step of the specified action in the specified phase.
Only actions specified in the ILM policy are considered valid.
An index cannot move to a step that is not part of its policy.
The name of the index whose lifecycle step is to change
curl \
--request POST 'http://api.example.com/_ilm/move/{index}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"current_step\": {\n \"phase\": \"new\",\n \"action\": \"complete\",\n \"name\": \"complete\"\n },\n \"next_step\": {\n \"phase\": \"warm\",\n \"action\": \"forcemerge\",\n \"name\": \"forcemerge\"\n }\n}"'{
"current_step": {
"phase": "new",
"action": "complete",
"name": "complete"
},
"next_step": {
"phase": "warm",
"action": "forcemerge",
"name": "forcemerge"
}
}{
"current_step": {
"phase": "hot",
"action": "complete",
"name": "complete"
},
"next_step": {
"phase": "warm"
}
}{
"acknowledged": true
}Halt all lifecycle management operations and stop the index lifecycle management plugin. This is useful when you are performing maintenance on the cluster and need to prevent ILM from performing any actions on your indices.
The API returns as soon as the stop request has been acknowledged, but the plugin might continue to run until in-progress operations complete and the plugin can be safely stopped. Use the get ILM status API to check whether ILM is running.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request POST 'http://api.example.com/_ilm/stop' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}The inference identifier.
curl \
--request DELETE 'http://api.example.com/_inference/{inference_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true,
"pipelines": [
"string"
]
}Create an inference endpoint to perform an inference task with the elasticsearch service.
Your Elasticsearch deployment contains preconfigured ELSER and E5 inference endpoints, you only need to create the enpoints using the API if you want to customize the settings.
If you use the ELSER or the E5 model through the elasticsearch service, the API request will automatically download and deploy the model if it isn't downloaded yet.
You might see a 502 bad gateway error in the response when using the Kibana Console. This error usually just reflects a timeout, while the model downloads in the background. You can check the download progress in the Machine Learning UI. If using the Python client, you can set the timeout parameter to a higher value.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The type of the inference task that the model will perform.
Values are rerank, sparse_embedding, or text_embedding.
The unique identifier of the inference endpoint.
The must not match the model_id.
Value is elasticsearch.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{elasticsearch_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"elasticsearch\",\n \"service_settings\": {\n \"adaptive_allocations\": { \n \"enabled\": true,\n \"min_number_of_allocations\": 1,\n \"max_number_of_allocations\": 4\n },\n \"num_threads\": 1,\n \"model_id\": \".elser_model_2\" \n }\n}"'{
"service": "elasticsearch",
"service_settings": {
"adaptive_allocations": {
"enabled": true,
"min_number_of_allocations": 1,
"max_number_of_allocations": 4
},
"num_threads": 1,
"model_id": ".elser_model_2"
}
}{
"service": "elasticsearch",
"service_settings": {
"model_id": ".rerank-v1",
"num_threads": 1,
"adaptive_allocations": {
"enabled": true,
"min_number_of_allocations": 1,
"max_number_of_allocations": 4
}
}
}{
"service": "elasticsearch",
"service_settings": {
"num_allocations": 1,
"num_threads": 1,
"model_id": ".multilingual-e5-small"
}
}{
"service": "elasticsearch",
"service_settings": {
"num_allocations": 1,
"num_threads": 1,
"model_id": "msmarco-MiniLM-L12-cos-v5"
}
}{
"service": "elasticsearch",
"service_settings": {
"adaptive_allocations": {
"enabled": true,
"min_number_of_allocations": 3,
"max_number_of_allocations": 10
},
"num_threads": 1,
"model_id": ".multilingual-e5-small"
}
}{
"service": "elasticsearch",
"service_settings": {
"deployment_id": ".elser_model_2"
}
}{
"inference_id": "use_existing_deployment",
"task_type": "sparse_embedding",
"service": "elasticsearch",
"service_settings": {
"num_allocations": 2,
"num_threads": 1,
"model_id": ".elser_model_2",
"deployment_id": ".elser_model_2"
},
"chunking_settings": {
"strategy": "sentence",
"max_chunk_size": 250,
"sentence_overlap": 1
}
}Create an inference endpoint to perform an inference task with the watsonxai service.
You need an IBM Cloud Databases for Elasticsearch deployment to use the watsonxai inference service.
You can provision one through the IBM catalog, the Cloud Databases CLI plug-in, the Cloud Databases API, or Terraform.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
The task type.
The only valid task type for the model to perform is text_embedding.
Value is text_embedding.
The unique identifier of the inference endpoint.
Value is watsonxai.
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{watsonx_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"watsonxai\",\n \"service_settings\": {\n \"api_key\": \"Watsonx-API-Key\", \n \"url\": \"Wastonx-URL\", \n \"model_id\": \"ibm/slate-30m-english-rtrvr\",\n \"project_id\": \"IBM-Cloud-ID\", \n \"api_version\": \"2024-03-14\"\n }\n}"'{
"service": "watsonxai",
"service_settings": {
"api_key": "Watsonx-API-Key",
"url": "Wastonx-URL",
"model_id": "ibm/slate-30m-english-rtrvr",
"project_id": "IBM-Cloud-ID",
"api_version": "2024-03-14"
}
}{
"chunking_settings": {
"max_chunk_size": 42.0,
"overlap": 42.0,
"sentence_overlap": 42.0,
"strategy": "string"
},
"service": "string",
"service_settings": {},
"task_settings": {},
"inference_id": "string",
"task_type": "sparse_embedding"
}The inference Id
Specifies the amount of time to wait for the inference request to complete.
Inference input. Either a string or an array of strings.
curl \
--request POST 'http://api.example.com/_inference/sparse_embedding/{inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"input\": \"The sky above the port was the color of television tuned to a dead channel.\"\n}"'{
"input": "The sky above the port was the color of television tuned to a dead channel."
}{
"sparse_embedding": [
{
"port": 2.1259406,
"sky": 1.7073475,
"color": 1.6922266,
"dead": 1.6247464,
"television": 1.3525393,
"above": 1.2425821,
"tuned": 1.1440028,
"colors": 1.1218185,
"tv": 1.0111054,
"ports": 1.0067928,
"poem": 1.0042328,
"channel": 0.99471164,
"tune": 0.96235967,
"scene": 0.9020516
}
]
}Modify task_settings, secrets (within service_settings), or num_allocations for an inference endpoint, depending on the specific endpoint service and task_type.
IMPORTANT: The inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio, Google Vertex AI, Anthropic, Watsonx.ai, or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.
The type of inference task that the model performs.
Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
The unique identifier of the inference endpoint.
The service type
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{inference_id}/_update' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"chunking_settings":{"max_chunk_size":42.0,"overlap":42.0,"sentence_overlap":42.0,"strategy":"string"},"service":"string","service_settings":{},"task_settings":{}}'{
"chunking_settings": {
"max_chunk_size": 42.0,
"overlap": 42.0,
"sentence_overlap": 42.0,
"strategy": "string"
},
"service": "string",
"service_settings": {},
"task_settings": {}
}{
"chunking_settings": {
"max_chunk_size": 42.0,
"overlap": 42.0,
"sentence_overlap": 42.0,
"strategy": "string"
},
"service": "string",
"service_settings": {},
"task_settings": {},
"inference_id": "string",
"task_type": "sparse_embedding"
}Get information about one or more ingest pipelines. This API returns a local reference of the pipeline.
Comma-separated list of pipeline IDs to retrieve.
Wildcard (*) expressions are supported.
To get all ingest pipelines, omit this parameter or use *.
Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
Return pipelines without their definitions (default: false)
curl \
--request GET 'http://api.example.com/_ingest/pipeline/{id}' \
--header "Authorization: $API_KEY"{
"my-pipeline-id" : {
"description" : "describe pipeline",
"version" : 123,
"processors" : [
{
"set" : {
"field" : "foo",
"value" : "bar"
}
}
]
}
}Get download statistics for GeoIP2 databases that are used with the GeoIP processor.
curl \
--request GET 'http://api.example.com/_ingest/geoip/stats' \
--header "Authorization: $API_KEY"{
"stats": {
"successful_downloads": 42.0,
"failed_downloads": 42.0,
"": 42.0,
"databases_count": 42.0,
"skipped_updates": 42.0,
"expired_databases": 42.0
},
"nodes": {
"additionalProperty1": {
"databases": [
{
"name": "string"
}
],
"files_in_temp": [
"string"
]
},
"additionalProperty2": {
"databases": [
{
"name": "string"
}
],
"files_in_temp": [
"string"
]
}
}
}Remove all scheduled events from a calendar, then delete it.
A string that uniquely identifies a calendar.
curl \
--request DELETE 'http://api.example.com/_ml/calendars/{calendar_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}A string that uniquely identifies a calendar.
Identifier for the scheduled event. You can obtain this identifier by using the get calendar events API.
curl \
--request DELETE 'http://api.example.com/_ml/calendars/{calendar_id}/events/{event_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}You can get information for multiple datafeeds in a single API request by
using a comma-separated list of datafeeds or a wildcard expression. You can
get information for all datafeeds by using _all, by specifying * as the
<feed_id>, or by omitting the <feed_id>.
This API returns a maximum of 10,000 datafeeds.
Identifier for the datafeed. It can be a datafeed identifier or a wildcard expression. If you do not specify one of these options, the API returns information about all datafeeds.
Specifies what to do when the request:
_all string or no identifiers and there are no matches.The default value is true, which returns an empty datafeeds array
when there are no matches and the subset of results when there are
partial matches. If this parameter is false, the request returns a
404 status code when there are no matches or only partial matches.
Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.
curl \
--request GET 'http://api.example.com/_ml/datafeeds/{datafeed_id}' \
--header "Authorization: $API_KEY"{
"count": 42.0,
"datafeeds": [
{
"aggregations": {},
"authorization": {
"api_key": {
"id": "string",
"name": "string"
},
"roles": [
"string"
],
"service_account": "string"
},
"chunking_config": {
"mode": "auto",
"time_span": "string"
},
"datafeed_id": "string",
"frequency": "string",
"indices": [
"string"
],
"indexes": [
"string"
],
"job_id": "string",
"max_empty_searches": 42.0,
"query_delay": "string",
"script_fields": {
"additionalProperty1": {
"script": {
"id": "string",
"params": {},
"options": {}
},
"ignore_failure": true
},
"additionalProperty2": {
"script": {
"id": "string",
"params": {},
"options": {}
},
"ignore_failure": true
}
},
"scroll_size": 42.0,
"delayed_data_check_config": {
"check_window": "string",
"enabled": true
},
"runtime_mappings": {
"additionalProperty1": {
"fields": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"fetch_fields": [
{}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"id": "string",
"params": {},
"options": {}
},
"type": "boolean"
},
"additionalProperty2": {
"fields": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"fetch_fields": [
{}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"id": "string",
"params": {},
"options": {}
},
"type": "boolean"
}
},
"indices_options": {
"allow_no_indices": true,
"expand_wildcards": "string",
"ignore_unavailable": true,
"ignore_throttled": true
},
"query": {}
}
]
}Make an estimation of the memory usage for an anomaly detection job model. The estimate is based on analysis configuration details for the job and cardinality estimates for the fields it references.
Estimates of the highest cardinality in a single bucket that is observed
for influencer fields over the time period that the job analyzes data.
To produce a good answer, values must be provided for all influencer
fields. Providing values for fields that are not listed as influencers
has no effect on the estimation.
Estimates of the cardinality that is observed for fields over the whole
time period that the job analyzes data. To produce a good answer, values
must be provided for fields referenced in the by_field_name,
over_field_name and partition_field_name of any detectors. Providing
values for other fields has no effect on the estimation. It can be
omitted from the request if no detectors have a by_field_name,
over_field_name or partition_field_name.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/_estimate_model_memory' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"analysis_config\": {\n \"bucket_span\": \"5m\",\n \"detectors\": [\n {\n \"function\": \"sum\",\n \"field_name\": \"bytes\",\n \"by_field_name\": \"status\",\n \"partition_field_name\": \"app\"\n }\n ],\n \"influencers\": [\n \"source_ip\",\n \"dest_ip\"\n ]\n },\n \"overall_cardinality\": {\n \"status\": 10,\n \"app\": 50\n },\n \"max_bucket_cardinality\": {\n \"source_ip\": 300,\n \"dest_ip\": 30\n }\n}"'{
"analysis_config": {
"bucket_span": "5m",
"detectors": [
{
"function": "sum",
"field_name": "bytes",
"by_field_name": "status",
"partition_field_name": "app"
}
],
"influencers": [
"source_ip",
"dest_ip"
]
},
"overall_cardinality": {
"status": 10,
"app": 50
},
"max_bucket_cardinality": {
"source_ip": 300,
"dest_ip": 30
}
}{
"model_memory_estimate": "21mb"
}Identifier for the anomaly detection job.
Skips the specified number of categories.
Only return categories for the specified partition.
Specifies the maximum number of categories to obtain.
curl \
--request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/results/categories' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"page":{"from":42.0,"size":42.0}}'{
"page": {
"from": 42.0,
"size": 42.0
}
}{
"categories": [
{
"category_id": 42.0,
"examples": [
"string"
],
"grok_pattern": "string",
"job_id": "string",
"max_matching_length": 42.0,
"partition_field_name": "string",
"partition_field_value": "string",
"regex": "string",
"terms": "string",
"num_matches": 42.0,
"preferred_to_categories": [
"string"
],
"p": "string",
"result_type": "string",
"mlcategory": "string"
}
],
"count": 42.0
}Specifies what to do when the request:
If true, the API returns an empty jobs array when
there are no matches and the subset of results when there are partial
matches. If false, the API returns a 404 status
code when there are no matches or only partial matches.
curl \
--request GET 'http://api.example.com/_ml/anomaly_detectors/_stats' \
--header "Authorization: $API_KEY"{
"count": 42.0,
"jobs": [
{
"assignment_explanation": "string",
"data_counts": {
"bucket_count": 42.0,
"earliest_record_timestamp": 42.0,
"empty_bucket_count": 42.0,
"input_bytes": 42.0,
"input_field_count": 42.0,
"input_record_count": 42.0,
"invalid_date_count": 42.0,
"job_id": "string",
"last_data_time": 42.0,
"latest_empty_bucket_timestamp": 42.0,
"latest_record_timestamp": 42.0,
"latest_sparse_bucket_timestamp": 42.0,
"latest_bucket_timestamp": 42.0,
"log_time": 42.0,
"missing_field_count": 42.0,
"out_of_order_timestamp_count": 42.0,
"processed_field_count": 42.0,
"processed_record_count": 42.0,
"sparse_bucket_count": 42.0
},
"forecasts_stats": {
"memory_bytes": {
"avg": 42.0,
"max": 42.0,
"min": 42.0,
"total": 42.0
},
"processing_time_ms": {
"avg": 42.0,
"max": 42.0,
"min": 42.0,
"total": 42.0
},
"records": {
"avg": 42.0,
"max": 42.0,
"min": 42.0,
"total": 42.0
},
"status": {
"additionalProperty1": 42.0,
"additionalProperty2": 42.0
},
"total": 42.0,
"forecasted_jobs": 42.0
},
"job_id": "string",
"model_size_stats": {
"bucket_allocation_failures_count": 42.0,
"job_id": "string",
"": 42.0,
"memory_status": "ok",
"assignment_memory_basis": "string",
"result_type": "string",
"total_by_field_count": 42.0,
"total_over_field_count": 42.0,
"total_partition_field_count": 42.0,
"categorization_status": "ok",
"categorized_doc_count": 42.0,
"dead_category_count": 42.0,
"failed_category_count": 42.0,
"frequent_category_count": 42.0,
"rare_category_count": 42.0,
"total_category_count": 42.0,
"timestamp": 42.0
},
"node": {
"name": "string",
"ephemeral_id": "string",
"id": "string",
"transport_address": "string",
"attributes": {
"additionalProperty1": "string",
"additionalProperty2": "string"
}
},
"": "string",
"state": "closing",
"timing_stats": {
"": 42.0,
"bucket_count": 42.0,
"job_id": "string"
},
"deleting": true
}
]
}Updates the description of a filter, adds items, or removes items from the list.
A string that uniquely identifies a filter.
The items to add to the filter.
A description for the filter.
The items to remove from the filter.
curl \
--request POST 'http://api.example.com/_ml/filters/{filter_id}/_update' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"add_items":["string"],"description":"string","remove_items":["string"]}'{
"add_items": [
"string"
],
"description": "string",
"remove_items": [
"string"
]
}{
"description": "string",
"filter_id": "string",
"items": [
"string"
]
}The API packages together commonly used evaluation metrics for various types of machine learning features. This has been designed for use on indexes created by data frame analytics. Evaluation requires both a ground truth field and an analytics result field to be present.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
curl \
--request POST 'http://api.example.com/_ml/data_frame/_evaluate' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"index\": \"animal_classification\",\n \"evaluation\": {\n \"classification\": {\n \"actual_field\": \"animal_class\",\n \"predicted_field\": \"ml.animal_class_prediction\",\n \"metrics\": {\n \"multiclass_confusion_matrix\": {}\n }\n }\n }\n}"'{
"index": "animal_classification",
"evaluation": {
"classification": {
"actual_field": "animal_class",
"predicted_field": "ml.animal_class_prediction",
"metrics": {
"multiclass_confusion_matrix": {}
}
}
}
}{
"index": "animal_classification",
"evaluation": {
"classification": {
"actual_field": "animal_class",
"metrics": {
"auc_roc": {
"class_name": "dog"
}
}
}
}
}{
"index": "my_analytics_dest_index",
"evaluation": {
"outlier_detection": {
"actual_field": "is_outlier",
"predicted_probability_field": "ml.outlier_score"
}
}
}{
"index": "house_price_predictions",
"query": {
"bool": {
"filter": [
{
"term": {
"ml.is_training": false
}
}
]
}
},
"evaluation": {
"regression": {
"actual_field": "price",
"predicted_field": "ml.price_prediction",
"metrics": {
"r_squared": {},
"mse": {},
"msle": {
"offset": 10
},
"huber": {
"delta": 1.5
}
}
}
}
}{
"index": "house_price_predictions",
"query": {
"term": {
"ml.is_training": {
"value": true
}
}
},
"evaluation": {
"regression": {
"actual_field": "price",
"predicted_field": "ml.price_prediction",
"metrics": {
"r_squared": {},
"mse": {},
"msle": {},
"huber": {}
}
}
}
}{
"classification": {
"multiclass_confusion_matrix": {
"confusion_matrix": [
{
"actual_class": "cat",
"actual_class_doc_count": 12,
"predicted_classes": [
{
"predicted_class": "cat",
"count": 12
},
{
"predicted_class": "dog",
"count": 0
}
],
"other_predicted_class_doc_count": 0
},
{
"actual_class": "dog",
"actual_class_doc_count": 11,
"predicted_classes": [
{
"predicted_class": "dog",
"count": 7
},
{
"predicted_class": "cat",
"count": 4
}
],
"other_predicted_class_doc_count": 0
}
],
"other_actual_class_count": 0
}
}
}{
"classification": {
"auc_roc": {
"value": 0.8941788639536681
}
}
}{
"outlier_detection": {
"auc_roc": {
"value": 0.9258475774641445
},
"confusion_matrix": {
"0.25": {
"tp": 5,
"fp": 9,
"tn": 204,
"fn": 5
},
"0.5": {
"tp": 1,
"fp": 5,
"tn": 208,
"fn": 9
},
"0.75": {
"tp": 0,
"fp": 4,
"tn": 209,
"fn": 10
}
},
"precision": {
"0.25": 0.35714285714285715,
"0.5": 0.16666666666666666,
"0.75": 0
},
"recall": {
"0.25": 0.5,
"0.5": 0.1,
"0.75": 0
}
}
}A data frame analytics job can be started and stopped multiple times throughout its lifecycle.
Identifier for the data frame analytics job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
Specifies what to do when the request:
The default value is true, which returns an empty data_frame_analytics array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.
If true, the data frame analytics job is stopped forcefully.
Controls the amount of time to wait until the data frame analytics job stops. Defaults to 20 seconds.
curl \
--request POST 'http://api.example.com/_ml/data_frame/analytics/{id}/_stop' \
--header "Authorization: $API_KEY"{
"stopped": true
}Cache will be cleared on all nodes where the trained model is assigned. A trained model deployment may have an inference cache enabled. As requests are handled by each allocated node, their responses may be cached on that individual node. Calling this API clears the caches without restarting the deployment.
The unique identifier of the trained model.
curl \
--request POST 'http://api.example.com/_ml/trained_models/{model_id}/deployment/cache/_clear' \
--header "Authorization: $API_KEY"{
"cleared": true
}A trained model alias is a logical name used to reference a single trained model. You can use aliases instead of trained model identifiers to make it easier to reference your models. For example, you can use aliases in inference aggregations and processors. An alias must be unique and refer to only a single trained model. However, you can have multiple aliases for each trained model. If you use this API to update an alias such that it references a different trained model ID and the model uses a different type of data frame analytics, an error occurs. For example, this situation occurs if you have a trained model for regression analysis and a trained model for classification analysis; you cannot reassign an alias from one type of trained model to another. If you use this API to update an alias and there are very few input fields in common between the old and new trained models for the model alias, the API returns a warning.
The identifier for the trained model that the alias refers to.
The alias to create or update. This value cannot end in numbers.
Specifies whether the alias gets reassigned to the specified trained model if it is already assigned to a different model. If the alias is already assigned and this parameter is false, the API returns an error.
curl \
--request PUT 'http://api.example.com/_ml/trained_models/{model_id}/model_aliases/{model_alias}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}You can get usage information for multiple trained models in a single API request by using a comma-separated list of model IDs or a wildcard expression.
The unique identifier of the trained model or a model alias. It can be a comma-separated list or a wildcard expression.
Specifies what to do when the request:
If true, it returns an empty array when there are no matches and the subset of results when there are partial matches.
Skips the specified number of models.
Specifies the maximum number of models to obtain.
curl \
--request GET 'http://api.example.com/_ml/trained_models/{model_id}/_stats' \
--header "Authorization: $API_KEY"{
"count": 42.0,
"trained_model_stats": [
{
"deployment_stats": {
"adaptive_allocations": {
"enabled": true,
"min_number_of_allocations": 42.0,
"max_number_of_allocations": 42.0
},
"allocation_status": {
"allocation_count": 42.0,
"state": "started",
"target_allocation_count": 42.0
},
"": 42.0,
"deployment_id": "string",
"error_count": 42.0,
"inference_count": 42.0,
"model_id": "string",
"nodes": [
{
"error_count": 42.0,
"inference_count": 42.0,
"inference_cache_hit_count": 42.0,
"inference_cache_hit_count_last_minute": 42.0,
"node": {},
"number_of_allocations": 42.0,
"number_of_pending_requests": 42.0,
"peak_throughput_per_minute": 42.0,
"rejected_execution_count": 42.0,
"routing_state": {},
"threads_per_allocation": 42.0,
"throughput_last_minute": 42.0,
"timeout_count": 42.0
}
],
"number_of_allocations": 42.0,
"peak_throughput_per_minute": 42.0,
"priority": "normal",
"queue_capacity": 42.0,
"rejected_execution_count": 42.0,
"reason": "string",
"state": "started",
"threads_per_allocation": 42.0,
"timeout_count": 42.0
},
"inference_stats": {
"cache_miss_count": 42.0,
"failure_count": 42.0,
"inference_count": 42.0,
"missing_all_fields_count": 42.0,
"": 42.0
},
"ingest": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"model_id": "string",
"model_size_stats": {
"": 42.0
},
"pipeline_count": 42.0
}
]
}You can get usage information for multiple trained models in a single API request by using a comma-separated list of model IDs or a wildcard expression.
Specifies what to do when the request:
If true, it returns an empty array when there are no matches and the subset of results when there are partial matches.
Skips the specified number of models.
Specifies the maximum number of models to obtain.
curl \
--request GET 'http://api.example.com/_ml/trained_models/_stats' \
--header "Authorization: $API_KEY"{
"count": 42.0,
"trained_model_stats": [
{
"deployment_stats": {
"adaptive_allocations": {
"enabled": true,
"min_number_of_allocations": 42.0,
"max_number_of_allocations": 42.0
},
"allocation_status": {
"allocation_count": 42.0,
"state": "started",
"target_allocation_count": 42.0
},
"": 42.0,
"deployment_id": "string",
"error_count": 42.0,
"inference_count": 42.0,
"model_id": "string",
"nodes": [
{
"error_count": 42.0,
"inference_count": 42.0,
"inference_cache_hit_count": 42.0,
"inference_cache_hit_count_last_minute": 42.0,
"node": {},
"number_of_allocations": 42.0,
"number_of_pending_requests": 42.0,
"peak_throughput_per_minute": 42.0,
"rejected_execution_count": 42.0,
"routing_state": {},
"threads_per_allocation": 42.0,
"throughput_last_minute": 42.0,
"timeout_count": 42.0
}
],
"number_of_allocations": 42.0,
"peak_throughput_per_minute": 42.0,
"priority": "normal",
"queue_capacity": 42.0,
"rejected_execution_count": 42.0,
"reason": "string",
"state": "started",
"threads_per_allocation": 42.0,
"timeout_count": 42.0
},
"inference_stats": {
"cache_miss_count": 42.0,
"failure_count": 42.0,
"inference_count": 42.0,
"missing_all_fields_count": 42.0,
"": 42.0
},
"ingest": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"model_id": "string",
"model_size_stats": {
"": 42.0
},
"pipeline_count": 42.0
}
]
}Cancel a migration reindex attempt for a data stream or index.
The index or data stream name
curl \
--request POST 'http://api.example.com/_migration/reindex/{index}/_cancel' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}Delete a query rule within a query ruleset. This is a destructive action that is only recoverable by re-adding the same rule with the create or update query rule API.
The unique identifier of the query ruleset containing the rule to delete
The unique identifier of the query rule within the specified ruleset to delete
curl \
--request DELETE 'http://api.example.com/_query_rules/{ruleset_id}/_rule/{rule_id}' \
--header "Authorization: $API_KEY"{
"acknowledged": true
}Get the configuration, stats, and status of rollup jobs.
NOTE: This API returns only active (both STARTED and STOPPED) jobs.
If a job was created, ran for a while, then was deleted, the API does not return any details about it.
For details about a historical rollup job, the rollup capabilities API may be more useful.
curl \
--request GET 'http://api.example.com/_rollup/job' \
--header "Authorization: $API_KEY"{
"jobs": [
{
"config": {
"id": "sensor",
"index_pattern": "sensor-*",
"rollup_index": "sensor_rollup",
"cron": "*/30 * * * * ?",
"groups": {
"date_histogram": {
"fixed_interval": "1h",
"delay": "7d",
"field": "timestamp",
"time_zone": "UTC"
},
"terms": {
"fields": [
"node"
]
}
},
"metrics": [
{
"field": "temperature",
"metrics": [
"min",
"max",
"sum"
]
},
{
"field": "voltage",
"metrics": [
"avg"
]
}
],
"timeout": "20s",
"page_size": 1000
},
"status": {
"job_state": "stopped"
},
"stats": {
"pages_processed": 0,
"documents_processed": 0,
"rollups_indexed": 0,
"trigger_count": 0,
"index_failures": 0,
"index_time_in_ms": 0,
"index_total": 0,
"search_failures": 0,
"search_time_in_ms": 0,
"search_total": 0,
"processing_time_in_ms": 0,
"processing_total": 0
}
}
]
}Get the capabilities of any rollup jobs that have been configured for a specific index or index pattern.
This API is useful because a rollup job is often configured to rollup only a subset of fields from the source index. Furthermore, only certain aggregations can be configured for various fields, leading to a limited subset of functionality depending on that configuration. This API enables you to inspect an index and determine:
curl \
--request GET 'http://api.example.com/_rollup/data' \
--header "Authorization: $API_KEY"{
"sensor-*" : {
"rollup_jobs" : [
{
"job_id" : "sensor",
"rollup_index" : "sensor_rollup",
"index_pattern" : "sensor-*",
"fields" : {
"node" : [
{
"agg" : "terms"
}
],
"temperature" : [
{
"agg" : "min"
},
{
"agg" : "max"
},
{
"agg" : "sum"
}
],
"timestamp" : [
{
"agg" : "date_histogram",
"time_zone" : "UTC",
"fixed_interval" : "1h",
"delay": "7d"
}
],
"voltage" : [
{
"agg" : "avg"
}
]
}
}
]
}
}The rollup search endpoint is needed because, internally, rolled-up documents utilize a different document structure than the original data. It rewrites standard Query DSL into a format that matches the rollup documents then takes the response and rewrites it back to what a client would expect given the original query.
The request body supports a subset of features from the regular search API. The following functionality is not available:
size: Because rollups work on pre-aggregated data, no search hits can be returned and so size must be set to zero or omitted entirely.
highlighter, suggestors, post_filter, profile, explain: These are similarly disallowed.
Searching both historical rollup and non-rollup data
The rollup search API has the capability to search across both "live" non-rollup data and the aggregated rollup data. This is done by simply adding the live indices to the URI. For example:
GET sensor-1,sensor_rollup/_rollup_search
{
"size": 0,
"aggregations": {
"max_temperature": {
"max": {
"field": "temperature"
}
}
}
}
The rollup search endpoint does two things when the search runs:
When the two responses are received, the endpoint rewrites the rollup response and merges the two together. During the merging process, if there is any overlap in buckets between the two responses, the buckets from the non-rollup index are used.
A comma-separated list of data streams and indices used to limit the request. This parameter has the following rules:
_all are not permitted.*) may be used. If they match more than one rollup index, an exception occurs. However, you can use an expression to match multiple non-rollup indices or data streams.Indicates whether hits.total should be rendered as an integer or an object in the rest search response
Specify whether aggregation and suggester names should be prefixed by their respective types in the response
Specifies aggregations.
An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
Must be zero if set, as rollups work on pre-aggregated data.
curl \
--request GET 'http://api.example.com/{index}/_rollup_search' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"size\": 0,\n \"aggregations\": {\n \"max_temperature\": {\n \"max\": {\n \"field\": \"temperature\"\n }\n }\n }\n}"'{
"size": 0,
"aggregations": {
"max_temperature": {
"max": {
"field": "temperature"
}
}
}
}{
"took" : 102,
"timed_out" : false,
"terminated_early" : false,
"_shards" : {} ,
"hits" : {
"total" : {
"value": 0,
"relation": "eq"
},
"max_score" : 0.0,
"hits" : [ ]
},
"aggregations" : {
"max_temperature" : {
"value" : 202.0
}
}
}If you try to start a job that does not exist, an exception occurs. If you try to start a job that is already started, nothing happens.
Identifier for the rollup job.
curl \
--request POST 'http://api.example.com/_rollup/job/{id}/_start' \
--header "Authorization: $API_KEY"{
"started": true
}