Get shard recovery information
Get information about ongoing and completed shard recoveries. Shard recovery is the process of initializing a shard copy, such as restoring a primary shard from a snapshot or syncing a replica shard from a primary shard. When a shard recovery completes, the recovered shard is available for search and indexing. For data streams, the API returns information about the stream’s backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the index recovery API.
Query parameters
-
active_only
boolean If
true
, the response only includes ongoing shard recoveries. -
bytes
string The unit used to display byte values.
Values are
b
,kb
,mb
,gb
,tb
, orpb
. -
detailed
boolean If
true
, the response includes detailed information about shard recoveries. -
index
string | array[string] Comma-separated list or wildcard expression of index names to limit the returned information
-
h
string | array[string] List of columns to appear in the response. Supports simple wildcards.
-
s
string | array[string] List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting
:asc
or:desc
as a suffix to the column name. -
time
string Unit used to display time values.
Values are
nanos
,micros
,ms
,s
,m
,h
, ord
.
curl \
--request GET 'http://api.example.com/_cat/recovery' \
--header "Authorization: $API_KEY"
[
{
"index": "my-index-000001 ",
"shard": "0",
"time": "13ms",
"type": "store",
"stage": "done",
"source_host": "n/a",
"source_node": "n/a",
"target_host": "127.0.0.1",
"target_node": "node-0",
"repository": "n/a",
"snapshot": "n/a",
"files": "0",
"files_recovered": "0",
"files_percent": "100.0%",
"files_total": "13",
"bytes": "0b",
"bytes_recovered": "0b",
"bytes_percent": "100.0%",
"bytes_total": "9928b",
"translog_ops": "0",
"translog_ops_recovered": "0",
"translog_ops_percent": "100.0%"
}
]
[
{
"i": "my-index-000001",
"s": "0",
"t": "1252ms",
"ty": "peer",
"st": "done",
"shost": "192.168.1.1",
"thost": "192.168.1.1",
"f": "0",
"fp": "100.0%",
"b": "0b",
"bp": "100.0%",
}
]
[
{
"i": "my-index-000001",
"s": "0",
"t": "1978ms",
"ty": "snapshot",
"st": "done",
"rep": "my-repo",
"snap": "snap-1",
"f": "79",
"fp": "8.0%",
"b": "12086",
"bp": "9.0%"
}
]
Get node information
Added in 1.3.0
By default, the API returns all attributes and core settings for cluster nodes.
Query parameters
-
flat_settings
boolean If true, returns settings in flat format.
-
timeout
string Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_nodes' \
--header "Authorization: $API_KEY"
{
"_nodes": {},
"cluster_name": "elasticsearch",
"nodes": {
"USpTGYaBSIKbgSUJR2Z9lg": {
"name": "node-0",
"transport_address": "192.168.17:9300",
"host": "node-0.elastic.co",
"ip": "192.168.17",
"version": "{version}",
"transport_version": 100000298,
"index_version": 100000074,
"component_versions": {
"ml_config_version": 100000162,
"transform_config_version": 100000096
},
"build_flavor": "default",
"build_type": "{build_type}",
"build_hash": "587409e",
"roles": [
"master",
"data",
"ingest"
],
"attributes": {},
"plugins": [
{
"name": "analysis-icu",
"version": "{version}",
"description": "The ICU Analysis plugin integrates Lucene ICU
module into elasticsearch, adding ICU relates analysis components.",
"classname":
"org.elasticsearch.plugin.analysis.icu.AnalysisICUPlugin",
"has_native_controller": false
}
],
"modules": [
{
"name": "lang-painless",
"version": "{version}",
"description": "An easy, safe and fast scripting language for
Elasticsearch",
"classname": "org.elasticsearch.painless.PainlessPlugin",
"has_native_controller": false
}
]
}
}
}
Get node statistics
Get statistics for nodes in a cluster. By default, all stats are returned. You can limit the returned information by using metrics.
Path parameters
-
metric
string | array[string] Required Limit the information returned to the specified metrics
-
index_metric
string | array[string] Required Limit the information returned for indices metric to the specific index metrics. It can be used only if indices (or all) metric is specified.
Query parameters
-
completion_fields
string | array[string] Comma-separated list or wildcard expressions of fields to include in fielddata and suggest statistics.
-
fielddata_fields
string | array[string] Comma-separated list or wildcard expressions of fields to include in fielddata statistics.
-
fields
string | array[string] Comma-separated list or wildcard expressions of fields to include in the statistics.
-
groups
boolean Comma-separated list of search groups to include in the search statistics.
-
include_segment_file_sizes
boolean If true, the call reports the aggregated disk usage of each one of the Lucene index files (only applies if segment stats are requested).
-
level
string Indicates whether statistics are aggregated at the cluster, index, or shard level.
Values are
cluster
,indices
, orshards
. -
timeout
string Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
-
types
array[string] A comma-separated list of document types for the indexing index metric.
-
include_unloaded_segments
boolean If
true
, the response includes information from segments that are not loaded into memory.
curl \
--request GET 'http://api.example.com/_nodes/stats/{metric}/{index_metric}' \
--header "Authorization: $API_KEY"
{
"_nodes": {
"failures": [
{
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
}
],
"total": 42.0,
"successful": 42.0,
"failed": 42.0
},
"cluster_name": "string",
"nodes": {
"additionalProperty1": {
"adaptive_selection": {
"additionalProperty1": {
"avg_queue_size": 42.0,
"avg_response_time": "string",
"avg_response_time_ns": 42.0,
"avg_service_time": "string",
"avg_service_time_ns": 42.0,
"outgoing_searches": 42.0,
"rank": "string"
},
"additionalProperty2": {
"avg_queue_size": 42.0,
"avg_response_time": "string",
"avg_response_time_ns": 42.0,
"avg_service_time": "string",
"avg_service_time_ns": 42.0,
"outgoing_searches": 42.0,
"rank": "string"
}
},
"breakers": {
"additionalProperty1": {
"estimated_size": "string",
"estimated_size_in_bytes": 42.0,
"limit_size": "string",
"limit_size_in_bytes": 42.0,
"overhead": 42.0,
"tripped": 42.0
},
"additionalProperty2": {
"estimated_size": "string",
"estimated_size_in_bytes": 42.0,
"limit_size": "string",
"limit_size_in_bytes": 42.0,
"overhead": 42.0,
"tripped": 42.0
}
},
"fs": {
"data": [
{}
],
"timestamp": 42.0,
"total": {
"available": "string",
"available_in_bytes": 42.0,
"free": "string",
"free_in_bytes": 42.0,
"total": "string",
"total_in_bytes": 42.0
},
"io_stats": {
"devices": [
{}
],
"total": {}
}
},
"host": "string",
"http": {
"current_open": 42.0,
"total_opened": 42.0,
"clients": [
{}
],
"routes": {
"additionalProperty1": {},
"additionalProperty2": {}
}
},
"ingest": {
"pipelines": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"total": {
"count": 42.0,
"current": 42.0,
"failed": 42.0
}
},
"ip": "string",
"jvm": {
"buffer_pools": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"classes": {
"current_loaded_count": 42.0,
"total_loaded_count": 42.0,
"total_unloaded_count": 42.0
},
"gc": {
"collectors": {}
},
"mem": {
"heap_used_in_bytes": 42.0,
"heap_used_percent": 42.0,
"heap_committed_in_bytes": 42.0,
"heap_max_in_bytes": 42.0,
"non_heap_used_in_bytes": 42.0,
"non_heap_committed_in_bytes": 42.0,
"pools": {}
},
"threads": {
"count": 42.0,
"peak_count": 42.0
},
"timestamp": 42.0,
"uptime": "string",
"uptime_in_millis": 42.0
},
"name": "string",
"os": {
"cpu": {
"percent": 42.0,
"sys": "string",
"total": "string",
"user": "string",
"load_average": {}
},
"": {},
"swap": {
"adjusted_total_in_bytes": 42.0,
"resident": "string",
"resident_in_bytes": 42.0,
"share": "string",
"share_in_bytes": 42.0,
"total_virtual": "string",
"total_virtual_in_bytes": 42.0,
"total_in_bytes": 42.0,
"free_in_bytes": 42.0,
"used_in_bytes": 42.0
},
"cgroup": {
"cpuacct": {},
"cpu": {},
"memory": {}
},
"timestamp": 42.0
},
"process": {
"cpu": {
"percent": 42.0,
"sys": "string",
"total": "string",
"user": "string",
"load_average": {}
},
"mem": {
"adjusted_total_in_bytes": 42.0,
"resident": "string",
"resident_in_bytes": 42.0,
"share": "string",
"share_in_bytes": 42.0,
"total_virtual": "string",
"total_virtual_in_bytes": 42.0,
"total_in_bytes": 42.0,
"free_in_bytes": 42.0,
"used_in_bytes": 42.0
},
"open_file_descriptors": 42.0,
"max_file_descriptors": 42.0,
"timestamp": 42.0
},
"roles": [
"master"
],
"script": {
"cache_evictions": 42.0,
"compilations": 42.0,
"compilations_history": {
"additionalProperty1": 42.0,
"additionalProperty2": 42.0
},
"compilation_limit_triggered": 42.0,
"contexts": [
{}
]
},
"script_cache": {},
"thread_pool": {
"additionalProperty1": {
"active": 42.0,
"completed": 42.0,
"largest": 42.0,
"queue": 42.0,
"rejected": 42.0,
"threads": 42.0
},
"additionalProperty2": {
"active": 42.0,
"completed": 42.0,
"largest": 42.0,
"queue": 42.0,
"rejected": 42.0,
"threads": 42.0
}
},
"timestamp": 42.0,
"transport": {
"inbound_handling_time_histogram": [
{}
],
"outbound_handling_time_histogram": [
{}
],
"rx_count": 42.0,
"rx_size": "string",
"rx_size_in_bytes": 42.0,
"server_open": 42.0,
"tx_count": 42.0,
"tx_size": "string",
"tx_size_in_bytes": 42.0,
"total_outbound_connections": 42.0
},
"transport_address": "string",
"attributes": {
"additionalProperty1": "string",
"additionalProperty2": "string"
},
"discovery": {
"cluster_state_queue": {
"total": 42.0,
"pending": 42.0,
"committed": 42.0
},
"published_cluster_states": {
"full_states": 42.0,
"incompatible_diffs": 42.0,
"compatible_diffs": 42.0
},
"cluster_state_update": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"serialized_cluster_states": {
"full_states": {},
"diffs": {}
},
"cluster_applier_stats": {
"recordings": [
{}
]
}
},
"indexing_pressure": {
"memory": {
"limit_in_bytes": 42.0,
"current": {},
"total": {}
}
},
"indices": {
"commit": {
"generation": 42.0,
"id": "string",
"num_docs": 42.0,
"user_data": {}
},
"completion": {
"size_in_bytes": 42.0,
"fields": {}
},
"docs": {
"count": 42.0,
"deleted": 42.0
},
"fielddata": {
"evictions": 42.0,
"memory_size_in_bytes": 42.0,
"fields": {}
},
"flush": {
"periodic": 42.0,
"total": 42.0,
"total_time": "string"
},
"get": {
"current": 42.0,
"exists_time": "string",
"exists_total": 42.0,
"missing_time": "string",
"missing_total": 42.0,
"time": "string",
"total": 42.0
},
"indexing": {
"index_current": 42.0,
"delete_current": 42.0,
"delete_time": "string",
"delete_total": 42.0,
"is_throttled": true,
"noop_update_total": 42.0,
"throttle_time": "string",
"index_time": "string",
"index_total": 42.0,
"index_failed": 42.0,
"types": {},
"write_load": 42.0,
"recent_write_load": 42.0,
"peak_write_load": 42.0
},
"mappings": {
"total_count": 42.0,
"total_estimated_overhead_in_bytes": 42.0
},
"merges": {
"current": 42.0,
"current_docs": 42.0,
"current_size": "string",
"current_size_in_bytes": 42.0,
"total": 42.0,
"total_auto_throttle": "string",
"total_auto_throttle_in_bytes": 42.0,
"total_docs": 42.0,
"total_size": "string",
"total_size_in_bytes": 42.0,
"total_stopped_time": "string",
"total_throttled_time": "string",
"total_time": "string"
},
"shard_path": {
"data_path": "string",
"is_custom_data_path": true,
"state_path": "string"
},
"query_cache": {
"cache_count": 42.0,
"cache_size": 42.0,
"evictions": 42.0,
"hit_count": 42.0,
"memory_size_in_bytes": 42.0,
"miss_count": 42.0,
"total_count": 42.0
},
"recovery": {
"current_as_source": 42.0,
"current_as_target": 42.0,
"throttle_time": "string"
},
"refresh": {
"external_total": 42.0,
"listeners": 42.0,
"total": 42.0,
"total_time": "string"
},
"request_cache": {
"evictions": 42.0,
"hit_count": 42.0,
"memory_size": "string",
"memory_size_in_bytes": 42.0,
"miss_count": 42.0
},
"retention_leases": {
"primary_term": 42.0,
"version": 42.0,
"leases": [
{}
]
},
"routing": {
"node": "string",
"primary": true,
"state": "UNASSIGNED"
},
"search": {
"fetch_current": 42.0,
"fetch_time": "string",
"fetch_total": 42.0,
"open_contexts": 42.0,
"query_current": 42.0,
"query_time": "string",
"query_total": 42.0,
"scroll_current": 42.0,
"scroll_time": "string",
"scroll_total": 42.0,
"suggest_current": 42.0,
"suggest_time": "string",
"suggest_total": 42.0,
"groups": {}
},
"segments": {
"count": 42.0,
"doc_values_memory_in_bytes": 42.0,
"file_sizes": {},
"fixed_bit_set_memory_in_bytes": 42.0,
"index_writer_max_memory_in_bytes": 42.0,
"index_writer_memory_in_bytes": 42.0,
"max_unsafe_auto_id_timestamp": 42.0,
"memory_in_bytes": 42.0,
"norms_memory_in_bytes": 42.0,
"points_memory_in_bytes": 42.0,
"stored_fields_memory_in_bytes": 42.0,
"terms_memory_in_bytes": 42.0,
"term_vectors_memory_in_bytes": 42.0,
"version_map_memory_in_bytes": 42.0
},
"seq_no": {
"global_checkpoint": 42.0,
"local_checkpoint": 42.0,
"max_seq_no": 42.0
},
"store": {
"size_in_bytes": 42.0,
"reserved_in_bytes": 42.0,
"total_data_set_size_in_bytes": 42.0
},
"translog": {
"earliest_last_modified_age": 42.0,
"operations": 42.0,
"size": "string",
"size_in_bytes": 42.0,
"uncommitted_operations": 42.0,
"uncommitted_size": "string",
"uncommitted_size_in_bytes": 42.0
},
"warmer": {
"current": 42.0,
"total": 42.0,
"total_time": "string"
},
"bulk": {
"total_operations": 42.0,
"total_time": "string",
"total_size_in_bytes": 42.0,
"avg_time": "string",
"avg_size_in_bytes": 42.0
},
"shards": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"shard_stats": {
"total_count": 42.0
},
"additionalProperty1": {
"primaries": {},
"shards": {},
"total": {},
"uuid": "string",
"health": "green",
"status": "open"
},
"additionalProperty2": {
"primaries": {},
"shards": {},
"total": {},
"uuid": "string",
"health": "green",
"status": "open"
}
}
},
"additionalProperty2": {
"adaptive_selection": {
"additionalProperty1": {
"avg_queue_size": 42.0,
"avg_response_time": "string",
"avg_response_time_ns": 42.0,
"avg_service_time": "string",
"avg_service_time_ns": 42.0,
"outgoing_searches": 42.0,
"rank": "string"
},
"additionalProperty2": {
"avg_queue_size": 42.0,
"avg_response_time": "string",
"avg_response_time_ns": 42.0,
"avg_service_time": "string",
"avg_service_time_ns": 42.0,
"outgoing_searches": 42.0,
"rank": "string"
}
},
"breakers": {
"additionalProperty1": {
"estimated_size": "string",
"estimated_size_in_bytes": 42.0,
"limit_size": "string",
"limit_size_in_bytes": 42.0,
"overhead": 42.0,
"tripped": 42.0
},
"additionalProperty2": {
"estimated_size": "string",
"estimated_size_in_bytes": 42.0,
"limit_size": "string",
"limit_size_in_bytes": 42.0,
"overhead": 42.0,
"tripped": 42.0
}
},
"fs": {
"data": [
{}
],
"timestamp": 42.0,
"total": {
"available": "string",
"available_in_bytes": 42.0,
"free": "string",
"free_in_bytes": 42.0,
"total": "string",
"total_in_bytes": 42.0
},
"io_stats": {
"devices": [
{}
],
"total": {}
}
},
"host": "string",
"http": {
"current_open": 42.0,
"total_opened": 42.0,
"clients": [
{}
],
"routes": {
"additionalProperty1": {},
"additionalProperty2": {}
}
},
"ingest": {
"pipelines": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"total": {
"count": 42.0,
"current": 42.0,
"failed": 42.0
}
},
"ip": "string",
"jvm": {
"buffer_pools": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"classes": {
"current_loaded_count": 42.0,
"total_loaded_count": 42.0,
"total_unloaded_count": 42.0
},
"gc": {
"collectors": {}
},
"mem": {
"heap_used_in_bytes": 42.0,
"heap_used_percent": 42.0,
"heap_committed_in_bytes": 42.0,
"heap_max_in_bytes": 42.0,
"non_heap_used_in_bytes": 42.0,
"non_heap_committed_in_bytes": 42.0,
"pools": {}
},
"threads": {
"count": 42.0,
"peak_count": 42.0
},
"timestamp": 42.0,
"uptime": "string",
"uptime_in_millis": 42.0
},
"name": "string",
"os": {
"cpu": {
"percent": 42.0,
"sys": "string",
"total": "string",
"user": "string",
"load_average": {}
},
"": {},
"swap": {
"adjusted_total_in_bytes": 42.0,
"resident": "string",
"resident_in_bytes": 42.0,
"share": "string",
"share_in_bytes": 42.0,
"total_virtual": "string",
"total_virtual_in_bytes": 42.0,
"total_in_bytes": 42.0,
"free_in_bytes": 42.0,
"used_in_bytes": 42.0
},
"cgroup": {
"cpuacct": {},
"cpu": {},
"memory": {}
},
"timestamp": 42.0
},
"process": {
"cpu": {
"percent": 42.0,
"sys": "string",
"total": "string",
"user": "string",
"load_average": {}
},
"mem": {
"adjusted_total_in_bytes": 42.0,
"resident": "string",
"resident_in_bytes": 42.0,
"share": "string",
"share_in_bytes": 42.0,
"total_virtual": "string",
"total_virtual_in_bytes": 42.0,
"total_in_bytes": 42.0,
"free_in_bytes": 42.0,
"used_in_bytes": 42.0
},
"open_file_descriptors": 42.0,
"max_file_descriptors": 42.0,
"timestamp": 42.0
},
"roles": [
"master"
],
"script": {
"cache_evictions": 42.0,
"compilations": 42.0,
"compilations_history": {
"additionalProperty1": 42.0,
"additionalProperty2": 42.0
},
"compilation_limit_triggered": 42.0,
"contexts": [
{}
]
},
"script_cache": {},
"thread_pool": {
"additionalProperty1": {
"active": 42.0,
"completed": 42.0,
"largest": 42.0,
"queue": 42.0,
"rejected": 42.0,
"threads": 42.0
},
"additionalProperty2": {
"active": 42.0,
"completed": 42.0,
"largest": 42.0,
"queue": 42.0,
"rejected": 42.0,
"threads": 42.0
}
},
"timestamp": 42.0,
"transport": {
"inbound_handling_time_histogram": [
{}
],
"outbound_handling_time_histogram": [
{}
],
"rx_count": 42.0,
"rx_size": "string",
"rx_size_in_bytes": 42.0,
"server_open": 42.0,
"tx_count": 42.0,
"tx_size": "string",
"tx_size_in_bytes": 42.0,
"total_outbound_connections": 42.0
},
"transport_address": "string",
"attributes": {
"additionalProperty1": "string",
"additionalProperty2": "string"
},
"discovery": {
"cluster_state_queue": {
"total": 42.0,
"pending": 42.0,
"committed": 42.0
},
"published_cluster_states": {
"full_states": 42.0,
"incompatible_diffs": 42.0,
"compatible_diffs": 42.0
},
"cluster_state_update": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"serialized_cluster_states": {
"full_states": {},
"diffs": {}
},
"cluster_applier_stats": {
"recordings": [
{}
]
}
},
"indexing_pressure": {
"memory": {
"limit_in_bytes": 42.0,
"current": {},
"total": {}
}
},
"indices": {
"commit": {
"generation": 42.0,
"id": "string",
"num_docs": 42.0,
"user_data": {}
},
"completion": {
"size_in_bytes": 42.0,
"fields": {}
},
"docs": {
"count": 42.0,
"deleted": 42.0
},
"fielddata": {
"evictions": 42.0,
"memory_size_in_bytes": 42.0,
"fields": {}
},
"flush": {
"periodic": 42.0,
"total": 42.0,
"total_time": "string"
},
"get": {
"current": 42.0,
"exists_time": "string",
"exists_total": 42.0,
"missing_time": "string",
"missing_total": 42.0,
"time": "string",
"total": 42.0
},
"indexing": {
"index_current": 42.0,
"delete_current": 42.0,
"delete_time": "string",
"delete_total": 42.0,
"is_throttled": true,
"noop_update_total": 42.0,
"throttle_time": "string",
"index_time": "string",
"index_total": 42.0,
"index_failed": 42.0,
"types": {},
"write_load": 42.0,
"recent_write_load": 42.0,
"peak_write_load": 42.0
},
"mappings": {
"total_count": 42.0,
"total_estimated_overhead_in_bytes": 42.0
},
"merges": {
"current": 42.0,
"current_docs": 42.0,
"current_size": "string",
"current_size_in_bytes": 42.0,
"total": 42.0,
"total_auto_throttle": "string",
"total_auto_throttle_in_bytes": 42.0,
"total_docs": 42.0,
"total_size": "string",
"total_size_in_bytes": 42.0,
"total_stopped_time": "string",
"total_throttled_time": "string",
"total_time": "string"
},
"shard_path": {
"data_path": "string",
"is_custom_data_path": true,
"state_path": "string"
},
"query_cache": {
"cache_count": 42.0,
"cache_size": 42.0,
"evictions": 42.0,
"hit_count": 42.0,
"memory_size_in_bytes": 42.0,
"miss_count": 42.0,
"total_count": 42.0
},
"recovery": {
"current_as_source": 42.0,
"current_as_target": 42.0,
"throttle_time": "string"
},
"refresh": {
"external_total": 42.0,
"listeners": 42.0,
"total": 42.0,
"total_time": "string"
},
"request_cache": {
"evictions": 42.0,
"hit_count": 42.0,
"memory_size": "string",
"memory_size_in_bytes": 42.0,
"miss_count": 42.0
},
"retention_leases": {
"primary_term": 42.0,
"version": 42.0,
"leases": [
{}
]
},
"routing": {
"node": "string",
"primary": true,
"state": "UNASSIGNED"
},
"search": {
"fetch_current": 42.0,
"fetch_time": "string",
"fetch_total": 42.0,
"open_contexts": 42.0,
"query_current": 42.0,
"query_time": "string",
"query_total": 42.0,
"scroll_current": 42.0,
"scroll_time": "string",
"scroll_total": 42.0,
"suggest_current": 42.0,
"suggest_time": "string",
"suggest_total": 42.0,
"groups": {}
},
"segments": {
"count": 42.0,
"doc_values_memory_in_bytes": 42.0,
"file_sizes": {},
"fixed_bit_set_memory_in_bytes": 42.0,
"index_writer_max_memory_in_bytes": 42.0,
"index_writer_memory_in_bytes": 42.0,
"max_unsafe_auto_id_timestamp": 42.0,
"memory_in_bytes": 42.0,
"norms_memory_in_bytes": 42.0,
"points_memory_in_bytes": 42.0,
"stored_fields_memory_in_bytes": 42.0,
"terms_memory_in_bytes": 42.0,
"term_vectors_memory_in_bytes": 42.0,
"version_map_memory_in_bytes": 42.0
},
"seq_no": {
"global_checkpoint": 42.0,
"local_checkpoint": 42.0,
"max_seq_no": 42.0
},
"store": {
"size_in_bytes": 42.0,
"reserved_in_bytes": 42.0,
"total_data_set_size_in_bytes": 42.0
},
"translog": {
"earliest_last_modified_age": 42.0,
"operations": 42.0,
"size": "string",
"size_in_bytes": 42.0,
"uncommitted_operations": 42.0,
"uncommitted_size": "string",
"uncommitted_size_in_bytes": 42.0
},
"warmer": {
"current": 42.0,
"total": 42.0,
"total_time": "string"
},
"bulk": {
"total_operations": 42.0,
"total_time": "string",
"total_size_in_bytes": 42.0,
"avg_time": "string",
"avg_size_in_bytes": 42.0
},
"shards": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"shard_stats": {
"total_count": 42.0
},
"additionalProperty1": {
"primaries": {},
"shards": {},
"total": {},
"uuid": "string",
"health": "green",
"status": "open"
},
"additionalProperty2": {
"primaries": {},
"shards": {},
"total": {},
"uuid": "string",
"health": "green",
"status": "open"
}
}
}
}
}
Bulk index or delete documents
Perform multiple index
, create
, delete
, and update
actions in a single request.
This reduces overhead and can greatly increase indexing speed.
If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:
- To use the
create
action, you must have thecreate_doc
,create
,index
, orwrite
index privilege. Data streams support only thecreate
action. - To use the
index
action, you must have thecreate
,index
, orwrite
index privilege. - To use the
delete
action, you must have thedelete
orwrite
index privilege. - To use the
update
action, you must have theindex
orwrite
index privilege. - To automatically create a data stream or index with a bulk API request, you must have the
auto_configure
,create_index
, ormanage
index privilege. - To make the result of a bulk operation visible to search using the
refresh
parameter, you must have themaintenance
ormanage
index privilege.
Automatic data stream creation requires a matching index template with data stream enabled.
The actions are specified in the request body using a newline delimited JSON (NDJSON) structure:
action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n
The index
and create
actions expect a source on the next line and have the same semantics as the op_type
parameter in the standard index API.
A create
action fails if a document with the same ID already exists in the target
An index
action adds or replaces a document as necessary.
NOTE: Data streams support only the create
action.
To update or delete a document in a data stream, you must target the backing index containing the document.
An update
action expects that the partial doc, upsert, and script and its options are specified on the next line.
A delete
action does not expect a source on the next line and has the same semantics as the standard delete API.
NOTE: The final line of data must end with a newline character (\n
).
Each newline character may be preceded by a carriage return (\r
).
When sending NDJSON data to the _bulk
endpoint, use a Content-Type
header of application/json
or application/x-ndjson
.
Because this format uses literal newline characters (\n
) as delimiters, make sure that the JSON actions and sources are not pretty printed.
If you provide a target in the request path, it is used for any actions that don't explicitly specify an _index
argument.
A note on the format: the idea here is to make processing as fast as possible.
As some of the actions are redirected to other shards on other nodes, only action_meta_data
is parsed on the receiving node side.
Client libraries using this protocol should try and strive to do something similar on the client side, and reduce buffering as much as possible.
There is no "correct" number of actions to perform in a single bulk request. Experiment with different settings to find the optimal size for your particular workload. Note that Elasticsearch limits the maximum size of a HTTP request to 100mb by default so clients must ensure that no request exceeds this size. It is not possible to index a single document that exceeds the size limit, so you must pre-process any such documents into smaller pieces before sending them to Elasticsearch. For instance, split documents into pages or chapters before indexing them, or store raw binary data in a system outside Elasticsearch and replace the raw data with a link to the external system in the documents that you send to Elasticsearch.
Client suppport for bulk requests
Some of the officially supported clients provide helpers to assist with bulk requests and reindexing:
- Go: Check out
esutil.BulkIndexer
- Perl: Check out
Search::Elasticsearch::Client::5_0::Bulk
andSearch::Elasticsearch::Client::5_0::Scroll
- Python: Check out
elasticsearch.helpers.*
- JavaScript: Check out
client.helpers.*
- .NET: Check out
BulkAllObservable
- PHP: Check out bulk indexing.
Submitting bulk requests with cURL
If you're providing text file input to curl
, you must use the --data-binary
flag instead of plain -d
.
The latter doesn't preserve newlines. For example:
$ cat requests
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}
Optimistic concurrency control
Each index
and delete
action within a bulk API call may include the if_seq_no
and if_primary_term
parameters in their respective action and meta data lines.
The if_seq_no
and if_primary_term
parameters control how operations are run, based on the last modification to existing documents. See Optimistic concurrency control for more details.
Versioning
Each bulk item can include the version value using the version
field.
It automatically follows the behavior of the index or delete operation based on the _version
mapping.
It also support the version_type
.
Routing
Each bulk item can include the routing value using the routing
field.
It automatically follows the behavior of the index or delete operation based on the _routing
mapping.
NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing
setting enabled in the template.
Wait for active shards
When making bulk calls, you can set the wait_for_active_shards
parameter to require a minimum number of shard copies to be active before starting to process the bulk request.
Refresh
Control when the changes made by this request are visible to search.
NOTE: Only the shards that receive the bulk request will be affected by refresh.
Imagine a _bulk?refresh=wait_for
request with three documents in it that happen to be routed to different shards in an index with five shards.
The request will only wait for those three shards to refresh.
The other two shards that make up the index do not participate in the _bulk
request at all.
Path parameters
-
index
string Required The name of the data stream, index, or index alias to perform bulk actions on.
Query parameters
-
include_source_on_error
boolean True or false if to include the document source in the error message in case of parsing errors.
-
list_executed_pipelines
boolean If
true
, the response will include the ingest pipelines that were run for each index or create. -
pipeline
string The pipeline identifier to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to
_none
turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter. -
refresh
string If
true
, Elasticsearch refreshes the affected shards to make this operation visible to search. Ifwait_for
, wait for a refresh to make this operation visible to search. Iffalse
, do nothing with refreshes. Valid values:true
,false
,wait_for
.Values are
true
,false
, orwait_for
. -
routing
string A custom value that is used to route operations to a specific shard.
-
_source
boolean | string | array[string] Indicates whether to return the
_source
field (true
orfalse
) or contains a list of fields to return. -
_source_excludes
string | array[string] A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in
_source_includes
query parameter. If the_source
parameter isfalse
, this parameter is ignored. -
_source_includes
string | array[string] A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the
_source_excludes
query parameter. If the_source
parameter isfalse
, this parameter is ignored. -
timeout
string The period each action waits for the following operations: automatic index creation, dynamic mapping updates, and waiting for active shards. The default is
1m
(one minute), which guarantees Elasticsearch waits for at least the timeout before failing. The actual wait time could be longer, particularly when multiple waits occur. -
wait_for_active_shards
number | string The number of shard copies that must be active before proceeding with the operation. Set to
all
or any positive integer up to the total number of shards in the index (number_of_replicas+1
). The default is1
, which waits for each primary shard to be active. -
require_alias
boolean If
true
, the request's actions must target an index alias. -
require_data_stream
boolean If
true
, the request's actions must target a data stream (existing or to be created).
curl \
--request PUT 'http://api.example.com/{index}/_bulk' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{ \"index\" : { \"_index\" : \"test\", \"_id\" : \"1\" } }\n{ \"field1\" : \"value1\" }\n{ \"delete\" : { \"_index\" : \"test\", \"_id\" : \"2\" } }\n{ \"create\" : { \"_index\" : \"test\", \"_id\" : \"3\" } }\n{ \"field1\" : \"value3\" }\n{ \"update\" : {\"_id\" : \"1\", \"_index\" : \"test\"} }\n{ \"doc\" : {\"field2\" : \"value2\"} }"'
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }
{ "update" : {"_id" : "1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_index" : "index1", "retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}
{ "update": {"_id": "5", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "update": {"_id": "6", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "create": {"_id": "7", "_index": "index1"} }
{ "my_field": "foo" }
{ "index" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"work_location": "geo_point"}} }
{ "field" : "value1", "work_location": "41.12,-71.34", "raw_location": "41.12,-71.34"}
{ "create" : { "_index" : "my_index", "_id" : "2", "dynamic_templates": {"home_location": "geo_point"}} }
{ "field" : "value2", "home_location": "41.12,-71.34"}
{
"took": 30,
"errors": false,
"items": [
{
"index": {
"_index": "test",
"_id": "1",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 0,
"_primary_term": 1
}
},
{
"delete": {
"_index": "test",
"_id": "2",
"_version": 1,
"result": "not_found",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 404,
"_seq_no" : 1,
"_primary_term" : 2
}
},
{
"create": {
"_index": "test",
"_id": "3",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 201,
"_seq_no" : 2,
"_primary_term" : 3
}
},
{
"update": {
"_index": "test",
"_id": "1",
"_version": 2,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"status": 200,
"_seq_no" : 3,
"_primary_term" : 4
}
}
]
}
{
"took": 486,
"errors": true,
"items": [
{
"update": {
"_index": "index1",
"_id": "5",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"_index": "index1",
"_id": "6",
"status": 404,
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"create": {
"_index": "index1",
"_id": "7",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"_seq_no": 0,
"_primary_term": 1,
"status": 201
}
}
]
}
{
"items": [
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[5]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
},
{
"update": {
"error": {
"type": "document_missing_exception",
"reason": "[6]: document missing",
"index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
"shard": "0",
"index": "index1"
}
}
}
]
}
Update field mappings
Add new fields to an existing data stream or index. You can also use this API to change the search settings of existing fields and add new properties to existing object fields. For data streams, these changes are applied to all backing indices by default.
Add multi-fields to an existing field
Multi-fields let you index the same field in different ways. You can use this API to update the fields mapping parameter and enable multi-fields for an existing field. WARNING: If an index (or data stream) contains documents when you add a multi-field, those documents will not have values for the new multi-field. You can populate the new multi-field with the update by query API.
Change supported mapping parameters for an existing field
The documentation for each mapping parameter indicates whether you can update it for an existing field using this API.
For example, you can use the update mapping API to update the ignore_above
parameter.
Change the mapping of an existing field
Except for supported mapping parameters, you can't change the mapping or field type of an existing field. Changing an existing field could invalidate data that's already indexed.
If you need to change the mapping of a field in a data stream's backing indices, refer to documentation about modifying data streams. If you need to change the mapping of a field in other indices, create a new index with the correct mapping and reindex your data into that index.
Rename a field
Renaming a field would invalidate data already indexed under the old field name. Instead, add an alias field to create an alternate field name.
Path parameters
-
index
string | array[string] Required A comma-separated list of index names the mapping should be added to (supports wildcards); use
_all
or omit to add the mapping on all indices.
Query parameters
-
allow_no_indices
boolean If
false
, the request returns an error if any wildcard expression, index alias, or_all
value targets only missing or closed indices. This behavior applies even if the request targets other open indices. -
expand_wildcards
string | array[string] Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as
open,hidden
. Valid values are:all
,open
,closed
,hidden
,none
. -
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
-
timeout
string Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
-
write_index_only
boolean If
true
, the mappings are applied only to the current write index for the target.
Body
Required
-
date_detection
boolean Controls whether dynamic date detection is enabled.
-
dynamic
string Values are
strict
,runtime
,true
, orfalse
. -
dynamic_date_formats
array[string] If date detection is enabled then new string fields are checked against 'dynamic_date_formats' and if the value matches then a new date field is added instead of string.
-
dynamic_templates
array[object] Specify dynamic templates for the mapping.
-
_field_names
object -
_meta
object -
numeric_detection
boolean Automatically map strings into numeric data types for all fields.
-
properties
object Mapping for a field. For new fields, this mapping can include:
- Field name
- Field data type
- Mapping parameters
-
_routing
object -
_source
object -
runtime
object
curl \
--request POST 'http://api.example.com/{index}/_mapping' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"properties\": {\n \"user\": {\n \"properties\": {\n \"name\": {\n \"type\": \"keyword\"\n }\n }\n }\n }\n}"'
{
"properties": {
"user": {
"properties": {
"name": {
"type": "keyword"
}
}
}
}
}
{
"acknowledged": true,
"_shards": {
"failed": 42.0,
"successful": 42.0,
"total": 42.0,
"failures": [
{
"index": "string",
"node": "string",
"reason": {
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
},
"shard": 42.0,
"status": "string"
}
],
"skipped": 42.0
}
}
Query parameters
-
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
-
timeout
string Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request POST 'http://api.example.com/_aliases' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '{"actions":[{"add":{"alias":"string","aliases":"string","filter":{},"index":"string","indices":"string","index_routing":"string","is_hidden":true,"is_write_index":true,"routing":"string","search_routing":"string","must_exist":true},"remove":{"alias":"string","aliases":"string","index":"string","indices":"string","must_exist":true},"remove_index":{"index":"string","indices":"string","must_exist":true}}]}'
{
"actions": [
{
"add": {
"alias": "string",
"aliases": "string",
"filter": {},
"index": "string",
"indices": "string",
"index_routing": "string",
"is_hidden": true,
"is_write_index": true,
"routing": "string",
"search_routing": "string",
"must_exist": true
},
"remove": {
"alias": "string",
"aliases": "string",
"index": "string",
"indices": "string",
"must_exist": true
},
"remove_index": {
"index": "string",
"indices": "string",
"must_exist": true
}
}
]
}
{
"acknowledged": true
}
Create an Azure OpenAI inference endpoint
Added in 8.14.0
Create an inference endpoint to perform an inference task with the azureopenai
service.
The list of chat completion models that you can choose from in your Azure OpenAI deployment include:
The list of embeddings models that you can choose from in your deployment can be found in the Azure models documentation.
When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated"
in the response and ensure that the "allocation_count"
matches the "target_allocation_count"
.
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
Path parameters
-
task_type
string Required The type of the inference task that the model will perform. NOTE: The
chat_completion
task type only supports streaming and only through the _stream API.Values are
completion
ortext_embedding
. -
azureopenai_inference_id
string Required The unique identifier of the inference endpoint.
Body
-
chunking_settings
object -
service
string Required Value is
azureopenai
. -
service_settings
object Required -
task_settings
object
curl \
--request PUT 'http://api.example.com/_inference/{task_type}/{azureopenai_inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"service\": \"azureopenai\",\n \"service_settings\": {\n \"api_key\": \"Api-Key\",\n \"resource_name\": \"Resource-name\",\n \"deployment_id\": \"Deployment-id\",\n \"api_version\": \"2024-02-01\"\n }\n}"'
{
"service": "azureopenai",
"service_settings": {
"api_key": "Api-Key",
"resource_name": "Resource-name",
"deployment_id": "Deployment-id",
"api_version": "2024-02-01"
}
}
{
"service": "azureopenai",
"service_settings": {
"api_key": "Api-Key",
"resource_name": "Resource-name",
"deployment_id": "Deployment-id",
"api_version": "2024-02-01"
}
}
{
"chunking_settings": {
"max_chunk_size": 42.0,
"overlap": 42.0,
"sentence_overlap": 42.0,
"strategy": "string"
},
"service": "string",
"service_settings": {},
"task_settings": {},
"inference_id": "string",
"task_type": "sparse_embedding"
}
Perform text embedding inference on the service
Added in 8.11.0
Path parameters
-
inference_id
string Required The inference Id
Query parameters
-
timeout
string Specifies the amount of time to wait for the inference request to complete.
Body
input
string | array[string] Required Inference input. Either a string or an array of strings.
-
task_settings
object
curl \
--request POST 'http://api.example.com/_inference/text_embedding/{inference_id}' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"input\": \"The sky above the port was the color of television tuned to a dead channel.\",\n \"task_settings\": {\n \"input_type\": \"ingest\"\n }\n}"'
{
"input": "The sky above the port was the color of television tuned to a dead channel.",
"task_settings": {
"input_type": "ingest"
}
}
{
"text_embedding": [
{
"embedding": [
{
0.018569946,
-0.036895752,
0.01486969,
-0.0045204163,
-0.04385376,
0.0075950623,
0.04260254,
-0.004005432,
0.007865906,
0.030792236,
-0.050476074,
0.011795044,
-0.011642456,
-0.010070801
}
]
}
]
}
Get machine learning memory usage info
Added in 8.2.0
Get information about how machine learning jobs and trained models are using memory, on each node, both within the JVM heap, and natively, outside of the JVM.
Query parameters
-
master_timeout
string Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
-
timeout
string Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
curl \
--request GET 'http://api.example.com/_ml/memory/_stats' \
--header "Authorization: $API_KEY"
{
"_nodes": {
"failures": [
{
"type": "string",
"reason": "string",
"stack_trace": "string",
"caused_by": {},
"root_cause": [
{}
],
"suppressed": [
{}
]
}
],
"total": 42.0,
"successful": 42.0,
"failed": 42.0
},
"cluster_name": "string",
"nodes": {
"additionalProperty1": {
"attributes": {
"additionalProperty1": "string",
"additionalProperty2": "string"
},
"jvm": {
"": 42.0,
"heap_max_in_bytes": 42.0,
"java_inference_in_bytes": 42.0,
"java_inference_max_in_bytes": 42.0
},
"mem": {
"": 42.0,
"adjusted_total_in_bytes": 42.0,
"total_in_bytes": 42.0,
"ml": {
"": 42.0,
"anomaly_detectors_in_bytes": 42.0,
"data_frame_analytics_in_bytes": 42.0,
"max_in_bytes": 42.0,
"native_code_overhead_in_bytes": 42.0,
"native_inference_in_bytes": 42.0
}
},
"name": "string",
"roles": [
"string"
],
"transport_address": "string",
"ephemeral_id": "string"
},
"additionalProperty2": {
"attributes": {
"additionalProperty1": "string",
"additionalProperty2": "string"
},
"jvm": {
"": 42.0,
"heap_max_in_bytes": 42.0,
"java_inference_in_bytes": 42.0,
"java_inference_max_in_bytes": 42.0
},
"mem": {
"": 42.0,
"adjusted_total_in_bytes": 42.0,
"total_in_bytes": 42.0,
"ml": {
"": 42.0,
"anomaly_detectors_in_bytes": 42.0,
"data_frame_analytics_in_bytes": 42.0,
"max_in_bytes": 42.0,
"native_code_overhead_in_bytes": 42.0,
"native_inference_in_bytes": 42.0
}
},
"name": "string",
"roles": [
"string"
],
"transport_address": "string",
"ephemeral_id": "string"
}
}
}
Set upgrade_mode for ML indices
Added in 6.7.0
Sets a cluster wide upgrade_mode setting that prepares machine learning indices for an upgrade. When upgrading your cluster, in some circumstances you must restart your nodes and reindex your machine learning indices. In those circumstances, there must be no machine learning jobs running. You can close the machine learning jobs, do the upgrade, then open all the jobs again. Alternatively, you can use this API to temporarily halt tasks associated with the jobs and datafeeds and prevent new jobs from opening. You can also use this API during upgrades that do not require you to reindex your machine learning indices, though stopping jobs is not a requirement in that case. You can see the current value for the upgrade_mode setting by using the get machine learning info API.
curl \
--request POST 'http://api.example.com/_ml/set_upgrade_mode' \
--header "Authorization: $API_KEY"
{
"acknowledged": true
}
Delete anomaly jobs from a calendar
Added in 6.2.0
Path parameters
-
calendar_id
string Required A string that uniquely identifies a calendar.
-
job_id
string | array[string] Required An identifier for the anomaly detection jobs. It can be a job identifier, a group name, or a comma-separated list of jobs or groups.
curl \
--request DELETE 'http://api.example.com/_ml/calendars/{calendar_id}/jobs/{job_id}' \
--header "Authorization: $API_KEY"
{
"calendar_id": "planned-outages",
"job_ids": []
}
Delete an anomaly detection job
Added in 5.4.0
All job configuration, model state and results are deleted. It is not currently possible to delete multiple jobs using wildcards or a comma separated list. If you delete a job that has a datafeed, the request first tries to delete the datafeed. This behavior is equivalent to calling the delete datafeed API with the same timeout and force parameters as the delete job request.
Path parameters
-
job_id
string Required Identifier for the anomaly detection job.
Query parameters
-
force
boolean Use to forcefully delete an opened job; this method is quicker than closing and deleting the job.
-
delete_user_annotations
boolean Specifies whether annotations that have been added by the user should be deleted along with any auto-generated annotations when the job is reset.
-
wait_for_completion
boolean Specifies whether the request should return immediately or wait until the job deletion completes.
curl \
--request DELETE 'http://api.example.com/_ml/anomaly_detectors/{job_id}' \
--header "Authorization: $API_KEY"
{
"acknowledged": true
}
{
"task": "oTUltX4IQMOUUVeiohTt8A:39"
}
Get datafeeds configuration info
Added in 5.5.0
You can get information for multiple datafeeds in a single API request by
using a comma-separated list of datafeeds or a wildcard expression. You can
get information for all datafeeds by using _all
, by specifying *
as the
<feed_id>
, or by omitting the <feed_id>
.
This API returns a maximum of 10,000 datafeeds.
Query parameters
-
allow_no_match
boolean Specifies what to do when the request:
- Contains wildcard expressions and there are no datafeeds that match.
- Contains the
_all
string or no identifiers and there are no matches. - Contains wildcard expressions and there are only partial matches.
The default value is
true
, which returns an emptydatafeeds
array when there are no matches and the subset of results when there are partial matches. If this parameter isfalse
, the request returns a404
status code when there are no matches or only partial matches. -
exclude_generated
boolean Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.
curl \
--request GET 'http://api.example.com/_ml/datafeeds' \
--header "Authorization: $API_KEY"
{
"count": 42.0,
"datafeeds": [
{
"aggregations": {},
"authorization": {
"api_key": {
"id": "string",
"name": "string"
},
"roles": [
"string"
],
"service_account": "string"
},
"chunking_config": {
"mode": "auto",
"time_span": "string"
},
"datafeed_id": "string",
"frequency": "string",
"indices": [
"string"
],
"indexes": [
"string"
],
"job_id": "string",
"max_empty_searches": 42.0,
"query_delay": "string",
"script_fields": {
"additionalProperty1": {
"script": {
"id": "string",
"params": {},
"options": {}
},
"ignore_failure": true
},
"additionalProperty2": {
"script": {
"id": "string",
"params": {},
"options": {}
},
"ignore_failure": true
}
},
"scroll_size": 42.0,
"delayed_data_check_config": {
"check_window": "string",
"enabled": true
},
"runtime_mappings": {
"additionalProperty1": {
"fields": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"fetch_fields": [
{}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"id": "string",
"params": {},
"options": {}
},
"type": "boolean"
},
"additionalProperty2": {
"fields": {
"additionalProperty1": {},
"additionalProperty2": {}
},
"fetch_fields": [
{}
],
"format": "string",
"input_field": "string",
"target_field": "string",
"target_index": "string",
"script": {
"id": "string",
"params": {},
"options": {}
},
"type": "boolean"
}
},
"indices_options": {
"allow_no_indices": true,
"expand_wildcards": "string",
"ignore_unavailable": true,
"ignore_throttled": true
},
"query": {}
}
]
}
Explain data frame analytics config
Added in 7.3.0
This API provides explanations for a data frame analytics config that either exists already or one that has not been created yet. The following explanations are provided:
- which fields are included or not in the analysis and why,
- how much memory is estimated to be required. The estimate can be used when deciding the appropriate value for model_memory_limit setting later on. If you have object fields or fields that are excluded via source filtering, they are not included in the explanation.
Path parameters
-
id
string Required Identifier for the data frame analytics job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.
Body
-
source
object -
dest
object -
analysis
object -
description
string A description of the job.
-
model_memory_limit
string The approximate maximum amount of memory resources that are permitted for analytical processing. If your
elasticsearch.yml
file contains anxpack.ml.max_model_memory_limit
setting, an error occurs when you try to create data frame analytics jobs that havemodel_memory_limit
values greater than that setting. -
max_num_threads
number The maximum number of threads to be used by the analysis. Using more threads may decrease the time necessary to complete the analysis at the cost of using more CPU. Note that the process may use additional threads for operational functionality other than the analysis itself.
-
analyzed_fields
object -
allow_lazy_start
boolean Specifies whether this job can start when there is insufficient machine learning node capacity for it to be immediately assigned to a node.
curl \
--request POST 'http://api.example.com/_ml/data_frame/analytics/{id}/_explain' \
--header "Authorization: $API_KEY" \
--header "Content-Type: application/json" \
--data '"{\n \"source\": {\n \"index\": \"houses_sold_last_10_yrs\"\n },\n \"analysis\": {\n \"regression\": {\n \"dependent_variable\": \"price\"\n }\n }\n}"'
{
"source": {
"index": "houses_sold_last_10_yrs"
},
"analysis": {
"regression": {
"dependent_variable": "price"
}
}
}
{
"field_selection": [
{
"field": "number_of_bedrooms",
"mappings_types": [
"integer"
],
"is_included": true,
"is_required": false,
"feature_type": "numerical"
},
{
"field": "postcode",
"mappings_types": [
"text"
],
"is_included": false,
"is_required": false,
"reason": "[postcode.keyword] is preferred because it is aggregatable"
},
{
"field": "postcode.keyword",
"mappings_types": [
"keyword"
],
"is_included": true,
"is_required": false,
"feature_type": "categorical"
},
{
"field": "price",
"mappings_types": [
"float"
],
"is_included": true,
"is_required": true,
"feature_type": "numerical"
}
],
"memory_estimation": {
"expected_memory_without_disk": "128MB",
"expected_memory_with_disk": "32MB"
}
}