Get index information

GET /_cat/indices
Api key auth Basic auth Bearer auth

Get high-level information about indices in a cluster, including backing indices for data streams.

Use this request to get the following information for each index in a cluster:

  • shard count
  • document count
  • deleted document count
  • primary store size
  • total store size of all shards, including shard replicas

These metrics are retrieved directly from Lucene, which Elasticsearch uses internally to power indexing and search. As a result, all document counts include hidden nested documents. To get an accurate count of Elasticsearch documents, use the cat count or count APIs.

CAT APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use an index endpoint.

Query parameters

  • bytes string

    The unit used to display byte values.

    Values are b, kb, mb, gb, tb, or pb.

  • expand_wildcards string | array[string]

    The type of index that wildcard patterns can match.

  • health string

    The health status used to limit returned indices. By default, the response includes indices of any health status.

    Values are green, GREEN, yellow, YELLOW, red, or RED.

  • include_unloaded_segments boolean

    If true, the response includes information from segments that are not loaded into memory.

  • pri boolean

    If true, the response only includes information from primary shards.

  • time string

    The unit used to display time values.

    Values are nanos, micros, ms, s, m, h, or d.

  • master_timeout string

    Period to wait for a connection to the master node.

  • h string | array[string]

    List of columns to appear in the response. Supports simple wildcards.

  • s string | array[string]

    List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.

Responses

GET /_cat/indices 
curl \
 --request GET 'http://api.example.com/_cat/indices' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_cat/indices/my-index-*?v=true&s=index&format=json`. 
[
  {
    "health": "yellow",
    "status": "open",
    "index": "my-index-000001",
    "uuid": "u8FNjxh8Rfy_awN11oDKYQ",
    "pri": "1",
    "rep": "1",
    "docs.count": "1200",
    "docs.deleted": "0",
    "store.size": "88.1kb",
    "pri.store.size": "88.1kb",
    "dataset.size": "88.1kb"
  },
  {
    "health": "green",
    "status": "open",
    "index": "my-index-000002",
    "uuid": "nYFWZEO7TUiOjLQXBaYJpA ",
    "pri": "1",
    "rep": "0",
    "docs.count": "0",
    "docs.deleted": "0",
    "store.size": "260b",
    "pri.store.size": "260b",
    "dataset.size": "260b"
  }
]

Get cluster repositories metering Technical preview

GET /_nodes/{node_id}/_repositories_metering
Api key auth Basic auth Bearer auth

Get repositories metering information for a cluster. This API exposes monotonically non-decreasing counters and it is expected that clients would durably store the information needed to compute aggregations over a period of time. Additionally, the information exposed by this API is volatile, meaning that it will not be present after node restarts.

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information. All the nodes selective options are explained here.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object
      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]
        Hide failures attributes Show failures attributes object
      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required

      Contains repositories metering information for the nodes selected by the request.

      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • repository_name string Required
        • repository_type string Required

          Repository type.

        • repository_location object Required
          Hide repository_location attributes Show repository_location attributes object
        • repository_ephemeral_id string Required

        • Time unit for milliseconds

        • Time unit for milliseconds

        • archived boolean Required

          A flag that tells whether or not this object has been archived. When a repository is closed or updated the repository metering information is archived and kept for a certain period of time. This allows retrieving the repository metering information of previous repository instantiations.

        • cluster_version number
        • request_counts object Required
          Hide request_counts attributes Show request_counts attributes object
          • GetBlobProperties number

            Number of Get Blob Properties requests (Azure)

          • GetBlob number

            Number of Get Blob requests (Azure)

          • ListBlobs number

            Number of List Blobs requests (Azure)

          • PutBlob number

            Number of Put Blob requests (Azure)

          • PutBlock number

            Number of Put Block (Azure)

          • PutBlockList number

            Number of Put Block List requests

          • GetObject number

            Number of get object requests (GCP, S3)

          • ListObjects number

            Number of list objects requests (GCP, S3)

          • InsertObject number

            Number of insert object requests, including simple, multipart and resumable uploads. Resumable uploads can perform multiple http requests to insert a single object but they are considered as a single request since they are billed as an individual operation. (GCP)

          • PutObject number

            Number of PutObject requests (S3)

          • PutMultipartObject number

            Number of Multipart requests, including CreateMultipartUpload, UploadPart and CompleteMultipartUpload requests (S3)
GET /_nodes/{node_id}/_repositories_metering 
curl \
 --request GET 'http://api.example.com/_nodes/{node_id}/_repositories_metering' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "_nodes": {
    "failures": [
      {
        "type": "string",
        "reason": "string",
        "stack_trace": "string",
        "caused_by": {},
        "root_cause": [
          {}
        ],
        "suppressed": [
          {}
        ]
      }
    ],
    "total": 42.0,
    "successful": 42.0,
    "failed": 42.0
  },
  "cluster_name": "string",
  "nodes": {
    "additionalProperty1": {
      "repository_name": "string",
      "repository_type": "string",
      "repository_location": {
        "base_path": "string",
        "container": "string",
        "bucket": "string"
      },
      "repository_ephemeral_id": "string",
      "": 42.0,
      "archived": true,
      "cluster_version": 42.0,
      "request_counts": {
        "GetBlobProperties": 42.0,
        "GetBlob": 42.0,
        "ListBlobs": 42.0,
        "PutBlob": 42.0,
        "PutBlock": 42.0,
        "PutBlockList": 42.0,
        "GetObject": 42.0,
        "ListObjects": 42.0,
        "InsertObject": 42.0,
        "PutObject": 42.0,
        "PutMultipartObject": 42.0
      }
    },
    "additionalProperty2": {
      "repository_name": "string",
      "repository_type": "string",
      "repository_location": {
        "base_path": "string",
        "container": "string",
        "bucket": "string"
      },
      "repository_ephemeral_id": "string",
      "": 42.0,
      "archived": true,
      "cluster_version": 42.0,
      "request_counts": {
        "GetBlobProperties": 42.0,
        "GetBlob": 42.0,
        "ListBlobs": 42.0,
        "PutBlob": 42.0,
        "PutBlock": 42.0,
        "PutBlockList": 42.0,
        "GetObject": 42.0,
        "ListObjects": 42.0,
        "InsertObject": 42.0,
        "PutObject": 42.0,
        "PutMultipartObject": 42.0
      }
    }
  }
}

Get node statistics

GET /_nodes/{node_id}/stats/{metric}
Api key auth Basic auth Bearer auth

Get statistics for nodes in a cluster. By default, all stats are returned. You can limit the returned information by using metrics.

Path parameters

  • node_id string | array[string] Required

    Comma-separated list of node IDs or names used to limit returned information.

  • metric string | array[string] Required

    Limit the information returned to the specified metrics

Query parameters

  • completion_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata and suggest statistics.

  • fielddata_fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in fielddata statistics.

  • fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in the statistics.

  • groups boolean

    Comma-separated list of search groups to include in the search statistics.

  • include_segment_file_sizes boolean

    If true, the call reports the aggregated disk usage of each one of the Lucene index files (only applies if segment stats are requested).

  • level string

    Indicates whether statistics are aggregated at the cluster, index, or shard level.

    Values are cluster, indices, or shards.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

  • types array[string]

    A comma-separated list of document types for the indexing index metric.

  • include_unloaded_segments boolean

    If true, the response includes information from segments that are not loaded into memory.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object
      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]
        Hide failures attributes Show failures attributes object
      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string
    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • adaptive_selection object

          Statistics about adaptive replica selection.

          Hide adaptive_selection attribute Show adaptive_selection attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • avg_queue_size number

              The exponentially weighted moving average queue size of search requests on the keyed node.

            • avg_response_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_response_time_ns number

              The exponentially weighted moving average response time, in nanoseconds, of search requests on the keyed node.

            • avg_service_time string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • avg_service_time_ns number

              The exponentially weighted moving average service time, in nanoseconds, of search requests on the keyed node.

            • outgoing_searches number

              The number of outstanding search requests to the keyed node from the node these stats are for.

            • rank string

              The rank of this node; used for shard selection when routing search requests.

        • breakers object

          Statistics about the field data circuit breaker.

          Hide breakers attribute Show breakers attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • estimated_size string

              Estimated memory used for the operation.

            • estimated_size_in_bytes number

              Estimated memory used, in bytes, for the operation.

            • limit_size string

              Memory limit for the circuit breaker.

            • limit_size_in_bytes number

              Memory limit, in bytes, for the circuit breaker.

            • overhead number

              A constant that all estimates for the circuit breaker are multiplied with to calculate a final estimate.

            • tripped number

              Total number of times the circuit breaker has been triggered and prevented an out of memory error.

        • fs object
          Hide fs attributes Show fs attributes object
          • data array[object]

            List of all file stores.

          • timestamp number

            Last time the file stores statistics were refreshed. Recorded in milliseconds since the Unix Epoch.

          • total object
            Hide total attributes Show total attributes object
            • available string

              Total disk space available to this Java virtual machine on all file stores. Depending on OS or process level restrictions, this might appear less than free. This is the actual amount of free disk space the Elasticsearch node can utilise.

            • available_in_bytes number

              Total number of bytes available to this Java virtual machine on all file stores. Depending on OS or process level restrictions, this might appear less than free_in_bytes. This is the actual amount of free disk space the Elasticsearch node can utilise.

            • free string

              Total unallocated disk space in all file stores.

            • free_in_bytes number

              Total number of unallocated bytes in all file stores.

            • total string

              Total size of all file stores.

            • total_in_bytes number

              Total size of all file stores in bytes.

          • io_stats object
            Hide io_stats attributes Show io_stats attributes object
            • devices array[object]

              Array of disk metrics for each device that is backing an Elasticsearch data path. These disk metrics are probed periodically and averages between the last probe and the current probe are computed.

            • total object
        • host string
        • http object
          Hide http attributes Show http attributes object
          • current_open number

            Current number of open HTTP connections for the node.

          • total_opened number

            Total number of HTTP connections opened for the node.

          • clients array[object]

            Information on current and recently-closed HTTP client connections. Clients that have been closed longer than the http.client_stats.closed_channels.max_age setting will not be represented here.

          • routes object Required Added in 8.12.0

            Detailed HTTP stats broken down by route

            Hide routes attribute Show routes attribute object
            • * object Additional properties
        • ingest object
          Hide ingest attributes Show ingest attributes object
          • pipelines object

            Contains statistics about ingest pipelines for the node.

            Hide pipelines attribute Show pipelines attribute object
            • * object Additional properties
          • total object
            Hide total attributes Show total attributes object
            • count number Required

              Total number of documents ingested during the lifetime of this node.

            • current number Required

              Total number of documents currently being ingested.

            • failed number Required

              Total number of failed ingest operations during the lifetime of this node.

        • ip string | array[string]

          IP address and port for the node.

          One of:
          Ip string array-2 array[string]
        • jvm object
          Hide jvm attributes Show jvm attributes object
          • buffer_pools object

            Contains statistics about JVM buffer pools for the node.

            Hide buffer_pools attribute Show buffer_pools attribute object
            • * object Additional properties
          • classes object
            Hide classes attributes Show classes attributes object
          • gc object
            Hide gc attribute Show gc attribute object
            • collectors object

              Contains statistics about JVM garbage collectors for the node.

          • mem object
            Hide mem attributes Show mem attributes object
          • threads object
            Hide threads attributes Show threads attributes object
            • count number

              Number of active threads in use by JVM.

            • peak_count number

              Highest number of threads used by JVM.

          • timestamp number

            Last time JVM statistics were refreshed.

          • uptime string

            Human-readable JVM uptime. Only returned if the human query parameter is true.

          • uptime_in_millis number

            JVM uptime in milliseconds.

        • name string
        • os object
          Hide os attributes Show os attributes object
          • cpu object
            Hide cpu attributes Show cpu attributes object
            • percent number
            • sys string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • user string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • load_average object
          • swap object
            Hide swap attributes Show swap attributes object
          • cgroup object
            Hide cgroup attributes Show cgroup attributes object
          • timestamp number
        • process object
          Hide process attributes Show process attributes object
          • cpu object
            Hide cpu attributes Show cpu attributes object
            • percent number
            • sys string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • total string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • user string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • load_average object
          • mem object
            Hide mem attributes Show mem attributes object
          • open_file_descriptors number

            Number of opened file descriptors associated with the current or -1 if not supported.

          • max_file_descriptors number

            Maximum number of file descriptors allowed on the system, or -1 if not supported.

          • timestamp number

            Last time the statistics were refreshed. Recorded in milliseconds since the Unix Epoch.

        • roles array[string]
          • @doc_id node-roles

          Values are master, data, data_cold, data_content, data_frozen, data_hot, data_warm, client, ingest, ml, voting_only, transform, remote_cluster_client, or coordinating_only.

        • script object
          Hide script attributes Show script attributes object
          • cache_evictions number

            Total number of times the script cache has evicted old data.

          • compilations number

            Total number of inline script compilations performed by the node.

          • compilations_history object

            Contains this recent history of script compilations.

            Hide compilations_history attribute Show compilations_history attribute object
            • * number Additional properties
          • compilation_limit_triggered number

            Total number of times the script compilation circuit breaker has limited inline script compilations.

          • contexts array[object]
        • script_cache object
        • thread_pool object

          Statistics about each thread pool, including current size, queue and rejected tasks.

          Hide thread_pool attribute Show thread_pool attribute object
          • * object Additional properties
            Hide * attributes Show * attributes object
            • active number

              Number of active threads in the thread pool.

            • completed number

              Number of tasks completed by the thread pool executor.

            • largest number

              Highest number of active threads in the thread pool.

            • queue number

              Number of tasks in queue for the thread pool.

            • rejected number

              Number of tasks rejected by the thread pool executor.

            • threads number

              Number of threads in the thread pool.

        • timestamp number
        • transport object
          Hide transport attributes Show transport attributes object
          • inbound_handling_time_histogram array[object]

            The distribution of the time spent handling each inbound message on a transport thread, represented as a histogram.

          • outbound_handling_time_histogram array[object]

            The distribution of the time spent sending each outbound transport message on a transport thread, represented as a histogram.

          • rx_count number

            Total number of RX (receive) packets received by the node during internal cluster communication.

          • rx_size string

            Size of RX packets received by the node during internal cluster communication.

          • rx_size_in_bytes number

            Size, in bytes, of RX packets received by the node during internal cluster communication.

          • server_open number

            Current number of inbound TCP connections used for internal communication between nodes.

          • tx_count number

            Total number of TX (transmit) packets sent by the node during internal cluster communication.

          • tx_size string

            Size of TX packets sent by the node during internal cluster communication.

          • tx_size_in_bytes number

            Size, in bytes, of TX packets sent by the node during internal cluster communication.

          • total_outbound_connections number

            The cumulative number of outbound transport connections that this node has opened since it started. Each transport connection may comprise multiple TCP connections but is only counted once in this statistic. Transport connections are typically long-lived so this statistic should remain constant in a stable cluster.

        • transport_address string
        • attributes object

          Contains a list of attributes for the node.

          Hide attributes attribute Show attributes attribute object
          • * string Additional properties
        • discovery object
          Hide discovery attributes Show discovery attributes object
          • cluster_state_queue object
            Hide cluster_state_queue attributes Show cluster_state_queue attributes object
            • total number

              Total number of cluster states in queue.

            • pending number

              Number of pending cluster states in queue.

            • committed number

              Number of committed cluster states in queue.

          • published_cluster_states object
            Hide published_cluster_states attributes Show published_cluster_states attributes object
          • cluster_state_update object

            Contains low-level statistics about how long various activities took during cluster state updates while the node was the elected master. Omitted if the node is not master-eligible. Every field whose name ends in _time within this object is also represented as a raw number of milliseconds in a field whose name ends in _time_millis. The human-readable fields with a _time suffix are only returned if requested with the ?human=true query parameter.

            Hide cluster_state_update attribute Show cluster_state_update attribute object
            • * object Additional properties
          • serialized_cluster_states object
            Hide serialized_cluster_states attributes Show serialized_cluster_states attributes object
          • cluster_applier_stats object
            Hide cluster_applier_stats attribute Show cluster_applier_stats attribute object
        • indexing_pressure object
          Hide indexing_pressure attribute Show indexing_pressure attribute object
          • memory object
            Hide memory attributes Show memory attributes object
        • indices object
          Hide indices attributes Show indices attributes object
GET /_nodes/{node_id}/stats/{metric} 
curl \
 --request GET 'http://api.example.com/_nodes/{node_id}/stats/{metric}' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "_nodes": {
    "failures": [
      {
        "type": "string",
        "reason": "string",
        "stack_trace": "string",
        "caused_by": {},
        "root_cause": [
          {}
        ],
        "suppressed": [
          {}
        ]
      }
    ],
    "total": 42.0,
    "successful": 42.0,
    "failed": 42.0
  },
  "cluster_name": "string",
  "nodes": {
    "additionalProperty1": {
      "adaptive_selection": {
        "additionalProperty1": {
          "avg_queue_size": 42.0,
          "avg_response_time": "string",
          "avg_response_time_ns": 42.0,
          "avg_service_time": "string",
          "avg_service_time_ns": 42.0,
          "outgoing_searches": 42.0,
          "rank": "string"
        },
        "additionalProperty2": {
          "avg_queue_size": 42.0,
          "avg_response_time": "string",
          "avg_response_time_ns": 42.0,
          "avg_service_time": "string",
          "avg_service_time_ns": 42.0,
          "outgoing_searches": 42.0,
          "rank": "string"
        }
      },
      "breakers": {
        "additionalProperty1": {
          "estimated_size": "string",
          "estimated_size_in_bytes": 42.0,
          "limit_size": "string",
          "limit_size_in_bytes": 42.0,
          "overhead": 42.0,
          "tripped": 42.0
        },
        "additionalProperty2": {
          "estimated_size": "string",
          "estimated_size_in_bytes": 42.0,
          "limit_size": "string",
          "limit_size_in_bytes": 42.0,
          "overhead": 42.0,
          "tripped": 42.0
        }
      },
      "fs": {
        "data": [
          {}
        ],
        "timestamp": 42.0,
        "total": {
          "available": "string",
          "available_in_bytes": 42.0,
          "free": "string",
          "free_in_bytes": 42.0,
          "total": "string",
          "total_in_bytes": 42.0
        },
        "io_stats": {
          "devices": [
            {}
          ],
          "total": {}
        }
      },
      "host": "string",
      "http": {
        "current_open": 42.0,
        "total_opened": 42.0,
        "clients": [
          {}
        ],
        "routes": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        }
      },
      "ingest": {
        "pipelines": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "total": {
          "count": 42.0,
          "current": 42.0,
          "failed": 42.0
        }
      },
      "ip": "string",
      "jvm": {
        "buffer_pools": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "classes": {
          "current_loaded_count": 42.0,
          "total_loaded_count": 42.0,
          "total_unloaded_count": 42.0
        },
        "gc": {
          "collectors": {}
        },
        "mem": {
          "heap_used_in_bytes": 42.0,
          "heap_used_percent": 42.0,
          "heap_committed_in_bytes": 42.0,
          "heap_max_in_bytes": 42.0,
          "non_heap_used_in_bytes": 42.0,
          "non_heap_committed_in_bytes": 42.0,
          "pools": {}
        },
        "threads": {
          "count": 42.0,
          "peak_count": 42.0
        },
        "timestamp": 42.0,
        "uptime": "string",
        "uptime_in_millis": 42.0
      },
      "name": "string",
      "os": {
        "cpu": {
          "percent": 42.0,
          "sys": "string",
          "total": "string",
          "user": "string",
          "load_average": {}
        },
        "": {},
        "swap": {
          "adjusted_total_in_bytes": 42.0,
          "resident": "string",
          "resident_in_bytes": 42.0,
          "share": "string",
          "share_in_bytes": 42.0,
          "total_virtual": "string",
          "total_virtual_in_bytes": 42.0,
          "total_in_bytes": 42.0,
          "free_in_bytes": 42.0,
          "used_in_bytes": 42.0
        },
        "cgroup": {
          "cpuacct": {},
          "cpu": {},
          "memory": {}
        },
        "timestamp": 42.0
      },
      "process": {
        "cpu": {
          "percent": 42.0,
          "sys": "string",
          "total": "string",
          "user": "string",
          "load_average": {}
        },
        "mem": {
          "adjusted_total_in_bytes": 42.0,
          "resident": "string",
          "resident_in_bytes": 42.0,
          "share": "string",
          "share_in_bytes": 42.0,
          "total_virtual": "string",
          "total_virtual_in_bytes": 42.0,
          "total_in_bytes": 42.0,
          "free_in_bytes": 42.0,
          "used_in_bytes": 42.0
        },
        "open_file_descriptors": 42.0,
        "max_file_descriptors": 42.0,
        "timestamp": 42.0
      },
      "roles": [
        "master"
      ],
      "script": {
        "cache_evictions": 42.0,
        "compilations": 42.0,
        "compilations_history": {
          "additionalProperty1": 42.0,
          "additionalProperty2": 42.0
        },
        "compilation_limit_triggered": 42.0,
        "contexts": [
          {}
        ]
      },
      "script_cache": {},
      "thread_pool": {
        "additionalProperty1": {
          "active": 42.0,
          "completed": 42.0,
          "largest": 42.0,
          "queue": 42.0,
          "rejected": 42.0,
          "threads": 42.0
        },
        "additionalProperty2": {
          "active": 42.0,
          "completed": 42.0,
          "largest": 42.0,
          "queue": 42.0,
          "rejected": 42.0,
          "threads": 42.0
        }
      },
      "timestamp": 42.0,
      "transport": {
        "inbound_handling_time_histogram": [
          {}
        ],
        "outbound_handling_time_histogram": [
          {}
        ],
        "rx_count": 42.0,
        "rx_size": "string",
        "rx_size_in_bytes": 42.0,
        "server_open": 42.0,
        "tx_count": 42.0,
        "tx_size": "string",
        "tx_size_in_bytes": 42.0,
        "total_outbound_connections": 42.0
      },
      "transport_address": "string",
      "attributes": {
        "additionalProperty1": "string",
        "additionalProperty2": "string"
      },
      "discovery": {
        "cluster_state_queue": {
          "total": 42.0,
          "pending": 42.0,
          "committed": 42.0
        },
        "published_cluster_states": {
          "full_states": 42.0,
          "incompatible_diffs": 42.0,
          "compatible_diffs": 42.0
        },
        "cluster_state_update": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "serialized_cluster_states": {
          "full_states": {},
          "diffs": {}
        },
        "cluster_applier_stats": {
          "recordings": [
            {}
          ]
        }
      },
      "indexing_pressure": {
        "memory": {
          "limit_in_bytes": 42.0,
          "current": {},
          "total": {}
        }
      },
      "indices": {
        "commit": {
          "generation": 42.0,
          "id": "string",
          "num_docs": 42.0,
          "user_data": {}
        },
        "completion": {
          "size_in_bytes": 42.0,
          "fields": {}
        },
        "docs": {
          "count": 42.0,
          "deleted": 42.0
        },
        "fielddata": {
          "evictions": 42.0,
          "memory_size_in_bytes": 42.0,
          "fields": {}
        },
        "flush": {
          "periodic": 42.0,
          "total": 42.0,
          "total_time": "string"
        },
        "get": {
          "current": 42.0,
          "exists_time": "string",
          "exists_total": 42.0,
          "missing_time": "string",
          "missing_total": 42.0,
          "time": "string",
          "total": 42.0
        },
        "indexing": {
          "index_current": 42.0,
          "delete_current": 42.0,
          "delete_time": "string",
          "delete_total": 42.0,
          "is_throttled": true,
          "noop_update_total": 42.0,
          "throttle_time": "string",
          "index_time": "string",
          "index_total": 42.0,
          "index_failed": 42.0,
          "types": {},
          "write_load": 42.0
        },
        "mappings": {
          "total_count": 42.0,
          "total_estimated_overhead_in_bytes": 42.0
        },
        "merges": {
          "current": 42.0,
          "current_docs": 42.0,
          "current_size": "string",
          "current_size_in_bytes": 42.0,
          "total": 42.0,
          "total_auto_throttle": "string",
          "total_auto_throttle_in_bytes": 42.0,
          "total_docs": 42.0,
          "total_size": "string",
          "total_size_in_bytes": 42.0,
          "total_stopped_time": "string",
          "total_throttled_time": "string",
          "total_time": "string"
        },
        "shard_path": {
          "data_path": "string",
          "is_custom_data_path": true,
          "state_path": "string"
        },
        "query_cache": {
          "cache_count": 42.0,
          "cache_size": 42.0,
          "evictions": 42.0,
          "hit_count": 42.0,
          "memory_size_in_bytes": 42.0,
          "miss_count": 42.0,
          "total_count": 42.0
        },
        "recovery": {
          "current_as_source": 42.0,
          "current_as_target": 42.0,
          "throttle_time": "string"
        },
        "refresh": {
          "external_total": 42.0,
          "listeners": 42.0,
          "total": 42.0,
          "total_time": "string"
        },
        "request_cache": {
          "evictions": 42.0,
          "hit_count": 42.0,
          "memory_size": "string",
          "memory_size_in_bytes": 42.0,
          "miss_count": 42.0
        },
        "retention_leases": {
          "primary_term": 42.0,
          "version": 42.0,
          "leases": [
            {}
          ]
        },
        "routing": {
          "node": "string",
          "primary": true,
          "state": "UNASSIGNED"
        },
        "search": {
          "fetch_current": 42.0,
          "fetch_time": "string",
          "fetch_total": 42.0,
          "open_contexts": 42.0,
          "query_current": 42.0,
          "query_time": "string",
          "query_total": 42.0,
          "scroll_current": 42.0,
          "scroll_time": "string",
          "scroll_total": 42.0,
          "suggest_current": 42.0,
          "suggest_time": "string",
          "suggest_total": 42.0,
          "groups": {}
        },
        "segments": {
          "count": 42.0,
          "doc_values_memory_in_bytes": 42.0,
          "file_sizes": {},
          "fixed_bit_set_memory_in_bytes": 42.0,
          "index_writer_max_memory_in_bytes": 42.0,
          "index_writer_memory_in_bytes": 42.0,
          "max_unsafe_auto_id_timestamp": 42.0,
          "memory_in_bytes": 42.0,
          "norms_memory_in_bytes": 42.0,
          "points_memory_in_bytes": 42.0,
          "stored_fields_memory_in_bytes": 42.0,
          "terms_memory_in_bytes": 42.0,
          "term_vectors_memory_in_bytes": 42.0,
          "version_map_memory_in_bytes": 42.0
        },
        "seq_no": {
          "global_checkpoint": 42.0,
          "local_checkpoint": 42.0,
          "max_seq_no": 42.0
        },
        "store": {
          "size_in_bytes": 42.0,
          "reserved_in_bytes": 42.0,
          "total_data_set_size_in_bytes": 42.0
        },
        "translog": {
          "earliest_last_modified_age": 42.0,
          "operations": 42.0,
          "size": "string",
          "size_in_bytes": 42.0,
          "uncommitted_operations": 42.0,
          "uncommitted_size": "string",
          "uncommitted_size_in_bytes": 42.0
        },
        "warmer": {
          "current": 42.0,
          "total": 42.0,
          "total_time": "string"
        },
        "bulk": {
          "total_operations": 42.0,
          "total_time": "string",
          "total_size_in_bytes": 42.0,
          "avg_time": "string",
          "avg_size_in_bytes": 42.0
        },
        "shards": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "shard_stats": {
          "total_count": 42.0
        },
        "additionalProperty1": {
          "primaries": {},
          "shards": {},
          "total": {},
          "uuid": "string",
          "health": "green",
          "status": "open"
        },
        "additionalProperty2": {
          "primaries": {},
          "shards": {},
          "total": {},
          "uuid": "string",
          "health": "green",
          "status": "open"
        }
      }
    },
    "additionalProperty2": {
      "adaptive_selection": {
        "additionalProperty1": {
          "avg_queue_size": 42.0,
          "avg_response_time": "string",
          "avg_response_time_ns": 42.0,
          "avg_service_time": "string",
          "avg_service_time_ns": 42.0,
          "outgoing_searches": 42.0,
          "rank": "string"
        },
        "additionalProperty2": {
          "avg_queue_size": 42.0,
          "avg_response_time": "string",
          "avg_response_time_ns": 42.0,
          "avg_service_time": "string",
          "avg_service_time_ns": 42.0,
          "outgoing_searches": 42.0,
          "rank": "string"
        }
      },
      "breakers": {
        "additionalProperty1": {
          "estimated_size": "string",
          "estimated_size_in_bytes": 42.0,
          "limit_size": "string",
          "limit_size_in_bytes": 42.0,
          "overhead": 42.0,
          "tripped": 42.0
        },
        "additionalProperty2": {
          "estimated_size": "string",
          "estimated_size_in_bytes": 42.0,
          "limit_size": "string",
          "limit_size_in_bytes": 42.0,
          "overhead": 42.0,
          "tripped": 42.0
        }
      },
      "fs": {
        "data": [
          {}
        ],
        "timestamp": 42.0,
        "total": {
          "available": "string",
          "available_in_bytes": 42.0,
          "free": "string",
          "free_in_bytes": 42.0,
          "total": "string",
          "total_in_bytes": 42.0
        },
        "io_stats": {
          "devices": [
            {}
          ],
          "total": {}
        }
      },
      "host": "string",
      "http": {
        "current_open": 42.0,
        "total_opened": 42.0,
        "clients": [
          {}
        ],
        "routes": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        }
      },
      "ingest": {
        "pipelines": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "total": {
          "count": 42.0,
          "current": 42.0,
          "failed": 42.0
        }
      },
      "ip": "string",
      "jvm": {
        "buffer_pools": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "classes": {
          "current_loaded_count": 42.0,
          "total_loaded_count": 42.0,
          "total_unloaded_count": 42.0
        },
        "gc": {
          "collectors": {}
        },
        "mem": {
          "heap_used_in_bytes": 42.0,
          "heap_used_percent": 42.0,
          "heap_committed_in_bytes": 42.0,
          "heap_max_in_bytes": 42.0,
          "non_heap_used_in_bytes": 42.0,
          "non_heap_committed_in_bytes": 42.0,
          "pools": {}
        },
        "threads": {
          "count": 42.0,
          "peak_count": 42.0
        },
        "timestamp": 42.0,
        "uptime": "string",
        "uptime_in_millis": 42.0
      },
      "name": "string",
      "os": {
        "cpu": {
          "percent": 42.0,
          "sys": "string",
          "total": "string",
          "user": "string",
          "load_average": {}
        },
        "": {},
        "swap": {
          "adjusted_total_in_bytes": 42.0,
          "resident": "string",
          "resident_in_bytes": 42.0,
          "share": "string",
          "share_in_bytes": 42.0,
          "total_virtual": "string",
          "total_virtual_in_bytes": 42.0,
          "total_in_bytes": 42.0,
          "free_in_bytes": 42.0,
          "used_in_bytes": 42.0
        },
        "cgroup": {
          "cpuacct": {},
          "cpu": {},
          "memory": {}
        },
        "timestamp": 42.0
      },
      "process": {
        "cpu": {
          "percent": 42.0,
          "sys": "string",
          "total": "string",
          "user": "string",
          "load_average": {}
        },
        "mem": {
          "adjusted_total_in_bytes": 42.0,
          "resident": "string",
          "resident_in_bytes": 42.0,
          "share": "string",
          "share_in_bytes": 42.0,
          "total_virtual": "string",
          "total_virtual_in_bytes": 42.0,
          "total_in_bytes": 42.0,
          "free_in_bytes": 42.0,
          "used_in_bytes": 42.0
        },
        "open_file_descriptors": 42.0,
        "max_file_descriptors": 42.0,
        "timestamp": 42.0
      },
      "roles": [
        "master"
      ],
      "script": {
        "cache_evictions": 42.0,
        "compilations": 42.0,
        "compilations_history": {
          "additionalProperty1": 42.0,
          "additionalProperty2": 42.0
        },
        "compilation_limit_triggered": 42.0,
        "contexts": [
          {}
        ]
      },
      "script_cache": {},
      "thread_pool": {
        "additionalProperty1": {
          "active": 42.0,
          "completed": 42.0,
          "largest": 42.0,
          "queue": 42.0,
          "rejected": 42.0,
          "threads": 42.0
        },
        "additionalProperty2": {
          "active": 42.0,
          "completed": 42.0,
          "largest": 42.0,
          "queue": 42.0,
          "rejected": 42.0,
          "threads": 42.0
        }
      },
      "timestamp": 42.0,
      "transport": {
        "inbound_handling_time_histogram": [
          {}
        ],
        "outbound_handling_time_histogram": [
          {}
        ],
        "rx_count": 42.0,
        "rx_size": "string",
        "rx_size_in_bytes": 42.0,
        "server_open": 42.0,
        "tx_count": 42.0,
        "tx_size": "string",
        "tx_size_in_bytes": 42.0,
        "total_outbound_connections": 42.0
      },
      "transport_address": "string",
      "attributes": {
        "additionalProperty1": "string",
        "additionalProperty2": "string"
      },
      "discovery": {
        "cluster_state_queue": {
          "total": 42.0,
          "pending": 42.0,
          "committed": 42.0
        },
        "published_cluster_states": {
          "full_states": 42.0,
          "incompatible_diffs": 42.0,
          "compatible_diffs": 42.0
        },
        "cluster_state_update": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "serialized_cluster_states": {
          "full_states": {},
          "diffs": {}
        },
        "cluster_applier_stats": {
          "recordings": [
            {}
          ]
        }
      },
      "indexing_pressure": {
        "memory": {
          "limit_in_bytes": 42.0,
          "current": {},
          "total": {}
        }
      },
      "indices": {
        "commit": {
          "generation": 42.0,
          "id": "string",
          "num_docs": 42.0,
          "user_data": {}
        },
        "completion": {
          "size_in_bytes": 42.0,
          "fields": {}
        },
        "docs": {
          "count": 42.0,
          "deleted": 42.0
        },
        "fielddata": {
          "evictions": 42.0,
          "memory_size_in_bytes": 42.0,
          "fields": {}
        },
        "flush": {
          "periodic": 42.0,
          "total": 42.0,
          "total_time": "string"
        },
        "get": {
          "current": 42.0,
          "exists_time": "string",
          "exists_total": 42.0,
          "missing_time": "string",
          "missing_total": 42.0,
          "time": "string",
          "total": 42.0
        },
        "indexing": {
          "index_current": 42.0,
          "delete_current": 42.0,
          "delete_time": "string",
          "delete_total": 42.0,
          "is_throttled": true,
          "noop_update_total": 42.0,
          "throttle_time": "string",
          "index_time": "string",
          "index_total": 42.0,
          "index_failed": 42.0,
          "types": {},
          "write_load": 42.0
        },
        "mappings": {
          "total_count": 42.0,
          "total_estimated_overhead_in_bytes": 42.0
        },
        "merges": {
          "current": 42.0,
          "current_docs": 42.0,
          "current_size": "string",
          "current_size_in_bytes": 42.0,
          "total": 42.0,
          "total_auto_throttle": "string",
          "total_auto_throttle_in_bytes": 42.0,
          "total_docs": 42.0,
          "total_size": "string",
          "total_size_in_bytes": 42.0,
          "total_stopped_time": "string",
          "total_throttled_time": "string",
          "total_time": "string"
        },
        "shard_path": {
          "data_path": "string",
          "is_custom_data_path": true,
          "state_path": "string"
        },
        "query_cache": {
          "cache_count": 42.0,
          "cache_size": 42.0,
          "evictions": 42.0,
          "hit_count": 42.0,
          "memory_size_in_bytes": 42.0,
          "miss_count": 42.0,
          "total_count": 42.0
        },
        "recovery": {
          "current_as_source": 42.0,
          "current_as_target": 42.0,
          "throttle_time": "string"
        },
        "refresh": {
          "external_total": 42.0,
          "listeners": 42.0,
          "total": 42.0,
          "total_time": "string"
        },
        "request_cache": {
          "evictions": 42.0,
          "hit_count": 42.0,
          "memory_size": "string",
          "memory_size_in_bytes": 42.0,
          "miss_count": 42.0
        },
        "retention_leases": {
          "primary_term": 42.0,
          "version": 42.0,
          "leases": [
            {}
          ]
        },
        "routing": {
          "node": "string",
          "primary": true,
          "state": "UNASSIGNED"
        },
        "search": {
          "fetch_current": 42.0,
          "fetch_time": "string",
          "fetch_total": 42.0,
          "open_contexts": 42.0,
          "query_current": 42.0,
          "query_time": "string",
          "query_total": 42.0,
          "scroll_current": 42.0,
          "scroll_time": "string",
          "scroll_total": 42.0,
          "suggest_current": 42.0,
          "suggest_time": "string",
          "suggest_total": 42.0,
          "groups": {}
        },
        "segments": {
          "count": 42.0,
          "doc_values_memory_in_bytes": 42.0,
          "file_sizes": {},
          "fixed_bit_set_memory_in_bytes": 42.0,
          "index_writer_max_memory_in_bytes": 42.0,
          "index_writer_memory_in_bytes": 42.0,
          "max_unsafe_auto_id_timestamp": 42.0,
          "memory_in_bytes": 42.0,
          "norms_memory_in_bytes": 42.0,
          "points_memory_in_bytes": 42.0,
          "stored_fields_memory_in_bytes": 42.0,
          "terms_memory_in_bytes": 42.0,
          "term_vectors_memory_in_bytes": 42.0,
          "version_map_memory_in_bytes": 42.0
        },
        "seq_no": {
          "global_checkpoint": 42.0,
          "local_checkpoint": 42.0,
          "max_seq_no": 42.0
        },
        "store": {
          "size_in_bytes": 42.0,
          "reserved_in_bytes": 42.0,
          "total_data_set_size_in_bytes": 42.0
        },
        "translog": {
          "earliest_last_modified_age": 42.0,
          "operations": 42.0,
          "size": "string",
          "size_in_bytes": 42.0,
          "uncommitted_operations": 42.0,
          "uncommitted_size": "string",
          "uncommitted_size_in_bytes": 42.0
        },
        "warmer": {
          "current": 42.0,
          "total": 42.0,
          "total_time": "string"
        },
        "bulk": {
          "total_operations": 42.0,
          "total_time": "string",
          "total_size_in_bytes": 42.0,
          "avg_time": "string",
          "avg_size_in_bytes": 42.0
        },
        "shards": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "shard_stats": {
          "total_count": 42.0
        },
        "additionalProperty1": {
          "primaries": {},
          "shards": {},
          "total": {},
          "uuid": "string",
          "health": "green",
          "status": "open"
        },
        "additionalProperty2": {
          "primaries": {},
          "shards": {},
          "total": {},
          "uuid": "string",
          "health": "green",
          "status": "open"
        }
      }
    }
  }
}

Set a connector sync job error Technical preview

PUT /_connector/_sync_job/{connector_sync_job_id}/_error
Api key auth Basic auth Bearer auth

Set the error field for a connector sync job and set its status to error.

To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.

Path parameters

application/json

Body Required

  • error string Required

    The error for the connector sync job error field.

Responses

PUT /_connector/_sync_job/{connector_sync_job_id}/_error 
curl \
 --request PUT 'http://api.example.com/_connector/_sync_job/{connector_sync_job_id}/_error' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"error\": \"some-error\"\n}"'
Request example 
{
    "error": "some-error"
}
Response examples (200) 
{}

Delete auto-follow patterns Added in 6.5.0

DELETE /_ccr/auto_follow/{name}
Api key auth Basic auth Bearer auth

Delete a collection of cross-cluster replication auto-follow patterns.

External documentation

Path parameters

  • name string Required

    The auto-follow pattern collection to delete.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
DELETE /_ccr/auto_follow/{name} 
curl \
 --request DELETE 'http://api.example.com/_ccr/auto_follow/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `DELETE /_ccr/auto_follow/my_auto_follow_pattern`, which deletes an auto-follow pattern. 
{
  "acknowledged" : true
}

Data stream

Learn more about data streams

Get data stream stats Added in 7.9.0

GET /_data_stream/{name}/_stats
Api key auth Basic auth Bearer auth

Get statistics for one or more data streams.

Path parameters

  • name string Required

    Comma-separated list of data streams used to limit the request. Wildcard expressions (*) are supported. To target all data streams in a cluster, omit this parameter or use *.

Query parameters

  • expand_wildcards string | array[string]

    Type of data stream that wildcard patterns can match. Supports comma-separated values, such as open,hidden.

Responses

GET /_data_stream/{name}/_stats 
curl \
 --request GET 'http://api.example.com/_data_stream/{name}/_stats' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for retrieving statistics for a data stream. 
{
  "_shards": {
    "total": 10,
    "successful": 5,
    "failed": 0
  },
  "data_stream_count": 2,
  "backing_indices": 5,
  "total_store_size": "7kb",
  "total_store_size_bytes": 7268,
  "data_streams": [
    {
      "data_stream": "my-data-stream",
      "backing_indices": 3,
      "store_size": "3.7kb",
      "store_size_bytes": 3772,
      "maximum_timestamp": 1607512028000
    },
    {
      "data_stream": "my-data-stream-two",
      "backing_indices": 2,
      "store_size": "3.4kb",
      "store_size_bytes": 3496,
      "maximum_timestamp": 1607425567000
    }
  ]
}

Bulk index or delete documents

PUT /_bulk
Api key auth Basic auth Bearer auth

Perform multiple index, create, delete, and update actions in a single request. This reduces overhead and can greatly increase indexing speed.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:

  • To use the create action, you must have the create_doc, create, index, or write index privilege. Data streams support only the create action.
  • To use the index action, you must have the create, index, or write index privilege.
  • To use the delete action, you must have the delete or write index privilege.
  • To use the update action, you must have the index or write index privilege.
  • To automatically create a data stream or index with a bulk API request, you must have the auto_configure, create_index, or manage index privilege.
  • To make the result of a bulk operation visible to search using the refresh parameter, you must have the maintenance or manage index privilege.

Automatic data stream creation requires a matching index template with data stream enabled.

The actions are specified in the request body using a newline delimited JSON (NDJSON) structure:

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n

The index and create actions expect a source on the next line and have the same semantics as the op_type parameter in the standard index API. A create action fails if a document with the same ID already exists in the target An index action adds or replaces a document as necessary.

NOTE: Data streams support only the create action. To update or delete a document in a data stream, you must target the backing index containing the document.

An update action expects that the partial doc, upsert, and script and its options are specified on the next line.

A delete action does not expect a source on the next line and has the same semantics as the standard delete API.

NOTE: The final line of data must end with a newline character (\n). Each newline character may be preceded by a carriage return (\r). When sending NDJSON data to the _bulk endpoint, use a Content-Type header of application/json or application/x-ndjson. Because this format uses literal newline characters (\n) as delimiters, make sure that the JSON actions and sources are not pretty printed.

If you provide a target in the request path, it is used for any actions that don't explicitly specify an _index argument.

A note on the format: the idea here is to make processing as fast as possible. As some of the actions are redirected to other shards on other nodes, only action_meta_data is parsed on the receiving node side.

Client libraries using this protocol should try and strive to do something similar on the client side, and reduce buffering as much as possible.

There is no "correct" number of actions to perform in a single bulk request. Experiment with different settings to find the optimal size for your particular workload. Note that Elasticsearch limits the maximum size of a HTTP request to 100mb by default so clients must ensure that no request exceeds this size. It is not possible to index a single document that exceeds the size limit, so you must pre-process any such documents into smaller pieces before sending them to Elasticsearch. For instance, split documents into pages or chapters before indexing them, or store raw binary data in a system outside Elasticsearch and replace the raw data with a link to the external system in the documents that you send to Elasticsearch.

Client suppport for bulk requests

Some of the officially supported clients provide helpers to assist with bulk requests and reindexing:

  • Go: Check out esutil.BulkIndexer
  • Perl: Check out Search::Elasticsearch::Client::5_0::Bulk and Search::Elasticsearch::Client::5_0::Scroll
  • Python: Check out elasticsearch.helpers.*
  • JavaScript: Check out client.helpers.*
  • .NET: Check out BulkAllObservable
  • PHP: Check out bulk indexing.

Submitting bulk requests with cURL

If you're providing text file input to curl, you must use the --data-binary flag instead of plain -d. The latter doesn't preserve newlines. For example:

$ cat requests
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}

Optimistic concurrency control

Each index and delete action within a bulk API call may include the if_seq_no and if_primary_term parameters in their respective action and meta data lines. The if_seq_no and if_primary_term parameters control how operations are run, based on the last modification to existing documents. See Optimistic concurrency control for more details.

Versioning

Each bulk item can include the version value using the version field. It automatically follows the behavior of the index or delete operation based on the _version mapping. It also support the version_type.

Routing

Each bulk item can include the routing value using the routing field. It automatically follows the behavior of the index or delete operation based on the _routing mapping.

NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.

Wait for active shards

When making bulk calls, you can set the wait_for_active_shards parameter to require a minimum number of shard copies to be active before starting to process the bulk request.

Refresh

Control when the changes made by this request are visible to search.

NOTE: Only the shards that receive the bulk request will be affected by refresh. Imagine a _bulk?refresh=wait_for request with three documents in it that happen to be routed to different shards in an index with five shards. The request will only wait for those three shards to refresh. The other two shards that make up the index do not participate in the _bulk request at all.

Query parameters

  • include_source_on_error boolean

    True or false if to include the document source in the error message in case of parsing errors.

  • list_executed_pipelines boolean

    If true, the response will include the ingest pipelines that were run for each index or create.

  • pipeline string

    The pipeline identifier to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.

  • refresh string

    If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, wait for a refresh to make this operation visible to search. If false, do nothing with refreshes. Valid values: true, false, wait_for.

    Values are true, false, or wait_for.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • _source boolean | string | array[string]

    Indicates whether to return the _source field (true or false) or contains a list of fields to return.

  • _source_excludes string | array[string]

    A comma-separated list of source fields to exclude from the response. You can also use this parameter to exclude fields from the subset specified in _source_includes query parameter. If the _source parameter is false, this parameter is ignored.

  • _source_includes string | array[string]

    A comma-separated list of source fields to include in the response. If this parameter is specified, only these source fields are returned. You can exclude fields from this subset using the _source_excludes query parameter. If the _source parameter is false, this parameter is ignored.

  • timeout string

    The period each action waits for the following operations: automatic index creation, dynamic mapping updates, and waiting for active shards. The default is 1m (one minute), which guarantees Elasticsearch waits for at least the timeout before failing. The actual wait time could be longer, particularly when multiple waits occur.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default is 1, which waits for each primary shard to be active.

  • require_alias boolean

    If true, the request's actions must target an index alias.

  • require_data_stream boolean

    If true, the request's actions must target a data stream (existing or to be created).

application/json

Body object Required

One of:
OperationContainer object UpdateAction object object-3 object
  • index object
    Hide index attributes Show index attributes object
    • _id string
    • _index string
    • routing string
    • if_primary_term number
    • if_seq_no number
    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

    • dynamic_templates object

      A map from the full name of fields to the name of dynamic templates. It defaults to an empty map. If a name matches a dynamic template, that template will be applied regardless of other match predicates defined in the template. If a field is already defined in the mapping, then this parameter won't be used.

      Hide dynamic_templates attribute Show dynamic_templates attribute object
      • * string Additional properties
    • pipeline string

      The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.

    • require_alias boolean

      If true, the request's actions must target an index alias.

  • create object
    Hide create attributes Show create attributes object
    • _id string
    • _index string
    • routing string
    • if_primary_term number
    • if_seq_no number
    • version number
    • version_type string

      Values are internal, external, external_gte, or force.

    • dynamic_templates object

      A map from the full name of fields to the name of dynamic templates. It defaults to an empty map. If a name matches a dynamic template, that template will be applied regardless of other match predicates defined in the template. If a field is already defined in the mapping, then this parameter won't be used.

      Hide dynamic_templates attribute Show dynamic_templates attribute object
      • * string Additional properties
    • pipeline string

      The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.

    • require_alias boolean

      If true, the request's actions must target an index alias.

  • update object
    Hide update attributes Show update attributes object
  • delete object
    Hide delete attributes Show delete attributes object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • errors boolean Required

      If true, one or more of the operations in the bulk request did not complete successfully.

    • items array[object] Required

      The result of each operation in the bulk request, in the order they were submitted.

      Hide items attribute Show items attribute object
    • took number Required

      The length of time, in milliseconds, it took to process the bulk request.

    • ingest_took number
PUT /_bulk 
curl \
 --request PUT 'http://api.example.com/_bulk' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{ \"index\" : { \"_index\" : \"test\", \"_id\" : \"1\" } }\n{ \"field1\" : \"value1\" }\n{ \"delete\" : { \"_index\" : \"test\", \"_id\" : \"2\" } }\n{ \"create\" : { \"_index\" : \"test\", \"_id\" : \"3\" } }\n{ \"field1\" : \"value3\" }\n{ \"update\" : {\"_id\" : \"1\", \"_index\" : \"test\"} }\n{ \"doc\" : {\"field2\" : \"value2\"} }"'
Request examples
Run `POST _bulk` to perform multiple operations. 
{ "index" : { "_index" : "test", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_id" : "2" } }
{ "create" : { "_index" : "test", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }
When you run `POST _bulk` and use the `update` action, you can use `retry_on_conflict` as a field in the action itself (not in the extra payload line) to specify how many times an update should be retried in the case of a version conflict. 
{ "update" : {"_id" : "1", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_index" : "index1", "retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_index" : "index1", "retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}
To return only information about failed operations, run `POST /_bulk?filter_path=items.*.error`. 
{ "update": {"_id": "5", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "update": {"_id": "6", "_index": "index1"} }
{ "doc": {"my_field": "foo"} }
{ "create": {"_id": "7", "_index": "index1"} }
{ "my_field": "foo" }
Run `POST /_bulk` to perform a bulk request that consists of index and create actions with the `dynamic_templates` parameter. The bulk request creates two new fields `work_location` and `home_location` with type `geo_point` according to the `dynamic_templates` parameter. However, the `raw_location` field is created using default dynamic mapping rules, as a text field in that case since it is supplied as a string in the JSON document. 
{ "index" : { "_index" : "my_index", "_id" : "1", "dynamic_templates": {"work_location": "geo_point"}} }
{ "field" : "value1", "work_location": "41.12,-71.34", "raw_location": "41.12,-71.34"}
{ "create" : { "_index" : "my_index", "_id" : "2", "dynamic_templates": {"home_location": "geo_point"}} }
{ "field" : "value2", "home_location": "41.12,-71.34"}
Response examples (200) 
{
   "took": 30,
   "errors": false,
   "items": [
      {
         "index": {
            "_index": "test",
            "_id": "1",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 201,
            "_seq_no" : 0,
            "_primary_term": 1
         }
      },
      {
         "delete": {
            "_index": "test",
            "_id": "2",
            "_version": 1,
            "result": "not_found",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 404,
            "_seq_no" : 1,
            "_primary_term" : 2
         }
      },
      {
         "create": {
            "_index": "test",
            "_id": "3",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 201,
            "_seq_no" : 2,
            "_primary_term" : 3
         }
      },
      {
         "update": {
            "_index": "test",
            "_id": "1",
            "_version": 2,
            "result": "updated",
            "_shards": {
                "total": 2,
                "successful": 1,
                "failed": 0
            },
            "status": 200,
            "_seq_no" : 3,
            "_primary_term" : 4
         }
      }
   ]
}
If you run `POST /_bulk` with operations that update non-existent documents, the operations cannot complete successfully. The API returns a response with an `errors` property value `true`. The response also includes an error object for any failed operations. The error object contains additional information about the failure, such as the error type and reason. 
{
  "took": 486,
  "errors": true,
  "items": [
    {
      "update": {
        "_index": "index1",
        "_id": "5",
        "status": 404,
        "error": {
          "type": "document_missing_exception",
          "reason": "[5]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "update": {
        "_index": "index1",
        "_id": "6",
        "status": 404,
        "error": {
          "type": "document_missing_exception",
          "reason": "[6]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "create": {
        "_index": "index1",
        "_id": "7",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 2,
          "successful": 1,
          "failed": 0
        },
        "_seq_no": 0,
        "_primary_term": 1,
        "status": 201
      }
    }
  ]
}
An example response from `POST /_bulk?filter_path=items.*.error`, which returns only information about failed operations. 
{
  "items": [
    {
      "update": {
        "error": {
          "type": "document_missing_exception",
          "reason": "[5]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    },
    {
      "update": {
        "error": {
          "type": "document_missing_exception",
          "reason": "[6]: document missing",
          "index_uuid": "aAsFqTI0Tc2W0LCWgPNrOA",
          "shard": "0",
          "index": "index1"
        }
      }
    }
  ]
}

The purpose of the fleet search api is to provide a search api where the search will only be executed Technical preview

GET /{index}/_fleet/_fleet_search
Api key auth Basic auth Bearer auth

after provided checkpoint has been processed and is visible for searches inside of Elasticsearch.

Path parameters

  • index string Required

    A single target to search. If the target is an index alias, it must resolve to a single index.

Query parameters

application/json

Body

  • aggregations object
  • collapse object
    External documentation
  • explain boolean

    If true, returns detailed information about score computation as part of a hit.

  • ext object

    Configuration of search extensions defined by Elasticsearch plugins.

    Hide ext attribute Show ext attribute object
    • * object Additional properties
  • from number

    Starting document offset. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • highlight object
    Hide highlight attributes Show highlight attributes object

    • type string

      Any of:
      HighlighterType string HighlighterType string

      Values are plain, fvh, or unified.

    • boundary_chars string

      A string that contains each boundary character.

    • boundary_max_scan number

      How far to scan for boundary characters.

    • boundary_scanner string

      Values are chars, sentence, or word.

    • boundary_scanner_locale string

      Controls which locale is used to search for sentence and word boundaries. This parameter takes a form of a language tag, for example: "en-US", "fr-FR", "ja-JP".

    • force_source boolean Deprecated
    • fragmenter string

      Values are simple or span.

    • fragment_size number

      The size of the highlighted fragment in characters.

    • highlight_filter boolean
    • highlight_query object

      An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

      External documentation
    • max_fragment_length number
    • max_analyzed_offset number

      If set to a non-negative value, highlighting stops at this defined maximum limit. The rest of the text is not processed, thus not highlighted and no error is returned The max_analyzed_offset query setting does not override the index.highlight.max_analyzed_offset setting, which prevails when it’s set to lower value than the query setting.

    • no_match_size number

      The amount of text you want to return from the beginning of the field if there are no matching fragments to highlight.

    • number_of_fragments number

      The maximum number of fragments to return. If the number of fragments is set to 0, no fragments are returned. Instead, the entire field contents are highlighted and returned. This can be handy when you need to highlight short texts such as a title or address, but fragmentation is not required. If number_of_fragments is 0, fragment_size is ignored.

    • options object
      Hide options attribute Show options attribute object
      • * object Additional properties
    • order string

      Value is score.

    • phrase_limit number

      Controls the number of matching phrases in a document that are considered. Prevents the fvh highlighter from analyzing too many phrases and consuming too much memory. When using matched_fields, phrase_limit phrases per matched field are considered. Raising the limit increases query time and consumes more memory. Only supported by the fvh highlighter.

    • post_tags array[string]

      Use in conjunction with pre_tags to define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in <em> and </em> tags.

    • pre_tags array[string]

      Use in conjunction with post_tags to define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in <em> and </em> tags.

    • require_field_match boolean

      By default, only fields that contains a query match are highlighted. Set to false to highlight all fields.

    • tags_schema string

      Value is styled.

    • encoder string

      Values are default or html.

    • fields object Required
  • track_total_hits boolean | number

    Number of hits matching the query to count accurately. If true, the exact number of hits is returned at the cost of some performance. If false, the response does not include the total number of hits matching the query. Defaults to 10,000 hits.

  • indices_boost array[object]

    Boosts the _score of documents from specified indices.

    Hide indices_boost attribute Show indices_boost attribute object
    • * number Additional properties
  • docvalue_fields array[object]

    Array of wildcard (*) patterns. The request returns doc values for field names matching these patterns in the hits.fields property of the response.

    Hide docvalue_fields attributes Show docvalue_fields attributes object
    • field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • format string

      The format in which the values are returned.

    • include_unmapped boolean
  • min_score number

    Minimum _score for matching documents. Documents with a lower _score are not included in the search results.

  • post_filter object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • profile boolean
  • query object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation

  • rescore object | array[object]

    One of:
    Rescore object array-2 array[object]
    Hide attributes Show attributes
    • window_size number
    • query object
      Hide query attributes Show query attributes object
    • learning_to_rank object
      Hide learning_to_rank attributes Show learning_to_rank attributes object
      • model_id string Required

        The unique identifier of the trained model uploaded to Elasticsearch

      • params object

        Named parameters to be passed to the query templates used for feature

        Hide params attribute Show params attribute object
        • * object Additional properties
  • script_fields object

    Retrieve a script evaluation (based on different fields) for each hit.

    Hide script_fields attribute Show script_fields attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • script object Required
        Hide script attributes Show script attributes object
        • source string

          The script source.

        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          Hide params attribute Show params attribute object
          • * object Additional properties

        • lang string

          Any of:
          ScriptLanguage string ScriptLanguage string

          Values are painless, expression, mustache, or java.

        • options object
          Hide options attribute Show options attribute object
          • * string Additional properties
      • ignore_failure boolean
  • search_after array[number | string | boolean | null | object]

    A field value.

    A field value.

    One of:
    FieldValue number FieldValue number FieldValue string FieldValue boolean FieldValue string | null FieldValue object
  • size number

    The number of hits to return. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • slice object
    Hide slice attributes Show slice attributes object
    • field string

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • id string Required
    • max number Required

  • sort string | object | array[string | object]

    One of:
    Field string SortOptions object Sort array[string | object]

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • _source boolean | object

    Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.

    One of:
    SourceConfig boolean SourceFilter object
  • fields array[object]

    Array of wildcard (*) patterns. The request returns values for field names matching these patterns in the hits.fields property of the response.

    Hide fields attributes Show fields attributes object
    • field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • format string

      The format in which the values are returned.

    • include_unmapped boolean
  • suggest object
    Hide suggest attribute Show suggest attribute object
    • text string

      Global suggest text, to avoid repetition when the same text is used in several suggesters

  • terminate_after number

    Maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting. Defaults to 0, which does not terminate query execution early.

  • timeout string

    Specifies the period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.

  • track_scores boolean

    If true, calculate and return document scores, even if the scores are not used for sorting.

  • version boolean

    If true, returns document version as part of a hit.

  • seq_no_primary_term boolean

    If true, returns sequence number and primary term of the last modification of each hit. See Optimistic concurrency control.

  • stored_fields string | array[string]
  • pit object
    Hide pit attributes Show pit attributes object
    • id string Required
    • keep_alive string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • runtime_mappings object
    Hide runtime_mappings attribute Show runtime_mappings attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • fields object

        For type composite

        Hide fields attribute Show fields attribute object
        • * object Additional properties
          Hide * attribute Show * attribute object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • fetch_fields array[object]

        For type lookup

        Hide fetch_fields attributes Show fetch_fields attributes object
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • format string
      • format string

        A custom format for date type runtime fields.

      • input_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_index string
      • script object
        Hide script attributes Show script attributes object
        • source string

          The script source.

        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          Hide params attribute Show params attribute object
          • * object Additional properties

        • lang string

          Any of:
          ScriptLanguage string ScriptLanguage string

          Values are painless, expression, mustache, or java.

        • options object
          Hide options attribute Show options attribute object
          • * string Additional properties
      • type string Required

        Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

  • stats array[string]

    Stats groups to associate with the search. Each group maintains a statistics aggregation for its associated searches. You can retrieve these stats using the indices stats API.

Responses

GET /{index}/_fleet/_fleet_search 
curl \
 --request GET 'http://api.example.com/{index}/_fleet/_fleet_search' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"aggregations":{},"collapse":{},"explain":true,"ext":{"additionalProperty1":{},"additionalProperty2":{}},"from":42.0,"highlight":{"":"plain","boundary_chars":"string","boundary_max_scan":42.0,"boundary_scanner":"chars","boundary_scanner_locale":"string","force_source":true,"fragmenter":"simple","fragment_size":42.0,"highlight_filter":true,"highlight_query":{},"max_fragment_length":42.0,"max_analyzed_offset":42.0,"no_match_size":42.0,"number_of_fragments":42.0,"options":{"additionalProperty1":{},"additionalProperty2":{}},"order":"score","phrase_limit":42.0,"post_tags":["string"],"pre_tags":["string"],"require_field_match":true,"tags_schema":"styled","encoder":"default","fields":{}},"track_total_hits":true,"indices_boost":[{"additionalProperty1":42.0,"additionalProperty2":42.0}],"docvalue_fields":[{"field":"string","format":"string","include_unmapped":true}],"min_score":42.0,"post_filter":{},"profile":true,"query":{},"rescore":{"window_size":42.0,"query":{"rescore_query":{},"query_weight":42.0,"rescore_query_weight":42.0,"score_mode":"avg"},"learning_to_rank":{"model_id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}}}},"script_fields":{"additionalProperty1":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"search_after":[42.0],"size":42.0,"slice":{"field":"string","id":"string","max":42.0},"":true,"fields":[{"field":"string","format":"string","include_unmapped":true}],"suggest":{"text":"string"},"terminate_after":42.0,"timeout":"string","track_scores":true,"version":true,"seq_no_primary_term":true,"stored_fields":"string","pit":{"id":"string","keep_alive":"string"},"runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"stats":["string"]}'
Request examples 
{
  "aggregations": {},
  "collapse": {},
  "explain": true,
  "ext": {
    "additionalProperty1": {},
    "additionalProperty2": {}
  },
  "from": 42.0,
  "highlight": {
    "": "plain",
    "boundary_chars": "string",
    "boundary_max_scan": 42.0,
    "boundary_scanner": "chars",
    "boundary_scanner_locale": "string",
    "force_source": true,
    "fragmenter": "simple",
    "fragment_size": 42.0,
    "highlight_filter": true,
    "highlight_query": {},
    "max_fragment_length": 42.0,
    "max_analyzed_offset": 42.0,
    "no_match_size": 42.0,
    "number_of_fragments": 42.0,
    "options": {
      "additionalProperty1": {},
      "additionalProperty2": {}
    },
    "order": "score",
    "phrase_limit": 42.0,
    "post_tags": [
      "string"
    ],
    "pre_tags": [
      "string"
    ],
    "require_field_match": true,
    "tags_schema": "styled",
    "encoder": "default",
    "fields": {}
  },
  "track_total_hits": true,
  "indices_boost": [
    {
      "additionalProperty1": 42.0,
      "additionalProperty2": 42.0
    }
  ],
  "docvalue_fields": [
    {
      "field": "string",
      "format": "string",
      "include_unmapped": true
    }
  ],
  "min_score": 42.0,
  "post_filter": {},
  "profile": true,
  "query": {},
  "rescore": {
    "window_size": 42.0,
    "query": {
      "rescore_query": {},
      "query_weight": 42.0,
      "rescore_query_weight": 42.0,
      "score_mode": "avg"
    },
    "learning_to_rank": {
      "model_id": "string",
      "params": {
        "additionalProperty1": {},
        "additionalProperty2": {}
      }
    }
  },
  "script_fields": {
    "additionalProperty1": {
      "script": {
        "source": "string",
        "id": "string",
        "params": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "": "painless",
        "options": {
          "additionalProperty1": "string",
          "additionalProperty2": "string"
        }
      },
      "ignore_failure": true
    },
    "additionalProperty2": {
      "script": {
        "source": "string",
        "id": "string",
        "params": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "": "painless",
        "options": {
          "additionalProperty1": "string",
          "additionalProperty2": "string"
        }
      },
      "ignore_failure": true
    }
  },
  "search_after": [
    42.0
  ],
  "size": 42.0,
  "slice": {
    "field": "string",
    "id": "string",
    "max": 42.0
  },
  "": true,
  "fields": [
    {
      "field": "string",
      "format": "string",
      "include_unmapped": true
    }
  ],
  "suggest": {
    "text": "string"
  },
  "terminate_after": 42.0,
  "timeout": "string",
  "track_scores": true,
  "version": true,
  "seq_no_primary_term": true,
  "stored_fields": "string",
  "pit": {
    "id": "string",
    "keep_alive": "string"
  },
  "runtime_mappings": {
    "additionalProperty1": {
      "fields": {
        "additionalProperty1": {
          "type": "boolean"
        },
        "additionalProperty2": {
          "type": "boolean"
        }
      },
      "fetch_fields": [
        {
          "field": "string",
          "format": "string"
        }
      ],
      "format": "string",
      "input_field": "string",
      "target_field": "string",
      "target_index": "string",
      "script": {
        "source": "string",
        "id": "string",
        "params": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "": "painless",
        "options": {
          "additionalProperty1": "string",
          "additionalProperty2": "string"
        }
      },
      "type": "boolean"
    },
    "additionalProperty2": {
      "fields": {
        "additionalProperty1": {
          "type": "boolean"
        },
        "additionalProperty2": {
          "type": "boolean"
        }
      },
      "fetch_fields": [
        {
          "field": "string",
          "format": "string"
        }
      ],
      "format": "string",
      "input_field": "string",
      "target_field": "string",
      "target_index": "string",
      "script": {
        "source": "string",
        "id": "string",
        "params": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "": "painless",
        "options": {
          "additionalProperty1": "string",
          "additionalProperty2": "string"
        }
      },
      "type": "boolean"
    }
  },
  "stats": [
    "string"
  ]
}
Response examples (200) 
{
  "took": 42.0,
  "timed_out": true,
  "_shards": {
    "failed": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "failures": [
      {
        "index": "string",
        "node": "string",
        "reason": {
          "type": "string",
          "reason": "string",
          "stack_trace": "string",
          "caused_by": {},
          "root_cause": [
            {}
          ],
          "suppressed": [
            {}
          ]
        },
        "shard": 42.0,
        "status": "string"
      }
    ],
    "skipped": 42.0
  },
  "hits": {
    "total": {
      "relation": "eq",
      "value": 42.0
    },
    "hits": [
      {
        "_index": "string",
        "_id": "string",
        "_score": 42.0,
        "_explanation": {
          "description": "string",
          "details": [
            {}
          ],
          "value": 42.0
        },
        "fields": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "highlight": {
          "additionalProperty1": [
            "string"
          ],
          "additionalProperty2": [
            "string"
          ]
        },
        "inner_hits": {
          "additionalProperty1": {
            "hits": {}
          },
          "additionalProperty2": {
            "hits": {}
          }
        },
        "matched_queries": [
          "string"
        ],
        "_nested": {
          "field": "string",
          "offset": 42.0,
          "_nested": {}
        },
        "_ignored": [
          "string"
        ],
        "ignored_field_values": {
          "additionalProperty1": [
            {}
          ],
          "additionalProperty2": [
            {}
          ]
        },
        "_shard": "string",
        "_node": "string",
        "_routing": "string",
        "_source": {},
        "_rank": 42.0,
        "_seq_no": 42.0,
        "_primary_term": 42.0,
        "_version": 42.0,
        "sort": [
          42.0
        ]
      }
    ],
    "max_score": 42.0
  },
  "aggregations": {},
  "_clusters": {
    "skipped": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "running": 42.0,
    "partial": 42.0,
    "failed": 42.0,
    "details": {
      "additionalProperty1": {
        "status": "running",
        "indices": "string",
        "": 42.0,
        "timed_out": true,
        "_shards": {
          "failed": 42.0,
          "successful": 42.0,
          "total": 42.0,
          "failures": [
            {}
          ],
          "skipped": 42.0
        },
        "failures": [
          {
            "index": "string",
            "node": "string",
            "reason": {},
            "shard": 42.0,
            "status": "string"
          }
        ]
      },
      "additionalProperty2": {
        "status": "running",
        "indices": "string",
        "": 42.0,
        "timed_out": true,
        "_shards": {
          "failed": 42.0,
          "successful": 42.0,
          "total": 42.0,
          "failures": [
            {}
          ],
          "skipped": 42.0
        },
        "failures": [
          {
            "index": "string",
            "node": "string",
            "reason": {},
            "shard": 42.0,
            "status": "string"
          }
        ]
      }
    }
  },
  "fields": {
    "additionalProperty1": {},
    "additionalProperty2": {}
  },
  "max_score": 42.0,
  "num_reduce_phases": 42.0,
  "profile": {
    "shards": [
      {
        "aggregations": [
          {
            "breakdown": {},
            "description": "string",
            "type": "string",
            "debug": {},
            "children": [
              {}
            ]
          }
        ],
        "cluster": "string",
        "dfs": {
          "statistics": {
            "type": "string",
            "description": "string",
            "time": "string",
            "breakdown": {},
            "debug": {},
            "children": [
              {}
            ]
          },
          "knn": [
            {}
          ]
        },
        "fetch": {
          "type": "string",
          "description": "string",
          "": 42.0,
          "breakdown": {
            "load_source": 42.0,
            "load_source_count": 42.0,
            "load_stored_fields": 42.0,
            "load_stored_fields_count": 42.0,
            "next_reader": 42.0,
            "next_reader_count": 42.0,
            "process_count": 42.0,
            "process": 42.0
          },
          "debug": {
            "stored_fields": [
              "string"
            ],
            "fast_path": 42.0
          },
          "children": [
            {}
          ]
        },
        "id": "string",
        "index": "string",
        "node_id": "string",
        "searches": [
          {
            "collector": [
              {}
            ],
            "query": [
              {}
            ],
            "rewrite_time": 42.0
          }
        ],
        "shard_id": 42.0
      }
    ]
  },
  "pit_id": "string",
  "_scroll_id": "string",
  "suggest": {
    "additionalProperty1": [
      {
        "length": 42.0,
        "offset": 42.0,
        "text": "string"
      }
    ],
    "additionalProperty2": [
      {
        "length": 42.0,
        "offset": 42.0,
        "text": "string"
      }
    ]
  },
  "terminated_early": true
}

Create or update an index template Added in 7.9.0

PUT /_index_template/{name}
Api key auth Basic auth Bearer auth

Index templates define settings, mappings, and aliases that can be applied automatically to new indices.

Elasticsearch applies templates to new indices based on an wildcard pattern that matches the index name. Index templates are applied during data stream or index creation. For data streams, these settings and mappings are applied when the stream's backing indices are created. Settings and mappings specified in a create index API request override any settings or mappings specified in an index template. Changes to index templates do not affect existing indices, including the existing backing indices of a data stream.

You can use C-style /* *\/ block comments in index templates. You can include comments anywhere in the request body, except before the opening curly bracket.

Multiple matching templates

If multiple index templates match the name of a new index or data stream, the template with the highest priority is used.

Multiple templates with overlapping index patterns at the same priority are not allowed and an error will be thrown when attempting to create a template matching an existing index template at identical priorities.

Composing aliases, mappings, and settings

When multiple component templates are specified in the composed_of field for an index template, they are merged in the order specified, meaning that later component templates override earlier component templates. Any mappings, settings, or aliases from the parent index template are merged in next. Finally, any configuration on the index request itself is merged. Mapping definitions are merged recursively, which means that later mapping components can introduce new field mappings and update the mapping configuration. If a field mapping is already contained in an earlier component, its definition will be completely overwritten by the later one. This recursive merging strategy applies not only to field mappings, but also root options like dynamic_templates and meta. If an earlier component contains a dynamic_templates block, then by default new dynamic_templates entries are appended onto the end. If an entry already exists with the same key, then it is overwritten by the new definition.

Path parameters

  • name string Required

    Index or template name

Query parameters

  • create boolean

    If true, this request cannot replace or update existing index templates.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • cause string

    User defined reason for creating/updating the index template

application/json

Body Required

  • index_patterns string | array[string]
  • composed_of array[string]

    An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.

  • template object
    Hide template attributes Show template attributes object
    • aliases object

      Aliases to add. If the index template includes a data_stream object, these are data stream aliases. Otherwise, these are index aliases. Data stream aliases ignore the index_routing, routing, and search_routing options.

      Hide aliases attribute Show aliases attribute object
    • mappings object
      Hide mappings attributes Show mappings attributes object
    • settings object
      Hide settings attributes Show settings attributes object
      • index object
      • mode string

      • routing_path string | array[string]

        One of:
        string-1 string array-2 array[string]
      • soft_deletes object
        Hide soft_deletes attributes Show soft_deletes attributes object
        • enabled boolean

          Indicates whether soft deletes are enabled on the index.

        • retention_lease object
          Hide retention_lease attribute Show retention_lease attribute object
          • period string Required

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • sort object
        Hide sort attributes Show sort attributes object

      • number_of_shards number | string

        One of:
        number-1 number string-2 string

      • number_of_replicas number | string

        One of:
        number-1 number string-2 string
      • number_of_routing_shards number
      • check_on_startup string

        Values are true, false, or checksum.

      • codec string

      • routing_partition_size number | string

        Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

        Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

        One of:
        Stringifiedinteger number Stringifiedinteger string
      • load_fixed_bitset_filters_eagerly boolean

      • hidden boolean | string

        One of:
        boolean-1 boolean string-2 string

      • auto_expand_replicas string | null

        One of:
        string-1 string NullValue string | null
      • merge object
        Hide merge attribute Show merge attribute object
        • scheduler object
          Hide scheduler attributes Show scheduler attributes object

          • max_thread_count number | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedinteger number Stringifiedinteger string

          • max_merge_count number | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedinteger number Stringifiedinteger string
      • search object
        Hide search attributes Show search attributes object
        • idle object
          Hide idle attribute Show idle attribute object
          • after string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • slowlog object
          Hide slowlog attributes Show slowlog attributes object
          • level string
          • source number
          • reformat boolean
          • threshold object
            Hide threshold attributes Show threshold attributes object
            • query object
              Hide query attributes Show query attributes object
              • warn string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

              • info string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

              • debug string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

              • trace string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • fetch object
              Hide fetch attributes Show fetch attributes object
              • warn string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

              • info string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

              • debug string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

              • trace string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • refresh_interval string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • max_result_window number
      • max_inner_result_window number
      • max_rescore_window number
      • max_docvalue_fields_search number
      • max_script_fields number
      • max_ngram_diff number
      • max_shingle_diff number
      • blocks object
        Hide blocks attributes Show blocks attributes object

        • read_only boolean | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          Stringifiedboolean boolean Stringifiedboolean string

        • read_only_allow_delete boolean | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          Stringifiedboolean boolean Stringifiedboolean string

        • read boolean | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          Stringifiedboolean boolean Stringifiedboolean string

        • write boolean | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          Stringifiedboolean boolean Stringifiedboolean string

        • metadata boolean | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          Stringifiedboolean boolean Stringifiedboolean string
      • max_refresh_listeners number
      • analyze object
        Hide analyze attribute Show analyze attribute object

        • max_token_count number | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          Stringifiedinteger number Stringifiedinteger string
      • highlight object
        Hide highlight attribute Show highlight attribute object
      • max_terms_count number
      • max_regex_length number
      • routing object
        Hide routing attributes Show routing attributes object
      • gc_deletes string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • default_pipeline string
      • final_pipeline string
      • lifecycle object
        Hide lifecycle attributes Show lifecycle attributes object
        • name string

        • indexing_complete boolean | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          Stringifiedboolean boolean Stringifiedboolean string
        • origination_date number

          If specified, this is the timestamp used to calculate the index age for its phase transitions. Use this setting if you create a new index that contains old data and want to use the original creation date to calculate the index age. Specified as a Unix epoch value in milliseconds.

        • parse_origination_date boolean

          Set to true to parse the origination date from the index name. This origination date is used to calculate the index age for its phase transitions. The index name must match the pattern .*-{date_format}-\d+, where the date_format is yyyy.MM.dd and the trailing digits are optional. An index that was rolled over would normally match the full format, for example logs-2016.10.31-000002). If the index name doesn’t match the pattern, index creation fails.

        • step object
          Hide step attribute Show step attribute object
          • wait_time_threshold string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • rollover_alias string

          The index alias to update when the index rolls over. Specify when using a policy that contains a rollover action. When the index rolls over, the alias is updated to reflect that the index is no longer the write index. For more information about rolling indices, see Rollover.

        • prefer_ilm boolean | string

          Preference for the system that manages a data stream backing index (preferring ILM when both ILM and DLM are applicable for an index).

          One of:
          boolean-1 boolean string-2 string
      • provided_name string

      • creation_date number | string

        Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

        Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

        One of:
        UnitMillis number StringifiedEpochTimeUnitMillis string

        Time unit for milliseconds

      • creation_date_string string | number

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
        DateTime string UnitMillis number
      • uuid string
      • version object
        Hide version attributes Show version attributes object

      • verified_before_close boolean | string

        One of:
        boolean-1 boolean string-2 string

      • format string | number

        One of:
        string-1 string number-2 number
      • max_slices_per_scroll number
      • translog object
        Hide translog attributes Show translog attributes object
      • query_string object
        Hide query_string attribute Show query_string attribute object

        • lenient boolean | string Required

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          Stringifiedboolean boolean Stringifiedboolean string

      • priority number | string

        One of:
        number-1 number string-2 string
      • top_metrics_max_size number
      • analysis object
        Hide analysis attributes Show analysis attributes object
      • settings object
      • time_series object
        Hide time_series attributes Show time_series attributes object

        • end_time string | number

          A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

          One of:
          DateTime string UnitMillis number

        • start_time string | number

          A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

          One of:
          DateTime string UnitMillis number
      • queries object
        Hide queries attribute Show queries attribute object
        • cache object
          Hide cache attribute Show cache attribute object
      • similarity object

        Configure custom similarity settings to customize how search results are scored.

      • mapping object
        Hide mapping attributes Show mapping attributes object
        • coerce boolean
        • total_fields object
          Hide total_fields attributes Show total_fields attributes object

          • limit number | string

            The maximum number of fields in an index. Field and object mappings, as well as field aliases count towards this limit. The limit is in place to prevent mappings and searches from becoming too large. Higher values can lead to performance degradations and memory issues, especially in clusters with a high load or few resources.

            One of:
            number-1 number string-2 string

          • ignore_dynamic_beyond_limit boolean | string

            This setting determines what happens when a dynamically mapped field would exceed the total fields limit. When set to false (the default), the index request of the document that tries to add a dynamic field to the mapping will fail with the message Limit of total fields [X] has been exceeded. When set to true, the index request will not fail. Instead, fields that would exceed the limit are not added to the mapping, similar to dynamic: false. The fields that were not added to the mapping will be added to the _ignored field.

            One of:
            boolean-1 boolean string-2 string
        • depth object
          Hide depth attribute Show depth attribute object
          • limit number

            The maximum depth for a field, which is measured as the number of inner objects. For instance, if all fields are defined at the root object level, then the depth is 1. If there is one object mapping, then the depth is 2, etc.

        • nested_fields object
          Hide nested_fields attribute Show nested_fields attribute object
          • limit number

            The maximum number of distinct nested mappings in an index. The nested type should only be used in special cases, when arrays of objects need to be queried independently of each other. To safeguard against poorly designed mappings, this setting limits the number of unique nested types per index.

        • nested_objects object
          Hide nested_objects attribute Show nested_objects attribute object
          • limit number

            The maximum number of nested JSON objects that a single document can contain across all nested types. This limit helps to prevent out of memory errors when a document contains too many nested objects.

        • field_name_length object
          Hide field_name_length attribute Show field_name_length attribute object
          • limit number

            Setting for the maximum length of a field name. This setting isn’t really something that addresses mappings explosion but might still be useful if you want to limit the field length. It usually shouldn’t be necessary to set this setting. The default is okay unless a user starts to add a huge number of fields with really long names. Default is Long.MAX_VALUE (no limit).

        • dimension_fields object
          Hide dimension_fields attribute Show dimension_fields attribute object
          • limit number

            [preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

        • source object
          Hide source attribute Show source attribute object
          • mode string Required

            Values are disabled, stored, or synthetic.

        • ignore_malformed boolean | string

          One of:
          boolean-1 boolean string-2 string
      • indexing.slowlog object
        Hide indexing.slowlog attributes Show indexing.slowlog attributes object
        • level string
        • source number
        • reformat boolean
        • threshold object
          Hide threshold attribute Show threshold attribute object
          • index object
            Hide index attributes Show index attributes object
            • warn string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • info string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • debug string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • trace string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • indexing_pressure object
        Hide indexing_pressure attribute Show indexing_pressure attribute object
        • memory object Required
          Hide memory attribute Show memory attribute object
          • limit number

            Number of outstanding bytes that may be consumed by indexing requests. When this limit is reached or exceeded, the node will reject new coordinating and primary operations. When replica operations consume 1.5x this limit, the node will reject new replica operations. Defaults to 10% of the heap.

      • store object
        Hide store attributes Show store attributes object

        • type string Required

          Any of:
          StorageType string StorageType string

          Values are fs, niofs, mmapfs, or hybridfs.

        • allow_mmap boolean

          You can restrict the use of the mmapfs and the related hybridfs store type via the setting node.store.allow_mmap. This is a boolean setting indicating whether or not memory-mapping is allowed. The default is to allow it. This setting is useful, for example, if you are in an environment where you can not control the ability to create a lot of memory maps so you need disable the ability to use memory-mapping.

    • lifecycle object
      Hide lifecycle attributes Show lifecycle attributes object
      • data_retention string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • downsampling object
        Hide downsampling attribute Show downsampling attribute object
        • rounds array[object] Required

          The list of downsampling rounds to execute as part of this downsampling configuration

          Hide rounds attributes Show rounds attributes object
          • after string Required

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • config object Required
            Hide config attribute Show config attribute object
            • fixed_interval string Required

              A date histogram interval. Similar to Duration with additional units: w (week), M (month), q (quarter) and y (year)

      • enabled boolean

        If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

  • data_stream object
    Hide data_stream attributes Show data_stream attributes object
  • priority number

    Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.

  • version number
  • _meta object
    Hide _meta attribute Show _meta attribute object
    • * object Additional properties
  • allow_auto_create boolean

    This setting overrides the value of the action.auto_create_index cluster setting. If set to true in a template, then indices can be automatically created using that template even if auto-creation of indices is disabled via actions.auto_create_index. If set to false, then indices or data streams matching the template must always be explicitly created, and may never be automatically created.

  • ignore_missing_component_templates array[string]

    The configuration option ignore_missing_component_templates can be used when an index template references a component template that might not exist

  • deprecated boolean

    Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
PUT /_index_template/{name} 
curl \
 --request PUT 'http://api.example.com/_index_template/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"index_patterns\" : [\"template*\"],\n  \"priority\" : 1,\n  \"template\": {\n    \"settings\" : {\n      \"number_of_shards\" : 2\n    }\n  }\n}"'
Request examples 
{
  "index_patterns" : ["template*"],
  "priority" : 1,
  "template": {
    "settings" : {
      "number_of_shards" : 2
    }
  }
}
You can include index aliases in an index template. During index creation, the `{index}` placeholder in the alias name will be replaced with the actual index name that the template gets applied to. 
{
  "index_patterns": [
    "template*"
  ],
  "template": {
    "settings": {
      "number_of_shards": 1
    },
    "aliases": {
      "alias1": {},
      "alias2": {
        "filter": {
          "term": {
            "user.id": "kimchy"
          }
        },
        "routing": "shard-1"
      },
      "{index}-alias": {}
    }
  }
}
Response examples (200) 
{
  "acknowledged": true
}

Get field usage stats Technical preview

GET /{index}/_field_usage_stats
Api key auth Basic auth Bearer auth

Get field usage information for each shard and field of an index. Field usage statistics are automatically captured when queries are running on a cluster. A shard-level search request that accesses a given field, even if multiple times during that request, is counted as a single use.

The response body reports the per-shard usage count of the data structures that back the fields in the index. A given request will increment each count by a maximum value of 1, even if the request accesses the same field multiple times.

Path parameters

  • index string | array[string] Required

    Comma-separated list or wildcard expression of index names used to limit the request.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden.

  • ignore_unavailable boolean

    If true, missing or closed indices are not included in the response.

  • fields string | array[string]

    Comma-separated list or wildcard expressions of fields to include in the statistics.

Responses

GET /{index}/_field_usage_stats 
curl \
 --request GET 'http://api.example.com/{index}/_field_usage_stats' \
 --header "Authorization: $API_KEY"
Response examples (200)
An abbreviated response from `GET /my-index-000001/_field_usage_stats`. The `all_fields` object reports the sums of the usage counts for all fields in the index (on the listed shard). 
{
  "_shards": {
    "total": 1,
    "successful": 1,
    "failed": 0
  },
  "my-index-000001": {
    "shards": [
      {
        "tracking_id": "MpOl0QlTQ4SYYhEe6KgJoQ",
        "tracking_started_at_millis": 1625558985010,
        "routing": {
          "state": "STARTED",
          "primary": true,
          "node": "gA6KeeVzQkGURFCUyV-e8Q",
          "relocating_node": null
        },
        "stats": {
          "all_fields": {
            "any": "6",
            "inverted_index": {
              "terms": 1,
              "postings": 1,
              "proximity": 1,
              "positions": 0,
              "term_frequencies": 1,
              "offsets": 0,
              "payloads": 0
            },
            "stored_fields": 2,
            "doc_values": 1,
            "points": 0,
            "norms": 1,
            "term_vectors": 0,
            "knn_vectors": 0
          },
          "fields": {
            "_id": {
              "any": 1,
              "inverted_index": {
                "terms": 1,
                "postings": 1,
                "proximity": 1,
                "positions": 0,
                "term_frequencies": 1,
                "offsets": 0,
                "payloads": 0
              },
              "stored_fields": 1,
              "doc_values": 0,
              "points": 0,
              "norms": 0,
              "term_vectors": 0,
              "knn_vectors": 0
            },
            "_source": {},
            "context": {},
            "message.keyword": {}
          }
        }
      }
    ]
  }
}

Get index templates Added in 7.9.0

GET /_index_template
Api key auth Basic auth Bearer auth

Get information about one or more index templates.

Query parameters

  • local boolean

    If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.

  • flat_settings boolean

    If true, returns settings in flat format.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • include_defaults boolean

    If true, returns all relevant default configurations for the index template.

Responses

GET /_index_template 
curl \
 --request GET 'http://api.example.com/_index_template' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "index_templates": [
    {
      "name": "string",
      "index_template": {
        "index_patterns": "string",
        "composed_of": [
          "string"
        ],
        "template": {
          "aliases": {
            "additionalProperty1": {},
            "additionalProperty2": {}
          },
          "mappings": {
            "all_field": {},
            "date_detection": true,
            "dynamic": "strict",
            "dynamic_date_formats": [
              "string"
            ],
            "dynamic_templates": [
              {}
            ],
            "_field_names": {},
            "index_field": {},
            "_meta": {},
            "numeric_detection": true,
            "properties": {},
            "_routing": {},
            "_size": {},
            "_source": {},
            "runtime": {},
            "enabled": true,
            "subobjects": "true",
            "_data_stream_timestamp": {}
          },
          "settings": {
            "index": {},
            "mode": "string",
            "soft_deletes": {},
            "sort": {},
            "number_of_routing_shards": 42.0,
            "check_on_startup": "true",
            "codec": "string",
            "load_fixed_bitset_filters_eagerly": true,
            "merge": {},
            "search": {},
            "refresh_interval": "string",
            "max_result_window": 42.0,
            "max_inner_result_window": 42.0,
            "max_rescore_window": 42.0,
            "max_docvalue_fields_search": 42.0,
            "max_script_fields": 42.0,
            "max_ngram_diff": 42.0,
            "max_shingle_diff": 42.0,
            "blocks": {},
            "max_refresh_listeners": 42.0,
            "analyze": {},
            "highlight": {},
            "max_terms_count": 42.0,
            "max_regex_length": 42.0,
            "routing": {},
            "gc_deletes": "string",
            "default_pipeline": "string",
            "final_pipeline": "string",
            "lifecycle": {},
            "provided_name": "string",
            "uuid": "string",
            "version": {},
            "max_slices_per_scroll": 42.0,
            "translog": {},
            "query_string": {},
            "top_metrics_max_size": 42.0,
            "analysis": {},
            "settings": {},
            "time_series": {},
            "queries": {},
            "similarity": {},
            "mapping": {},
            "indexing.slowlog": {},
            "indexing_pressure": {},
            "store": {}
          },
          "": {}
        },
        "version": 42.0,
        "priority": 42.0,
        "_meta": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "allow_auto_create": true,
        "data_stream": {
          "hidden": true,
          "allow_custom_routing": true
        },
        "deprecated": true,
        "ignore_missing_component_templates": "string"
      }
    }
  ]
}

Split an index Added in 6.1.0

PUT /{index}/_split/{target}
Api key auth Basic auth Bearer auth

Split an index into a new index with more primary shards.

  • Before you can split an index:

  • The index must be read-only.

  • The cluster health status must be green.

You can do make an index read-only with the following request using the add index block API:

PUT /my_source_index/_block/write

The current write index on a data stream cannot be split. In order to split the current write index, the data stream must first be rolled over so that a new write index is created and then the previous write index can be split.

The number of times the index can be split (and the number of shards that each original shard can be split into) is determined by the index.number_of_routing_shards setting. The number of routing shards specifies the hashing space that is used internally to distribute documents across shards with consistent hashing. For instance, a 5 shard index with number_of_routing_shards set to 30 (5 x 2 x 3) could be split by a factor of 2 or 3.

A split operation:

  • Creates a new target index with the same definition as the source index, but with a larger number of primary shards.
  • Hard-links segments from the source index into the target index. If the file system doesn't support hard-linking, all segments are copied into the new index, which is a much more time consuming process.
  • Hashes all documents again, after low level files are created, to delete documents that belong to a different shard.
  • Recovers the target index as though it were a closed index which had just been re-opened.

IMPORTANT: Indices can only be split if they satisfy the following requirements:

  • The target index must not exist.
  • The source index must have fewer primary shards than the target index.
  • The number of primary shards in the target index must be a multiple of the number of primary shards in the source index.
  • The node handling the split process must have sufficient free disk space to accommodate a second copy of the existing index.

Path parameters

  • index string Required

    Name of the source index to split.

  • target string Required

    Name of the target index to create.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

application/json

Body

  • aliases object

    Aliases for the resulting index.

    Hide aliases attribute Show aliases attribute object
  • settings object

    Configuration options for the target index.

    Hide settings attribute Show settings attribute object
    • * object Additional properties

Responses

PUT /{index}/_split/{target} 
curl \
 --request PUT 'http://api.example.com/{index}/_split/{target}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"settings\": {\n    \"index.number_of_shards\": 2\n  }\n}"'
Request example
Split an existing index into a new index with more primary shards. 
{
  "settings": {
    "index.number_of_shards": 2
  }
}
Response examples (200) 
{
  "acknowledged": true,
  "shards_acknowledged": true,
  "index": "string"
}

Unfreeze an index Deprecated Added in 6.6.0

POST /{index}/_unfreeze
Api key auth Basic auth Bearer auth

When a frozen index is unfrozen, the index goes through the normal recovery process and becomes writeable again.

Path parameters

  • index string Required

    Identifier for the index.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

  • wait_for_active_shards string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

Responses

POST /{index}/_unfreeze 
curl \
 --request POST 'http://api.example.com/{index}/_unfreeze' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "acknowledged": true,
  "shards_acknowledged": true
}

Create an Azure AI studio inference endpoint Added in 8.14.0

PUT /_inference/{task_type}/{azureaistudio_inference_id}
Api key auth Basic auth Bearer auth

Create an inference endpoint to perform an inference task with the azureaistudio service.

When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running. After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count". Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.

Path parameters

  • task_type string Required

    The type of the inference task that the model will perform.

    Values are completion or text_embedding.

  • azureaistudio_inference_id string Required

    The unique identifier of the inference endpoint.

application/json

Body

  • chunking_settings object
    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

    • strategy string

      The chunking strategy: sentence or word.

  • service string Required

    Value is azureaistudio.

  • service_settings object Required
    Hide service_settings attributes Show service_settings attributes object
    • api_key string Required

      A valid API key of your Azure AI Studio model deployment. This key can be found on the overview page for your deployment in the management section of your Azure AI Studio account.

      IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key. After creating the inference model, you cannot change the associated API key. If you want to use a different API key, delete the inference model and recreate it with the same name and the updated API key.

      External documentation
    • endpoint_type string Required

      The type of endpoint that is available for deployment through Azure AI Studio: token or realtime. The token endpoint type is for "pay as you go" endpoints that are billed per token. The realtime endpoint type is for "real-time" endpoints that are billed per hour of usage.

      External documentation
    • target string Required

      The target URL of your Azure AI Studio model deployment. This can be found on the overview page for your deployment in the management section of your Azure AI Studio account.

    • provider string Required

      The model provider for your deployment. Note that some providers may support only certain task types. Supported providers include:

      • cohere - available for text_embedding and completion task types
      • databricks - available for completion task type only
      • meta - available for completion task type only
      • microsoft_phi - available for completion task type only
      • mistral - available for completion task type only
      • openai - available for text_embedding and completion task types
    • rate_limit object
      Hide rate_limit attribute Show rate_limit attribute object
  • task_settings object
    Hide task_settings attributes Show task_settings attributes object
    • do_sample number

      For a completion task, instruct the inference process to perform sampling. It has no effect unless temperature or top_p is specified.

    • max_new_tokens number

      For a completion task, provide a hint for the maximum number of output tokens to be generated.

    • temperature number

      For a completion task, control the apparent creativity of generated completions with a sampling temperature. It must be a number in the range of 0.0 to 2.0. It should not be used if top_p is specified.

    • top_p number

      For a completion task, make the model consider the results of the tokens with nucleus sampling probability. It is an alternative value to temperature and must be a number in the range of 0.0 to 2.0. It should not be used if temperature is specified.

    • user string

      For a text_embedding task, specify the user issuing the request. This information can be used for abuse detection.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • chunking_settings object
      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      • strategy string

        The chunking strategy: sentence or word.

    • service string Required

      The service type

    • service_settings object Required
    • task_settings object
    • inference_id string Required

      The inference Id

    • task_type string Required

      Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
PUT /_inference/{task_type}/{azureaistudio_inference_id} 
curl \
 --request PUT 'http://api.example.com/_inference/{task_type}/{azureaistudio_inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"service\": \"azureaistudio\",\n    \"service_settings\": {\n        \"api_key\": \"Azure-AI-Studio-API-key\",\n        \"target\": \"Target-Uri\",\n        \"provider\": \"openai\",\n        \"endpoint_type\": \"token\"\n    }\n}"'
Request examples
Run `PUT _inference/text_embedding/azure_ai_studio_embeddings` to create an inference endpoint that performs a text_embedding task. Note that you do not specify a model here, as it is defined already in the Azure AI Studio deployment. 
{
    "service": "azureaistudio",
    "service_settings": {
        "api_key": "Azure-AI-Studio-API-key",
        "target": "Target-Uri",
        "provider": "openai",
        "endpoint_type": "token"
    }
}
Run `PUT _inference/completion/azure_ai_studio_completion` to create an inference endpoint that performs a completion task. 
{
    "service": "azureaistudio",
    "service_settings": {
        "api_key": "Azure-AI-Studio-API-key",
        "target": "Target-URI",
        "provider": "databricks",
        "endpoint_type": "realtime"
    }
}
Response examples (200) 
{
  "chunking_settings": {
    "max_chunk_size": 42.0,
    "overlap": 42.0,
    "sentence_overlap": 42.0,
    "strategy": "string"
  },
  "service": "string",
  "service_settings": {},
  "task_settings": {},
  "inference_id": "string",
  "task_type": "sparse_embedding"
}

Simulate data ingestion Technical preview

GET /_ingest/_simulate
Api key auth Basic auth Bearer auth

Run ingest pipelines against a set of provided documents, optionally with substitute pipeline definitions, to simulate ingesting data into an index.

This API is meant to be used for troubleshooting or pipeline development, as it does not actually index any data into Elasticsearch.

The API runs the default and final pipeline for that index against a set of documents provided in the body of the request. If a pipeline contains a reroute processor, it follows that reroute processor to the new index, running that index's pipelines as well the same way that a non-simulated ingest would. No data is indexed into Elasticsearch. Instead, the transformed document is returned, along with the list of pipelines that have been run and the name of the index where the document would have been indexed if this were not a simulation. The transformed document is validated against the mappings that would apply to this index, and any validation error is reported in the result.

This API differs from the simulate pipeline API in that you specify a single pipeline for that API, and it runs only that one pipeline. The simulate pipeline API is more useful for developing a single pipeline, while the simulate ingest API is more useful for troubleshooting the interaction of the various pipelines that get applied when ingesting into an index.

By default, the pipeline definitions that are currently in the system are used. However, you can supply substitute pipeline definitions in the body of the request. These will be used in place of the pipeline definitions that are already in the system. This can be used to replace existing pipeline definitions or to create new ones. The pipeline substitutions are used only within this request.

Query parameters

  • pipeline string

    The pipeline to use as the default pipeline. This value can be used to override the default pipeline of the index.

application/json

Body Required

  • docs array[object] Required

    Sample documents to test in the pipeline.

    Hide docs attributes Show docs attributes object
  • component_template_substitutions object

    A map of component template names to substitute component template definition objects.

    Hide component_template_substitutions attribute Show component_template_substitutions attribute object
  • index_template_substitutions object

    A map of index template names to substitute index template definition objects.

    Hide index_template_substitutions attribute Show index_template_substitutions attribute object
  • mapping_addition object
    Hide mapping_addition attributes Show mapping_addition attributes object
  • pipeline_substitutions object

    Pipelines to test. If you don’t specify the pipeline request path parameter, this parameter is required. If you specify both this and the request path parameter, the API only uses the request path parameter.

    Hide pipeline_substitutions attribute Show pipeline_substitutions attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • description string

        Description of the ingest pipeline.

      • on_failure array[object]

        Processors to run immediately after a processor failure.

        Hide on_failure attributes Show on_failure attributes object
        • append object
          Hide append attributes Show append attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • value
          • allow_duplicates boolean

            If false, the processor does not append values already present in the field.

        • attachment object
          Hide attachment attributes Show attachment attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • indexed_chars number

            The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

          • indexed_chars_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • remove_binary boolean

            If true, the binary field will be removed from the document

          • resource_name string

            Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

        • bytes object
          Hide bytes attributes Show bytes attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • circle object
          Hide circle attributes Show circle attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • error_distance number Required

            The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • shape_type string Required

            Values are geo_shape or shape.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • community_id object
          Hide community_id attributes Show community_id attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • source_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • iana_number string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_type string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_code string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • transport string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • seed number

            Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • convert object
          Hide convert attributes Show convert attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • type string Required

            Values are integer, long, double, float, boolean, ip, string, or auto.

        • csv object
          Hide csv attributes Show csv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • empty_value object

            Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • quote string

            Quote used in CSV, has to be single character string.

          • separator string

            Separator used in CSV, has to be single character string.

          • target_fields string | array[string] Required
          • trim boolean

            Trim whitespaces in unquoted fields.

        • date object
          Hide date attributes Show date attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • formats array[string] Required

            An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • locale string

            The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • timezone string

            The timezone to use when parsing the date. Supports template snippets.

          • output_format string

            The format to use when writing the date to target_field. Must be a valid java time pattern.

        • date_index_name object
          Hide date_index_name attributes Show date_index_name attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • date_formats array[string] Required

            An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • date_rounding string Required

            How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • index_name_format string

            The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

          • index_name_prefix string

            A prefix of the index name to be prepended before the printed date. Supports template snippets.

          • locale string

            The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

          • timezone string

            The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

        • dissect object
          Hide dissect attributes Show dissect attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • append_separator string

            The character(s) that separate the appended fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to apply to the field.

        • dot_expander object
          Hide dot_expander attributes Show dot_expander attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • override boolean

            Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

          • path string

            The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

        • drop object
          Hide drop attributes Show drop attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • enrich object
          Hide enrich attributes Show enrich attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • max_matches number

            The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

          • override boolean

            If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • policy_name string Required

            The name of the enrich policy to use.

          • shape_relation string

            Values are intersects, disjoint, within, or contains.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • fail object
          Hide fail attributes Show fail attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • message string Required

            The error message thrown by the processor. Supports template snippets.

        • fingerprint object
          Hide fingerprint attributes Show fingerprint attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • fields string | array[string] Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • salt string

            Salt value for the hash function.

          • method string

            Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

          • ignore_missing boolean

            If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

        • foreach object
          Hide foreach attributes Show foreach attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true, the processor silently exits without changing the document if the field is null or missing.

          • processor object Required
        • ip_location object
          Hide ip_location attributes Show ip_location attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found IP location data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the IP location lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • geo_grid object
          Hide geo_grid attributes Show geo_grid attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            The field to interpret as a geo-tile.= The field format is determined by the tile_type.

          • tile_type string Required

            Values are geotile, geohex, or geohash.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • parent_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • non_children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • precision_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_format string

            Values are geojson or wkt.

        • geoip object
          Hide geoip attributes Show geoip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found geoip data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the geoip lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • grok object
          Hide grok attributes Show grok attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • ecs_compatibility string

            Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern_definitions object

            A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          • patterns array[string] Required

            An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

          • trace_match boolean

            When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

        • gsub object
          Hide gsub attributes Show gsub attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to be replaced.

          • replacement string Required

            The string to replace the matching patterns with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • html_strip object
          Hide html_strip attributes Show html_strip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document,

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • inference object
          Hide inference attributes Show inference attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • model_id string Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_map object

            Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          • inference_config object
          • input_output
          • ignore_missing boolean

            If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

        • join object
          Hide join attributes Show join attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • separator string Required

            The separator character.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • json object
          Hide json attributes Show json attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • add_to_root boolean

            Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

          • add_to_root_conflict_strategy string

            Values are replace or merge.

          • allow_duplicate_keys boolean

            When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • kv object
          Hide kv attributes Show kv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • exclude_keys array[string]

            List of keys to exclude from document.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_split string Required

            Regex pattern to use for splitting key-value pairs.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • include_keys array[string]

            List of keys to filter and insert into document. Defaults to including all keys.

          • prefix string

            Prefix to be added to extracted keys.

          • strip_brackets boolean

            If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • trim_key string

            String of characters to trim from extracted keys.

          • trim_value string

            String of characters to trim from extracted values.

          • value_split string Required

            Regex pattern to use for splitting the key from the value within a key-value pair.

        • lowercase object
          Hide lowercase attributes Show lowercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • network_direction object
          Hide network_direction attributes Show network_direction attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • internal_networks array[string]

            List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

          • internal_networks_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • pipeline object
          Hide pipeline attributes Show pipeline attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • name string Required
          • ignore_missing_pipeline boolean

            Whether to ignore missing pipelines instead of failing.

        • redact object
          Hide redact attributes Show redact attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • patterns array[string] Required

            A list of grok expressions to match and redact named captures with

          • pattern_definitions object
          • prefix string

            Start a redacted section with this token

          • suffix string

            End a redacted section with this token

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • skip_if_unlicensed boolean

            If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

          • trace_redact boolean

            If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

        • registered_domain object
          Hide registered_domain attributes Show registered_domain attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • remove object
          Hide remove attributes Show remove attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string | array[string] Required
          • keep string | array[string]
          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • rename object
          Hide rename attributes Show rename attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • reroute object
          Hide reroute attributes Show reroute attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • destination string

            A static value for the target. Can’t be set when the dataset or namespace option is set.

          • dataset
          • namespace
        • script object
          Hide script attributes Show script attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • id string
          • lang string

            Script language.

          • params object

            Object containing parameters for the script.

          • source string

            Inline script. If no id is specified, this parameter is required.

        • set object
          Hide set attributes Show set attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • copy_from string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_empty_value boolean

            If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

          • media_type string

            The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

          • override boolean

            If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • value object

            The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

        • set_security_user object
          Hide set_security_user attributes Show set_security_user attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what user related properties are added to the field.

        • sort object
          Hide sort attributes Show sort attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • order string

            Values are asc or desc.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • split object
          Hide split attributes Show split attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • preserve_trailing boolean

            Preserves empty trailing fields, if any.

          • separator string Required

            A regex which matches the separator, for example, , or \s+.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • terminate object
          Hide terminate attributes Show terminate attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • trim object
          Hide trim attributes Show trim attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uppercase object
          Hide uppercase attributes Show uppercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • urldecode object
          Hide urldecode attributes Show urldecode attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uri_parts object
          Hide uri_parts attributes Show uri_parts attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • keep_original boolean

            If true, the processor copies the unparsed URI to <target_field>.original.

          • remove_if_successful boolean

            If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • user_agent object
          Hide user_agent attributes Show user_agent attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • regex_file string

            The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what properties are added to target_field.

            Values are name, os, device, original, or version.

          • extract_device_type boolean Beta

            Extracts device type from the user agent string on a best-effort basis.

      • processors array[object]

        Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.

        Hide processors attributes Show processors attributes object
        • append object
          Hide append attributes Show append attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • value
          • allow_duplicates boolean

            If false, the processor does not append values already present in the field.

        • attachment object
          Hide attachment attributes Show attachment attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • indexed_chars number

            The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

          • indexed_chars_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • remove_binary boolean

            If true, the binary field will be removed from the document

          • resource_name string

            Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

        • bytes object
          Hide bytes attributes Show bytes attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • circle object
          Hide circle attributes Show circle attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • error_distance number Required

            The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • shape_type string Required

            Values are geo_shape or shape.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • community_id object
          Hide community_id attributes Show community_id attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • source_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • iana_number string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_type string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_code string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • transport string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • seed number

            Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • convert object
          Hide convert attributes Show convert attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • type string Required

            Values are integer, long, double, float, boolean, ip, string, or auto.

        • csv object
          Hide csv attributes Show csv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • empty_value object

            Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • quote string

            Quote used in CSV, has to be single character string.

          • separator string

            Separator used in CSV, has to be single character string.

          • target_fields string | array[string] Required
          • trim boolean

            Trim whitespaces in unquoted fields.

        • date object
          Hide date attributes Show date attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • formats array[string] Required

            An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • locale string

            The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • timezone string

            The timezone to use when parsing the date. Supports template snippets.

          • output_format string

            The format to use when writing the date to target_field. Must be a valid java time pattern.

        • date_index_name object
          Hide date_index_name attributes Show date_index_name attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • date_formats array[string] Required

            An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • date_rounding string Required

            How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • index_name_format string

            The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

          • index_name_prefix string

            A prefix of the index name to be prepended before the printed date. Supports template snippets.

          • locale string

            The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

          • timezone string

            The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

        • dissect object
          Hide dissect attributes Show dissect attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • append_separator string

            The character(s) that separate the appended fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to apply to the field.

        • dot_expander object
          Hide dot_expander attributes Show dot_expander attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • override boolean

            Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

          • path string

            The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

        • drop object
          Hide drop attributes Show drop attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • enrich object
          Hide enrich attributes Show enrich attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • max_matches number

            The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

          • override boolean

            If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • policy_name string Required

            The name of the enrich policy to use.

          • shape_relation string

            Values are intersects, disjoint, within, or contains.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • fail object
          Hide fail attributes Show fail attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • message string Required

            The error message thrown by the processor. Supports template snippets.

        • fingerprint object
          Hide fingerprint attributes Show fingerprint attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • fields string | array[string] Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • salt string

            Salt value for the hash function.

          • method string

            Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

          • ignore_missing boolean

            If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

        • foreach object
          Hide foreach attributes Show foreach attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true, the processor silently exits without changing the document if the field is null or missing.

          • processor object Required
        • ip_location object
          Hide ip_location attributes Show ip_location attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found IP location data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the IP location lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • geo_grid object
          Hide geo_grid attributes Show geo_grid attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            The field to interpret as a geo-tile.= The field format is determined by the tile_type.

          • tile_type string Required

            Values are geotile, geohex, or geohash.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • parent_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • non_children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • precision_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_format string

            Values are geojson or wkt.

        • geoip object
          Hide geoip attributes Show geoip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found geoip data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the geoip lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • grok object
          Hide grok attributes Show grok attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • ecs_compatibility string

            Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern_definitions object

            A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          • patterns array[string] Required

            An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

          • trace_match boolean

            When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

        • gsub object
          Hide gsub attributes Show gsub attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to be replaced.

          • replacement string Required

            The string to replace the matching patterns with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • html_strip object
          Hide html_strip attributes Show html_strip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document,

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • inference object
          Hide inference attributes Show inference attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • model_id string Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_map object

            Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          • inference_config object
          • input_output
          • ignore_missing boolean

            If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

        • join object
          Hide join attributes Show join attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • separator string Required

            The separator character.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • json object
          Hide json attributes Show json attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • add_to_root boolean

            Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

          • add_to_root_conflict_strategy string

            Values are replace or merge.

          • allow_duplicate_keys boolean

            When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • kv object
          Hide kv attributes Show kv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • exclude_keys array[string]

            List of keys to exclude from document.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_split string Required

            Regex pattern to use for splitting key-value pairs.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • include_keys array[string]

            List of keys to filter and insert into document. Defaults to including all keys.

          • prefix string

            Prefix to be added to extracted keys.

          • strip_brackets boolean

            If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • trim_key string

            String of characters to trim from extracted keys.

          • trim_value string

            String of characters to trim from extracted values.

          • value_split string Required

            Regex pattern to use for splitting the key from the value within a key-value pair.

        • lowercase object
          Hide lowercase attributes Show lowercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • network_direction object
          Hide network_direction attributes Show network_direction attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • internal_networks array[string]

            List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

          • internal_networks_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • pipeline object
          Hide pipeline attributes Show pipeline attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • name string Required
          • ignore_missing_pipeline boolean

            Whether to ignore missing pipelines instead of failing.

        • redact object
          Hide redact attributes Show redact attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • patterns array[string] Required

            A list of grok expressions to match and redact named captures with

          • pattern_definitions object
          • prefix string

            Start a redacted section with this token

          • suffix string

            End a redacted section with this token

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • skip_if_unlicensed boolean

            If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

          • trace_redact boolean

            If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

        • registered_domain object
          Hide registered_domain attributes Show registered_domain attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • remove object
          Hide remove attributes Show remove attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string | array[string] Required
          • keep string | array[string]
          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • rename object
          Hide rename attributes Show rename attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • reroute object
          Hide reroute attributes Show reroute attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • destination string

            A static value for the target. Can’t be set when the dataset or namespace option is set.

          • dataset
          • namespace
        • script object
          Hide script attributes Show script attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • id string
          • lang string

            Script language.

          • params object

            Object containing parameters for the script.

          • source string

            Inline script. If no id is specified, this parameter is required.

        • set object
          Hide set attributes Show set attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • copy_from string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_empty_value boolean

            If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

          • media_type string

            The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

          • override boolean

            If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • value object

            The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

        • set_security_user object
          Hide set_security_user attributes Show set_security_user attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what user related properties are added to the field.

        • sort object
          Hide sort attributes Show sort attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • order string

            Values are asc or desc.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • split object
          Hide split attributes Show split attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • preserve_trailing boolean

            Preserves empty trailing fields, if any.

          • separator string Required

            A regex which matches the separator, for example, , or \s+.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • terminate object
          Hide terminate attributes Show terminate attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • trim object
          Hide trim attributes Show trim attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uppercase object
          Hide uppercase attributes Show uppercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • urldecode object
          Hide urldecode attributes Show urldecode attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uri_parts object
          Hide uri_parts attributes Show uri_parts attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • keep_original boolean

            If true, the processor copies the unparsed URI to <target_field>.original.

          • remove_if_successful boolean

            If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • user_agent object
          Hide user_agent attributes Show user_agent attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • regex_file string

            The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what properties are added to target_field.

            Values are name, os, device, original, or version.

          • extract_device_type boolean Beta

            Extracts device type from the user agent string on a best-effort basis.

      • version number
      • deprecated boolean

        Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required
      Hide docs attribute Show docs attribute object
      • doc object
        Hide doc attributes Show doc attributes object
        • _id string Required
        • _index string Required
        • _source object Required

          JSON body for the document.

          Hide _source attribute Show _source attribute object
          • * object Additional properties

        • _version number | string Required

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          VersionNumber number StringifiedVersionNumber string
        • executed_pipelines array[string] Required

          A list of the names of the pipelines executed on this document.

        • ignored_fields array[object]

          A list of the fields that would be ignored at the indexing step. For example, a field whose value is larger than the allowed limit would make it through all of the pipelines, but would not be indexed into Elasticsearch.

          Hide ignored_fields attribute Show ignored_fields attribute object
          • * string Additional properties
        • error object
          Hide error attributes Show error attributes object
GET /_ingest/_simulate 
curl \
 --request GET 'http://api.example.com/_ingest/_simulate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"docs\": [\n    {\n      \"_id\": 123,\n      \"_index\": \"my-index\",\n      \"_source\": {\n        \"foo\": \"bar\"\n      }\n    },\n    {\n      \"_id\": 456,\n      \"_index\": \"my-index\",\n      \"_source\": {\n        \"foo\": \"rab\"\n      }\n    }\n  ]\n}"'
Request examples
In this example the index `my-index` has a default pipeline called `my-pipeline` and a final pipeline called `my-final-pipeline`. Since both documents are being ingested into `my-index`, both pipelines are run using the pipeline definitions that are already in the system. 
{
  "docs": [
    {
      "_id": 123,
      "_index": "my-index",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_id": 456,
      "_index": "my-index",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}
In this example the index `my-index` has a default pipeline called `my-pipeline` and a final pipeline called `my-final-pipeline`. But a substitute definition of `my-pipeline` is provided in `pipeline_substitutions`. The substitute `my-pipeline` will be used in place of the `my-pipeline` that is in the system, and then the `my-final-pipeline` that is already defined in the system will run. 
{
  "docs": [
    {
      "_index": "my-index",
      "_id": 123,
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "my-index",
      "_id": 456,
      "_source": {
        "foo": "rab"
      }
    }
  ],
  "pipeline_substitutions": {
    "my-pipeline": {
      "processors": [
        {
          "uppercase": {
            "field": "foo"
          }
        }
      ]
    }
  }
}
In this example, imagine that the index `my-index` has a strict mapping with only the `foo` keyword field defined. Say that field mapping came from a component template named `my-mappings-template`. You want to test adding a new field, `bar`. So a substitute definition of `my-mappings-template` is provided in `component_template_substitutions`. The substitute `my-mappings-template` will be used in place of the existing mapping for `my-index` and in place of the `my-mappings-template` that is in the system. 
{
  "docs": [
    {
      "_index": "my-index",
      "_id": "123",
      "_source": {
        "foo": "foo"
      }
    },
    {
      "_index": "my-index",
      "_id": "456",
      "_source": {
        "bar": "rab"
      }
    }
  ],
  "component_template_substitutions": {
    "my-mappings_template": {
      "template": {
        "mappings": {
          "dynamic": "strict",
          "properties": {
            "foo": {
              "type": "keyword"
            },
            "bar": {
              "type": "keyword"
            }
          }
        }
      }
    }
  }
}
The pipeline, component template, and index template substitutions replace the existing pipeline details for the duration of this request. 
{
  "docs": [
    {
      "_id": "id",
      "_index": "my-index",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_id": "id",
      "_index": "my-index",
      "_source": {
        "foo": "rab"
      }
    }
  ],
  "pipeline_substitutions": {
    "my-pipeline": {
      "processors": [
        {
          "set": {
            "field": "field3",
            "value": "value3"
          }
        }
      ]
    }
  },
  "component_template_substitutions": {
    "my-component-template": {
      "template": {
        "mappings": {
          "dynamic": true,
          "properties": {
            "field3": {
              "type": "keyword"
            }
          }
        },
        "settings": {
          "index": {
            "default_pipeline": "my-pipeline"
          }
        }
      }
    }
  },
  "index_template_substitutions": {
    "my-index-template": {
      "index_patterns": [
        "my-index-*"
      ],
      "composed_of": [
        "component_template_1",
        "component_template_2"
      ]
    }
  },
  "mapping_addition": {
    "dynamic": "strict",
    "properties": {
      "foo": {
        "type": "keyword"
      }
    }
  }
}
Response examples (200)
A successful response when the simulation uses pipeline definitions that are already in the system. 
{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "bar"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": "-3,",
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "rab"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}
A successful response when the simulation uses pipeline substitutions. 
{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "BAR"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "RAB"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}
A successful response when the simulation uses pipeline substitutions. 
{
  "docs": [
    {
      "doc": {
        "_id": "123",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "foo": "foo"
        },
        "executed_pipelines": []
      }
    },
    {
      "doc": {
        "_id": "456",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "bar": "rab"
        },
      "executed_pipelines": []
      }
    }
  ]
}

Get the basic license status Added in 6.3.0

GET /_license/basic_status
Api key auth Basic auth Bearer auth

Responses

GET /_license/basic_status 
curl \
 --request GET 'http://api.example.com/_license/basic_status' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_license/basic_status`. 
{
  "eligible_to_start_basic": true
}

Delete a Logstash pipeline Added in 7.12.0

DELETE /_logstash/pipeline/{id}
Api key auth Basic auth Bearer auth

Delete a pipeline that is used for Logstash Central Management. If the request succeeds, you receive an empty response with an appropriate status code.

External documentation

Path parameters

  • id string Required

    An identifier for the pipeline.

Responses

DELETE /_logstash/pipeline/{id} 
curl \
 --request DELETE 'http://api.example.com/_logstash/pipeline/{id}' \
 --header "Authorization: $API_KEY"

Delete a calendar Added in 6.2.0

DELETE /_ml/calendars/{calendar_id}
Api key auth Basic auth Bearer auth

Remove all scheduled events from a calendar, then delete it.

Path parameters

  • calendar_id string Required

    A string that uniquely identifies a calendar.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
DELETE /_ml/calendars/{calendar_id} 
curl \
 --request DELETE 'http://api.example.com/_ml/calendars/{calendar_id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response when deleting a calendar. 
{
  "acknowledged": true
}

Get info about events in calendars Added in 6.2.0

GET /_ml/calendars/{calendar_id}/events
Api key auth Basic auth Bearer auth

Path parameters

  • calendar_id string Required

    A string that uniquely identifies a calendar. You can get information for multiple calendars by using a comma-separated list of ids or a wildcard expression. You can get information for all calendars by using _all or * or by omitting the calendar identifier.

Query parameters

  • end string | number

    Specifies to get events with timestamps earlier than this time.

  • from number

    Skips the specified number of events.

  • job_id string

    Specifies to get events for a specific anomaly detection job identifier or job group. It must be used with a calendar identifier of _all or *.

  • size number

    Specifies the maximum number of events to obtain.

  • start string | number

    Specifies to get events with timestamps after this time.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • events array[object] Required
      Hide events attributes Show events attributes object
      • calendar_id string
      • event_id string
      • description string Required

        A description of the scheduled event.

      • end_time string | number Required

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
        DateTime string UnitMillis number

      • start_time string | number Required

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
        DateTime string UnitMillis number
GET /_ml/calendars/{calendar_id}/events 
curl \
 --request GET 'http://api.example.com/_ml/calendars/{calendar_id}/events' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "count": 42.0,
  "events": [
    {
      "calendar_id": "string",
      "event_id": "string",
      "description": "string",
      "": "string"
    }
  ]
}

Update a snapshot Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_update
Api key auth Basic auth Bearer auth

Updates certain properties of a snapshot.

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

  • snapshot_id string Required

    Identifier for the model snapshot.

application/json

Body Required

  • description string

    A description of the model snapshot.

  • retain boolean

    If true, this snapshot will not be deleted during automatic cleanup of snapshots older than model_snapshot_retention_days. However, this snapshot will be deleted when the job is deleted.

Responses

POST /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_update 
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_update' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"description":"string","retain":true}'
Request examples 
{
  "description": "string",
  "retain": true
}
Response examples (200) 
{
  "acknowledged": true,
  "model": {
    "description": "string",
    "job_id": "string",
    "latest_record_time_stamp": 42.0,
    "latest_result_time_stamp": 42.0,
    "min_version": "string",
    "model_size_stats": {
      "bucket_allocation_failures_count": 42.0,
      "job_id": "string",
      "": 42.0,
      "memory_status": "ok",
      "assignment_memory_basis": "string",
      "result_type": "string",
      "total_by_field_count": 42.0,
      "total_over_field_count": 42.0,
      "total_partition_field_count": 42.0,
      "categorization_status": "ok",
      "categorized_doc_count": 42.0,
      "dead_category_count": 42.0,
      "failed_category_count": 42.0,
      "frequent_category_count": 42.0,
      "rare_category_count": 42.0,
      "total_category_count": 42.0,
      "timestamp": 42.0
    },
    "retain": true,
    "snapshot_doc_count": 42.0,
    "snapshot_id": "string",
    "timestamp": 42.0
  }
}

Preview features used by data frame analytics Added in 7.13.0

GET /_ml/data_frame/analytics/_preview
Api key auth Basic auth Bearer auth

Preview the extracted features used by a data frame analytics config.

application/json

Body

  • config object
    Hide config attributes Show config attributes object
    • source object Required
      Hide source attributes Show source attributes object
      • index string | array[string] Required
      • runtime_mappings object
        Hide runtime_mappings attribute Show runtime_mappings attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • fields object

            For type composite

            Hide fields attribute Show fields attribute object
            • * object Additional properties
              Hide * attribute Show * attribute object
              • type string Required

                Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

          • fetch_fields array[object]

            For type lookup

            Hide fetch_fields attributes Show fetch_fields attributes object
            • field string Required

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • format string
          • format string

            A custom format for date type runtime fields.

          • input_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_index string
          • script object
            Hide script attributes Show script attributes object
            • source string

              The script source.

            • id string
            • params object

              Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

              Hide params attribute Show params attribute object
              • * object Additional properties

            • lang string

              Any of:
              ScriptLanguage string ScriptLanguage string

              Values are painless, expression, mustache, or java.

            • options object
              Hide options attribute Show options attribute object
              • * string Additional properties
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • _source object
        Hide _source attributes Show _source attributes object
        • includes array[string] Required

          An array of strings that defines the fields that will be excluded from the analysis. You do not need to add fields with unsupported data types to excludes, these fields are excluded from the analysis automatically.

        • excludes array[string] Required

          An array of strings that defines the fields that will be included in the analysis.

      • query object

        The Elasticsearch query domain-specific language (DSL). This value corresponds to the query object in an Elasticsearch search POST body. All the options that are supported by Elasticsearch can be used, as this object is passed verbatim to Elasticsearch. By default, this property has the following value: {"match_all": {}}.

        Query DSL
    • analysis object Required
      Hide analysis attributes Show analysis attributes object
      • classification object
        Hide classification attributes Show classification attributes object
        • alpha number

          Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

        • dependent_variable string Required

          Defines which field of the document is to be predicted. It must match one of the fields in the index being used to train. If this field is missing from a document, then that document will not be used for training, but a prediction with the trained model will be generated for it. It is also known as continuous target variable. For classification analysis, the data type of the field must be numeric (integer, short, long, byte), categorical (ip or keyword), or boolean. There must be no more than 30 different values in this field. For regression analysis, the data type of the field must be numeric.

        • downsample_factor number

          Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

        • early_stopping_enabled boolean

          Advanced configuration option. Specifies whether the training process should finish if it is not finding any better performing models. If disabled, the training process can take significantly longer and the chance of finding a better performing model is unremarkable.

        • eta number

          Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

        • eta_growth_rate_per_tree number

          Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

        • feature_bag_fraction number

          Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

        • feature_processors array[object]

          Advanced configuration option. A collection of feature preprocessors that modify one or more included fields. The analysis uses the resulting one or more features instead of the original document field. However, these features are ephemeral; they are not stored in the destination index. Multiple feature_processors entries can refer to the same document fields. Automatic categorical feature encoding still occurs for the fields that are unprocessed by a custom processor or that have categorical values. Use this property only if you want to override the automatic feature encoding of the specified fields.

          Hide feature_processors attributes Show feature_processors attributes object
        • gamma number

          Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

        • lambda number

          Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

        • max_optimization_rounds_per_hyperparameter number

          Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

        • max_trees number

          Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

        • num_top_feature_importance_values number

          Advanced configuration option. Specifies the maximum number of feature importance values per document to return. By default, no feature importance calculation occurs.

        • prediction_field_name string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • randomize_seed number

          Defines the seed for the random generator that is used to pick training data. By default, it is randomly generated. Set it to a specific value to use the same training data each time you start a job (assuming other related parameters such as source and analyzed_fields are the same).

        • soft_tree_depth_limit number

          Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

        • soft_tree_depth_tolerance number

          Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

        • training_percent string | number

          One of:
          Percentage string Percentage number
        • class_assignment_objective string
        • num_top_classes number

          Defines the number of categories for which the predicted probabilities are reported. It must be non-negative or -1. If it is -1 or greater than the total number of categories, probabilities are reported for all categories; if you have a large number of categories, there could be a significant effect on the size of your destination index. NOTE: To use the AUC ROC evaluation method, num_top_classes must be set to -1 or a value greater than or equal to the total number of categories.

      • outlier_detection object
        Hide outlier_detection attributes Show outlier_detection attributes object
        • compute_feature_influence boolean

          Specifies whether the feature influence calculation is enabled.

        • feature_influence_threshold number

          The minimum outlier score that a document needs to have in order to calculate its feature influence score. Value range: 0-1.

        • method string

          The method that outlier detection uses. Available methods are lof, ldof, distance_kth_nn, distance_knn, and ensemble. The default value is ensemble, which means that outlier detection uses an ensemble of different methods and normalises and combines their individual outlier scores to obtain the overall outlier score.

        • n_neighbors number

          Defines the value for how many nearest neighbors each method of outlier detection uses to calculate its outlier score. When the value is not set, different values are used for different ensemble members. This default behavior helps improve the diversity in the ensemble; only override it if you are confident that the value you choose is appropriate for the data set.

        • outlier_fraction number

          The proportion of the data set that is assumed to be outlying prior to outlier detection. For example, 0.05 means it is assumed that 5% of values are real outliers and 95% are inliers.

        • standardization_enabled boolean

          If true, the following operation is performed on the columns before computing outlier scores: (x_i - mean(x_i)) / sd(x_i).

      • regression object
        Hide regression attributes Show regression attributes object
        • alpha number

          Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

        • dependent_variable string Required

          Defines which field of the document is to be predicted. It must match one of the fields in the index being used to train. If this field is missing from a document, then that document will not be used for training, but a prediction with the trained model will be generated for it. It is also known as continuous target variable. For classification analysis, the data type of the field must be numeric (integer, short, long, byte), categorical (ip or keyword), or boolean. There must be no more than 30 different values in this field. For regression analysis, the data type of the field must be numeric.

        • downsample_factor number

          Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

        • early_stopping_enabled boolean

          Advanced configuration option. Specifies whether the training process should finish if it is not finding any better performing models. If disabled, the training process can take significantly longer and the chance of finding a better performing model is unremarkable.

        • eta number

          Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

        • eta_growth_rate_per_tree number

          Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

        • feature_bag_fraction number

          Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

        • feature_processors array[object]

          Advanced configuration option. A collection of feature preprocessors that modify one or more included fields. The analysis uses the resulting one or more features instead of the original document field. However, these features are ephemeral; they are not stored in the destination index. Multiple feature_processors entries can refer to the same document fields. Automatic categorical feature encoding still occurs for the fields that are unprocessed by a custom processor or that have categorical values. Use this property only if you want to override the automatic feature encoding of the specified fields.

          Hide feature_processors attributes Show feature_processors attributes object
        • gamma number

          Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

        • lambda number

          Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

        • max_optimization_rounds_per_hyperparameter number

          Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

        • max_trees number

          Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

        • num_top_feature_importance_values number

          Advanced configuration option. Specifies the maximum number of feature importance values per document to return. By default, no feature importance calculation occurs.

        • prediction_field_name string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • randomize_seed number

          Defines the seed for the random generator that is used to pick training data. By default, it is randomly generated. Set it to a specific value to use the same training data each time you start a job (assuming other related parameters such as source and analyzed_fields are the same).

        • soft_tree_depth_limit number

          Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

        • soft_tree_depth_tolerance number

          Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

        • training_percent string | number

          One of:
          Percentage string Percentage number
        • loss_function string

          The loss function used during regression. Available options are mse (mean squared error), msle (mean squared logarithmic error), huber (Pseudo-Huber loss).

        • loss_function_parameter number

          A positive number that is used as a parameter to the loss_function.

    • model_memory_limit string
    • max_num_threads number
    • analyzed_fields object
      Hide analyzed_fields attributes Show analyzed_fields attributes object
      • includes array[string] Required

        An array of strings that defines the fields that will be excluded from the analysis. You do not need to add fields with unsupported data types to excludes, these fields are excluded from the analysis automatically.

      • excludes array[string] Required

        An array of strings that defines the fields that will be included in the analysis.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • feature_values array[object] Required

      An array of objects that contain feature name and value pairs. The features have been processed and indicate what will be sent to the model for training.

      Hide feature_values attribute Show feature_values attribute object
      • * string Additional properties
GET /_ml/data_frame/analytics/_preview 
curl \
 --request GET 'http://api.example.com/_ml/data_frame/analytics/_preview' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"config":{"source":{"index":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"_source":{"includes":["string"],"excludes":["string"]},"query":{}},"analysis":{"classification":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","class_assignment_objective":"string","num_top_classes":42.0},"outlier_detection":{"compute_feature_influence":true,"feature_influence_threshold":42.0,"method":"string","n_neighbors":42.0,"outlier_fraction":42.0,"standardization_enabled":true},"regression":{"alpha":42.0,"dependent_variable":"string","downsample_factor":42.0,"early_stopping_enabled":true,"eta":42.0,"eta_growth_rate_per_tree":42.0,"feature_bag_fraction":42.0,"feature_processors":[{"frequency_encoding":{},"multi_encoding":{},"n_gram_encoding":{},"one_hot_encoding":{},"target_mean_encoding":{}}],"gamma":42.0,"lambda":42.0,"max_optimization_rounds_per_hyperparameter":42.0,"max_trees":42.0,"num_top_feature_importance_values":42.0,"prediction_field_name":"string","randomize_seed":42.0,"soft_tree_depth_limit":42.0,"soft_tree_depth_tolerance":42.0,"":"string","loss_function":"string","loss_function_parameter":42.0}},"model_memory_limit":"string","max_num_threads":42.0,"analyzed_fields":{"includes":["string"],"excludes":["string"]}}}'
Request examples 
{
  "config": {
    "source": {
      "index": "string",
      "runtime_mappings": {
        "additionalProperty1": {
          "fields": {
            "additionalProperty1": {
              "type": "boolean"
            },
            "additionalProperty2": {
              "type": "boolean"
            }
          },
          "fetch_fields": [
            {
              "field": "string",
              "format": "string"
            }
          ],
          "format": "string",
          "input_field": "string",
          "target_field": "string",
          "target_index": "string",
          "script": {
            "source": "string",
            "id": "string",
            "params": {
              "additionalProperty1": {},
              "additionalProperty2": {}
            },
            "": "painless",
            "options": {
              "additionalProperty1": "string",
              "additionalProperty2": "string"
            }
          },
          "type": "boolean"
        },
        "additionalProperty2": {
          "fields": {
            "additionalProperty1": {
              "type": "boolean"
            },
            "additionalProperty2": {
              "type": "boolean"
            }
          },
          "fetch_fields": [
            {
              "field": "string",
              "format": "string"
            }
          ],
          "format": "string",
          "input_field": "string",
          "target_field": "string",
          "target_index": "string",
          "script": {
            "source": "string",
            "id": "string",
            "params": {
              "additionalProperty1": {},
              "additionalProperty2": {}
            },
            "": "painless",
            "options": {
              "additionalProperty1": "string",
              "additionalProperty2": "string"
            }
          },
          "type": "boolean"
        }
      },
      "_source": {
        "includes": [
          "string"
        ],
        "excludes": [
          "string"
        ]
      },
      "query": {}
    },
    "analysis": {
      "classification": {
        "alpha": 42.0,
        "dependent_variable": "string",
        "downsample_factor": 42.0,
        "early_stopping_enabled": true,
        "eta": 42.0,
        "eta_growth_rate_per_tree": 42.0,
        "feature_bag_fraction": 42.0,
        "feature_processors": [
          {
            "frequency_encoding": {},
            "multi_encoding": {},
            "n_gram_encoding": {},
            "one_hot_encoding": {},
            "target_mean_encoding": {}
          }
        ],
        "gamma": 42.0,
        "lambda": 42.0,
        "max_optimization_rounds_per_hyperparameter": 42.0,
        "max_trees": 42.0,
        "num_top_feature_importance_values": 42.0,
        "prediction_field_name": "string",
        "randomize_seed": 42.0,
        "soft_tree_depth_limit": 42.0,
        "soft_tree_depth_tolerance": 42.0,
        "": "string",
        "class_assignment_objective": "string",
        "num_top_classes": 42.0
      },
      "outlier_detection": {
        "compute_feature_influence": true,
        "feature_influence_threshold": 42.0,
        "method": "string",
        "n_neighbors": 42.0,
        "outlier_fraction": 42.0,
        "standardization_enabled": true
      },
      "regression": {
        "alpha": 42.0,
        "dependent_variable": "string",
        "downsample_factor": 42.0,
        "early_stopping_enabled": true,
        "eta": 42.0,
        "eta_growth_rate_per_tree": 42.0,
        "feature_bag_fraction": 42.0,
        "feature_processors": [
          {
            "frequency_encoding": {},
            "multi_encoding": {},
            "n_gram_encoding": {},
            "one_hot_encoding": {},
            "target_mean_encoding": {}
          }
        ],
        "gamma": 42.0,
        "lambda": 42.0,
        "max_optimization_rounds_per_hyperparameter": 42.0,
        "max_trees": 42.0,
        "num_top_feature_importance_values": 42.0,
        "prediction_field_name": "string",
        "randomize_seed": 42.0,
        "soft_tree_depth_limit": 42.0,
        "soft_tree_depth_tolerance": 42.0,
        "": "string",
        "loss_function": "string",
        "loss_function_parameter": 42.0
      }
    },
    "model_memory_limit": "string",
    "max_num_threads": 42.0,
    "analyzed_fields": {
      "includes": [
        "string"
      ],
      "excludes": [
        "string"
      ]
    }
  }
}
Response examples (200) 
{
  "feature_values": [
    {
      "additionalProperty1": "string",
      "additionalProperty2": "string"
    }
  ]
}

Create a trained model Added in 7.10.0

PUT /_ml/trained_models/{model_id}
Api key auth Basic auth Bearer auth

Enable you to supply a trained model that is not created by data frame analytics.

Path parameters

  • model_id string Required

    The unique identifier of the trained model.

Query parameters

  • defer_definition_decompression boolean

    If set to true and a compressed_definition is provided, the request defers definition decompression and skips relevant validations.

  • wait_for_completion boolean

    Whether to wait for all child operations (e.g. model download) to complete.

application/json

Body Required

  • compressed_definition string

    The compressed (GZipped and Base64 encoded) inference definition of the model. If compressed_definition is specified, then definition cannot be specified.

  • definition object
    Hide definition attributes Show definition attributes object
  • description string

    A human-readable description of the inference trained model.

  • inference_config object

    Inference configuration provided when storing the model config

    Hide inference_config attributes Show inference_config attributes object
    • regression object
      Hide regression attributes Show regression attributes object
    • classification object
      Hide classification attributes Show classification attributes object
      • num_top_classes number

        Specifies the number of top class predictions to return. Defaults to 0.

      • num_top_feature_importance_values number

        Specifies the maximum number of feature importance values per document.

      • prediction_field_type string

        Specifies the type of the predicted field to write. Acceptable values are: string, number, boolean. When boolean is provided 1.0 is transformed to true and 0.0 to false.

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • top_classes_results_field string

        Specifies the field to which the top classes are written. Defaults to top_classes.

    • text_classification object
      Hide text_classification attributes Show text_classification attributes object
      • num_top_classes number

        Specifies the number of top class predictions to return. Defaults to 0.

      • tokenization object

        Tokenization options stored in inference configuration

        Hide tokenization attributes Show tokenization attributes object
        • bert object
          Hide bert attributes Show bert attributes object
        • bert_ja object
          Hide bert_ja attributes Show bert_ja attributes object
        • mpnet object
          Hide mpnet attributes Show mpnet attributes object
        • roberta object
          Hide roberta attributes Show roberta attributes object
        • xlm_roberta object
          Hide xlm_roberta attributes Show xlm_roberta attributes object
      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • classification_labels array[string]

        Classification labels to apply other than the stored labels. Must have the same deminsions as the default configured labels

      • vocabulary object
        Hide vocabulary attribute Show vocabulary attribute object
    • zero_shot_classification object
      Hide zero_shot_classification attributes Show zero_shot_classification attributes object
      • tokenization object

        Tokenization options stored in inference configuration

        Hide tokenization attributes Show tokenization attributes object
        • bert object
          Hide bert attributes Show bert attributes object
        • bert_ja object
          Hide bert_ja attributes Show bert_ja attributes object
        • mpnet object
          Hide mpnet attributes Show mpnet attributes object
        • roberta object
          Hide roberta attributes Show roberta attributes object
        • xlm_roberta object
          Hide xlm_roberta attributes Show xlm_roberta attributes object
      • hypothesis_template string

        Hypothesis template used when tokenizing labels for prediction

      • classification_labels array[string] Required

        The zero shot classification labels indicating entailment, neutral, and contradiction Must contain exactly and only entailment, neutral, and contradiction

      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • multi_label boolean

        Indicates if more than one true label exists.

      • labels array[string]

        The labels to predict.

    • fill_mask object
      Hide fill_mask attributes Show fill_mask attributes object
      • mask_token string

        The string/token which will be removed from incoming documents and replaced with the inference prediction(s). In a response, this field contains the mask token for the specified model/tokenizer. Each model and tokenizer has a predefined mask token which cannot be changed. Thus, it is recommended not to set this value in requests. However, if this field is present in a request, its value must match the predefined value for that model/tokenizer, otherwise the request will fail.

      • num_top_classes number

        Specifies the number of top class predictions to return. Defaults to 0.

      • tokenization object

        Tokenization options stored in inference configuration

        Hide tokenization attributes Show tokenization attributes object
        • bert object
          Hide bert attributes Show bert attributes object
        • bert_ja object
          Hide bert_ja attributes Show bert_ja attributes object
        • mpnet object
          Hide mpnet attributes Show mpnet attributes object
        • roberta object
          Hide roberta attributes Show roberta attributes object
        • xlm_roberta object
          Hide xlm_roberta attributes Show xlm_roberta attributes object
      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • vocabulary object Required
        Hide vocabulary attribute Show vocabulary attribute object
    • learning_to_rank object
      Hide learning_to_rank attributes Show learning_to_rank attributes object
    • ner object
      Hide ner attributes Show ner attributes object
      • tokenization object

        Tokenization options stored in inference configuration

        Hide tokenization attributes Show tokenization attributes object
        • bert object
          Hide bert attributes Show bert attributes object
        • bert_ja object
          Hide bert_ja attributes Show bert_ja attributes object
        • mpnet object
          Hide mpnet attributes Show mpnet attributes object
        • roberta object
          Hide roberta attributes Show roberta attributes object
        • xlm_roberta object
          Hide xlm_roberta attributes Show xlm_roberta attributes object
      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • classification_labels array[string]

        The token classification labels. Must be IOB formatted tags

      • vocabulary object
        Hide vocabulary attribute Show vocabulary attribute object
    • pass_through object
      Hide pass_through attributes Show pass_through attributes object
      • tokenization object

        Tokenization options stored in inference configuration

        Hide tokenization attributes Show tokenization attributes object
        • bert object
          Hide bert attributes Show bert attributes object
        • bert_ja object
          Hide bert_ja attributes Show bert_ja attributes object
        • mpnet object
          Hide mpnet attributes Show mpnet attributes object
        • roberta object
          Hide roberta attributes Show roberta attributes object
        • xlm_roberta object
          Hide xlm_roberta attributes Show xlm_roberta attributes object
      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • vocabulary object
        Hide vocabulary attribute Show vocabulary attribute object
    • text_embedding object
      Hide text_embedding attributes Show text_embedding attributes object
      • embedding_size number

        The number of dimensions in the embedding output

      • tokenization object

        Tokenization options stored in inference configuration

        Hide tokenization attributes Show tokenization attributes object
        • bert object
          Hide bert attributes Show bert attributes object
        • bert_ja object
          Hide bert_ja attributes Show bert_ja attributes object
        • mpnet object
          Hide mpnet attributes Show mpnet attributes object
        • roberta object
          Hide roberta attributes Show roberta attributes object
        • xlm_roberta object
          Hide xlm_roberta attributes Show xlm_roberta attributes object
      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • vocabulary object Required
        Hide vocabulary attribute Show vocabulary attribute object
    • text_expansion object
      Hide text_expansion attributes Show text_expansion attributes object
      • tokenization object

        Tokenization options stored in inference configuration

        Hide tokenization attributes Show tokenization attributes object
        • bert object
          Hide bert attributes Show bert attributes object
        • bert_ja object
          Hide bert_ja attributes Show bert_ja attributes object
        • mpnet object
          Hide mpnet attributes Show mpnet attributes object
        • roberta object
          Hide roberta attributes Show roberta attributes object
        • xlm_roberta object
          Hide xlm_roberta attributes Show xlm_roberta attributes object
      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • vocabulary object Required
        Hide vocabulary attribute Show vocabulary attribute object
    • question_answering object
      Hide question_answering attributes Show question_answering attributes object
      • num_top_classes number

        Specifies the number of top class predictions to return. Defaults to 0.

      • tokenization object

        Tokenization options stored in inference configuration

        Hide tokenization attributes Show tokenization attributes object
        • bert object
          Hide bert attributes Show bert attributes object
        • bert_ja object
          Hide bert_ja attributes Show bert_ja attributes object
        • mpnet object
          Hide mpnet attributes Show mpnet attributes object
        • roberta object
          Hide roberta attributes Show roberta attributes object
        • xlm_roberta object
          Hide xlm_roberta attributes Show xlm_roberta attributes object
      • results_field string

        The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

      • max_answer_length number

        The maximum answer length to consider

  • input object
    Hide input attribute Show input attribute object
  • metadata object

    An object map that contains metadata about the model.

  • model_type string

    Values are tree_ensemble, lang_ident, or pytorch.

  • model_size_bytes number

    The estimated memory usage in bytes to keep the trained model in memory. This property is supported only if defer_definition_decompression is true or the model definition is not supplied.

  • platform_architecture string

    The platform architecture (if applicable) of the trained mode. If the model only works on one platform, because it is heavily optimized for a particular processor architecture and OS combination, then this field specifies which. The format of the string must match the platform identifiers used by Elasticsearch, so one of, linux-x86_64, linux-aarch64, darwin-x86_64, darwin-aarch64, or windows-x86_64. For portable models (those that work independent of processor architecture or OS features), leave this field unset.

  • tags array[string]

    An array of tags to organize the model.

  • prefix_strings object
    Hide prefix_strings attributes Show prefix_strings attributes object
    • ingest string

      String prepended to input at ingest

    • search string

      String prepended to input at search

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • model_id string Required
    • model_type string

      Values are tree_ensemble, lang_ident, or pytorch.

    • tags array[string] Required

      A comma delimited string of tags. A trained model can have many tags, or none.

    • version string
    • compressed_definition string
    • created_by string

      Information on the creator of the trained model.

    • create_time string | number

      A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

      One of:
      DateTime string UnitMillis number
    • default_field_map object

      Any field map described in the inference configuration takes precedence.

      Hide default_field_map attribute Show default_field_map attribute object
      • * string Additional properties
    • description string

      The free-text description of the trained model.

    • estimated_heap_memory_usage_bytes number

      The estimated heap usage in bytes to keep the trained model in memory.

    • estimated_operations number

      The estimated number of operations to use the trained model.

    • fully_defined boolean

      True if the full model definition is present.

    • inference_config object

      Inference configuration provided when storing the model config

      Hide inference_config attributes Show inference_config attributes object
      • regression object
        Hide regression attributes Show regression attributes object
      • classification object
        Hide classification attributes Show classification attributes object
        • num_top_classes number

          Specifies the number of top class predictions to return. Defaults to 0.

        • num_top_feature_importance_values number

          Specifies the maximum number of feature importance values per document.

        • prediction_field_type string

          Specifies the type of the predicted field to write. Acceptable values are: string, number, boolean. When boolean is provided 1.0 is transformed to true and 0.0 to false.

        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • top_classes_results_field string

          Specifies the field to which the top classes are written. Defaults to top_classes.

      • text_classification object
        Hide text_classification attributes Show text_classification attributes object
        • num_top_classes number

          Specifies the number of top class predictions to return. Defaults to 0.

        • tokenization object

          Tokenization options stored in inference configuration

          Hide tokenization attributes Show tokenization attributes object
          • bert object
            Hide bert attributes Show bert attributes object
          • bert_ja object
            Hide bert_ja attributes Show bert_ja attributes object
          • mpnet object
            Hide mpnet attributes Show mpnet attributes object
          • roberta object
            Hide roberta attributes Show roberta attributes object
          • xlm_roberta object
            Hide xlm_roberta attributes Show xlm_roberta attributes object
        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • classification_labels array[string]

          Classification labels to apply other than the stored labels. Must have the same deminsions as the default configured labels

        • vocabulary object
          Hide vocabulary attribute Show vocabulary attribute object
      • zero_shot_classification object
        Hide zero_shot_classification attributes Show zero_shot_classification attributes object
        • tokenization object

          Tokenization options stored in inference configuration

          Hide tokenization attributes Show tokenization attributes object
          • bert object
            Hide bert attributes Show bert attributes object
          • bert_ja object
            Hide bert_ja attributes Show bert_ja attributes object
          • mpnet object
            Hide mpnet attributes Show mpnet attributes object
          • roberta object
            Hide roberta attributes Show roberta attributes object
          • xlm_roberta object
            Hide xlm_roberta attributes Show xlm_roberta attributes object
        • hypothesis_template string

          Hypothesis template used when tokenizing labels for prediction

        • classification_labels array[string] Required

          The zero shot classification labels indicating entailment, neutral, and contradiction Must contain exactly and only entailment, neutral, and contradiction

        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • multi_label boolean

          Indicates if more than one true label exists.

        • labels array[string]

          The labels to predict.

      • fill_mask object
        Hide fill_mask attributes Show fill_mask attributes object
        • mask_token string

          The string/token which will be removed from incoming documents and replaced with the inference prediction(s). In a response, this field contains the mask token for the specified model/tokenizer. Each model and tokenizer has a predefined mask token which cannot be changed. Thus, it is recommended not to set this value in requests. However, if this field is present in a request, its value must match the predefined value for that model/tokenizer, otherwise the request will fail.

        • num_top_classes number

          Specifies the number of top class predictions to return. Defaults to 0.

        • tokenization object

          Tokenization options stored in inference configuration

          Hide tokenization attributes Show tokenization attributes object
          • bert object
            Hide bert attributes Show bert attributes object
          • bert_ja object
            Hide bert_ja attributes Show bert_ja attributes object
          • mpnet object
            Hide mpnet attributes Show mpnet attributes object
          • roberta object
            Hide roberta attributes Show roberta attributes object
          • xlm_roberta object
            Hide xlm_roberta attributes Show xlm_roberta attributes object
        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • vocabulary object Required
          Hide vocabulary attribute Show vocabulary attribute object
      • learning_to_rank object
        Hide learning_to_rank attributes Show learning_to_rank attributes object
      • ner object
        Hide ner attributes Show ner attributes object
        • tokenization object

          Tokenization options stored in inference configuration

          Hide tokenization attributes Show tokenization attributes object
          • bert object
            Hide bert attributes Show bert attributes object
          • bert_ja object
            Hide bert_ja attributes Show bert_ja attributes object
          • mpnet object
            Hide mpnet attributes Show mpnet attributes object
          • roberta object
            Hide roberta attributes Show roberta attributes object
          • xlm_roberta object
            Hide xlm_roberta attributes Show xlm_roberta attributes object
        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • classification_labels array[string]

          The token classification labels. Must be IOB formatted tags

        • vocabulary object
          Hide vocabulary attribute Show vocabulary attribute object
      • pass_through object
        Hide pass_through attributes Show pass_through attributes object
        • tokenization object

          Tokenization options stored in inference configuration

          Hide tokenization attributes Show tokenization attributes object
          • bert object
            Hide bert attributes Show bert attributes object
          • bert_ja object
            Hide bert_ja attributes Show bert_ja attributes object
          • mpnet object
            Hide mpnet attributes Show mpnet attributes object
          • roberta object
            Hide roberta attributes Show roberta attributes object
          • xlm_roberta object
            Hide xlm_roberta attributes Show xlm_roberta attributes object
        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • vocabulary object
          Hide vocabulary attribute Show vocabulary attribute object
      • text_embedding object
        Hide text_embedding attributes Show text_embedding attributes object
        • embedding_size number

          The number of dimensions in the embedding output

        • tokenization object

          Tokenization options stored in inference configuration

          Hide tokenization attributes Show tokenization attributes object
          • bert object
            Hide bert attributes Show bert attributes object
          • bert_ja object
            Hide bert_ja attributes Show bert_ja attributes object
          • mpnet object
            Hide mpnet attributes Show mpnet attributes object
          • roberta object
            Hide roberta attributes Show roberta attributes object
          • xlm_roberta object
            Hide xlm_roberta attributes Show xlm_roberta attributes object
        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • vocabulary object Required
          Hide vocabulary attribute Show vocabulary attribute object
      • text_expansion object
        Hide text_expansion attributes Show text_expansion attributes object
        • tokenization object

          Tokenization options stored in inference configuration

          Hide tokenization attributes Show tokenization attributes object
          • bert object
            Hide bert attributes Show bert attributes object
          • bert_ja object
            Hide bert_ja attributes Show bert_ja attributes object
          • mpnet object
            Hide mpnet attributes Show mpnet attributes object
          • roberta object
            Hide roberta attributes Show roberta attributes object
          • xlm_roberta object
            Hide xlm_roberta attributes Show xlm_roberta attributes object
        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • vocabulary object Required
          Hide vocabulary attribute Show vocabulary attribute object
      • question_answering object
        Hide question_answering attributes Show question_answering attributes object
        • num_top_classes number

          Specifies the number of top class predictions to return. Defaults to 0.

        • tokenization object

          Tokenization options stored in inference configuration

          Hide tokenization attributes Show tokenization attributes object
          • bert object
            Hide bert attributes Show bert attributes object
          • bert_ja object
            Hide bert_ja attributes Show bert_ja attributes object
          • mpnet object
            Hide mpnet attributes Show mpnet attributes object
          • roberta object
            Hide roberta attributes Show roberta attributes object
          • xlm_roberta object
            Hide xlm_roberta attributes Show xlm_roberta attributes object
        • results_field string

          The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

        • max_answer_length number

          The maximum answer length to consider

    • input object Required
      Hide input attribute Show input attribute object
      • field_names array[string] Required

        An array of input field names for the model.

    • license_level string

      The license level of the trained model.

    • metadata object
      Hide metadata attributes Show metadata attributes object
      • model_aliases array[string]
      • feature_importance_baseline object

        An object that contains the baseline for feature importance values. For regression analysis, it is a single value. For classification analysis, there is a value for each class.

        Hide feature_importance_baseline attribute Show feature_importance_baseline attribute object
        • * string Additional properties
      • hyperparameters array[object]

        List of the available hyperparameters optimized during the fine_parameter_tuning phase as well as specified by the user.

        Hide hyperparameters attributes Show hyperparameters attributes object
        • absolute_importance number

          A positive number showing how much the parameter influences the variation of the loss function. For hyperparameters with values that are not specified by the user but tuned during hyperparameter optimization.

        • name string Required
        • relative_importance number

          A number between 0 and 1 showing the proportion of influence on the variation of the loss function among all tuned hyperparameters. For hyperparameters with values that are not specified by the user but tuned during hyperparameter optimization.

        • supplied boolean Required

          Indicates if the hyperparameter is specified by the user (true) or optimized (false).

        • value number Required

          The value of the hyperparameter, either optimized or specified by the user.

      • total_feature_importance array[object]

        An array of the total feature importance for each feature used from the training data set. This array of objects is returned if data frame analytics trained the model and the request includes total_feature_importance in the include request parameter.

        Hide total_feature_importance attributes Show total_feature_importance attributes object
        • feature_name string Required
        • importance array[object] Required

          A collection of feature importance statistics related to the training data set for this particular feature.

          Hide importance attributes Show importance attributes object
          • mean_magnitude number Required

            The average magnitude of this feature across all the training data. This value is the average of the absolute values of the importance for this feature.

          • max number Required

            The maximum importance value across all the training data for this feature.

          • min number Required

            The minimum importance value across all the training data for this feature.

        • classes array[object] Required

          If the trained model is a classification model, feature importance statistics are gathered per target class value.

          Hide classes attributes Show classes attributes object
          • class_name string Required
          • importance array[object] Required

            A collection of feature importance statistics related to the training data set for this particular feature.

    • model_size_bytes number | string

      One of:
      ByteSize number ByteSize string
    • model_package object
      Hide model_package attributes Show model_package attributes object
    • location object
      Hide location attribute Show location attribute object
      • index object Required
        Hide index attribute Show index attribute object
    • platform_architecture string
    • prefix_strings object
      Hide prefix_strings attributes Show prefix_strings attributes object
      • ingest string

        String prepended to input at ingest

      • search string

        String prepended to input at search
PUT /_ml/trained_models/{model_id} 
curl \
 --request PUT 'http://api.example.com/_ml/trained_models/{model_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"compressed_definition":"string","definition":{"preprocessors":[{"frequency_encoding":{"field":"string","feature_name":"string","frequency_map":{"additionalProperty1":42.0,"additionalProperty2":42.0}},"one_hot_encoding":{"field":"string","hot_map":{"additionalProperty1":"string","additionalProperty2":"string"}},"target_mean_encoding":{"field":"string","feature_name":"string","target_map":{"additionalProperty1":42.0,"additionalProperty2":42.0},"default_value":42.0}}],"trained_model":{"tree":{"classification_labels":["string"],"feature_names":["string"],"target_type":"string","tree_structure":[{"decision_type":"string","default_left":true,"leaf_value":42.0,"left_child":42.0,"node_index":42.0,"right_child":42.0,"split_feature":42.0,"split_gain":42.0,"threshold":42.0}]},"tree_node":{"decision_type":"string","default_left":true,"leaf_value":42.0,"left_child":42.0,"node_index":42.0,"right_child":42.0,"split_feature":42.0,"split_gain":42.0,"threshold":42.0},"ensemble":{"aggregate_output":{"logistic_regression":{"weights":42.0},"weighted_sum":{"weights":42.0},"weighted_mode":{"weights":42.0},"exponent":{"weights":42.0}},"classification_labels":["string"],"feature_names":["string"],"target_type":"string","trained_models":[{}]}}},"description":"string","inference_config":{"regression":{"results_field":"string","num_top_feature_importance_values":42.0},"classification":{"num_top_classes":42.0,"num_top_feature_importance_values":42.0,"prediction_field_type":"string","results_field":"string","top_classes_results_field":"string"},"text_classification":{"num_top_classes":42.0,"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","classification_labels":["string"],"vocabulary":{"index":"string"}},"zero_shot_classification":{"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"hypothesis_template":"string","classification_labels":["string"],"results_field":"string","multi_label":true,"labels":["string"]},"fill_mask":{"mask_token":"string","num_top_classes":42.0,"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","vocabulary":{"index":"string"}},"learning_to_rank":{"default_params":{"additionalProperty1":{},"additionalProperty2":{}},"feature_extractors":[{}],"num_top_feature_importance_values":42.0},"ner":{"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","classification_labels":["string"],"vocabulary":{"index":"string"}},"pass_through":{"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","vocabulary":{"index":"string"}},"text_embedding":{"embedding_size":42.0,"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","vocabulary":{"index":"string"}},"text_expansion":{"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","vocabulary":{"index":"string"}},"question_answering":{"num_top_classes":42.0,"tokenization":{"bert":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"bert_ja":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"mpnet":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true},"roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true,"add_prefix_space":true},"xlm_roberta":{"do_lower_case":true,"max_sequence_length":42.0,"span":42.0,"truncate":"first","with_special_tokens":true}},"results_field":"string","max_answer_length":42.0}},"input":{"field_names":"string"},"metadata":{},"model_type":"tree_ensemble","model_size_bytes":42.0,"platform_architecture":"string","tags":["string"],"prefix_strings":{"ingest":"string","search":"string"}}'
Request examples 
{
  "compressed_definition": "string",
  "definition": {
    "preprocessors": [
      {
        "frequency_encoding": {
          "field": "string",
          "feature_name": "string",
          "frequency_map": {
            "additionalProperty1": 42.0,
            "additionalProperty2": 42.0
          }
        },
        "one_hot_encoding": {
          "field": "string",
          "hot_map": {
            "additionalProperty1": "string",
            "additionalProperty2": "string"
          }
        },
        "target_mean_encoding": {
          "field": "string",
          "feature_name": "string",
          "target_map": {
            "additionalProperty1": 42.0,
            "additionalProperty2": 42.0
          },
          "default_value": 42.0
        }
      }
    ],
    "trained_model": {
      "tree": {
        "classification_labels": [
          "string"
        ],
        "feature_names": [
          "string"
        ],
        "target_type": "string",
        "tree_structure": [
          {
            "decision_type": "string",
            "default_left": true,
            "leaf_value": 42.0,
            "left_child": 42.0,
            "node_index": 42.0,
            "right_child": 42.0,
            "split_feature": 42.0,
            "split_gain": 42.0,
            "threshold": 42.0
          }
        ]
      },
      "tree_node": {
        "decision_type": "string",
        "default_left": true,
        "leaf_value": 42.0,
        "left_child": 42.0,
        "node_index": 42.0,
        "right_child": 42.0,
        "split_feature": 42.0,
        "split_gain": 42.0,
        "threshold": 42.0
      },
      "ensemble": {
        "aggregate_output": {
          "logistic_regression": {
            "weights": 42.0
          },
          "weighted_sum": {
            "weights": 42.0
          },
          "weighted_mode": {
            "weights": 42.0
          },
          "exponent": {
            "weights": 42.0
          }
        },
        "classification_labels": [
          "string"
        ],
        "feature_names": [
          "string"
        ],
        "target_type": "string",
        "trained_models": [
          {}
        ]
      }
    }
  },
  "description": "string",
  "inference_config": {
    "regression": {
      "results_field": "string",
      "num_top_feature_importance_values": 42.0
    },
    "classification": {
      "num_top_classes": 42.0,
      "num_top_feature_importance_values": 42.0,
      "prediction_field_type": "string",
      "results_field": "string",
      "top_classes_results_field": "string"
    },
    "text_classification": {
      "num_top_classes": 42.0,
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "classification_labels": [
        "string"
      ],
      "vocabulary": {
        "index": "string"
      }
    },
    "zero_shot_classification": {
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "hypothesis_template": "string",
      "classification_labels": [
        "string"
      ],
      "results_field": "string",
      "multi_label": true,
      "labels": [
        "string"
      ]
    },
    "fill_mask": {
      "mask_token": "string",
      "num_top_classes": 42.0,
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "vocabulary": {
        "index": "string"
      }
    },
    "learning_to_rank": {
      "default_params": {
        "additionalProperty1": {},
        "additionalProperty2": {}
      },
      "feature_extractors": [
        {}
      ],
      "num_top_feature_importance_values": 42.0
    },
    "ner": {
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "classification_labels": [
        "string"
      ],
      "vocabulary": {
        "index": "string"
      }
    },
    "pass_through": {
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "vocabulary": {
        "index": "string"
      }
    },
    "text_embedding": {
      "embedding_size": 42.0,
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "vocabulary": {
        "index": "string"
      }
    },
    "text_expansion": {
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "vocabulary": {
        "index": "string"
      }
    },
    "question_answering": {
      "num_top_classes": 42.0,
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "max_answer_length": 42.0
    }
  },
  "input": {
    "field_names": "string"
  },
  "metadata": {},
  "model_type": "tree_ensemble",
  "model_size_bytes": 42.0,
  "platform_architecture": "string",
  "tags": [
    "string"
  ],
  "prefix_strings": {
    "ingest": "string",
    "search": "string"
  }
}
Response examples (200) 
{
  "model_id": "string",
  "model_type": "tree_ensemble",
  "tags": [
    "string"
  ],
  "version": "string",
  "compressed_definition": "string",
  "created_by": "string",
  "": 42.0,
  "default_field_map": {
    "additionalProperty1": "string",
    "additionalProperty2": "string"
  },
  "description": "string",
  "estimated_heap_memory_usage_bytes": 42.0,
  "estimated_operations": 42.0,
  "fully_defined": true,
  "inference_config": {
    "regression": {
      "results_field": "string",
      "num_top_feature_importance_values": 42.0
    },
    "classification": {
      "num_top_classes": 42.0,
      "num_top_feature_importance_values": 42.0,
      "prediction_field_type": "string",
      "results_field": "string",
      "top_classes_results_field": "string"
    },
    "text_classification": {
      "num_top_classes": 42.0,
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "classification_labels": [
        "string"
      ],
      "vocabulary": {
        "index": "string"
      }
    },
    "zero_shot_classification": {
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "hypothesis_template": "string",
      "classification_labels": [
        "string"
      ],
      "results_field": "string",
      "multi_label": true,
      "labels": [
        "string"
      ]
    },
    "fill_mask": {
      "mask_token": "string",
      "num_top_classes": 42.0,
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "vocabulary": {
        "index": "string"
      }
    },
    "learning_to_rank": {
      "default_params": {
        "additionalProperty1": {},
        "additionalProperty2": {}
      },
      "feature_extractors": [
        {}
      ],
      "num_top_feature_importance_values": 42.0
    },
    "ner": {
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "classification_labels": [
        "string"
      ],
      "vocabulary": {
        "index": "string"
      }
    },
    "pass_through": {
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "vocabulary": {
        "index": "string"
      }
    },
    "text_embedding": {
      "embedding_size": 42.0,
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "vocabulary": {
        "index": "string"
      }
    },
    "text_expansion": {
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "vocabulary": {
        "index": "string"
      }
    },
    "question_answering": {
      "num_top_classes": 42.0,
      "tokenization": {
        "bert": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "bert_ja": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "mpnet": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        },
        "roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true,
          "add_prefix_space": true
        },
        "xlm_roberta": {
          "do_lower_case": true,
          "max_sequence_length": 42.0,
          "span": 42.0,
          "truncate": "first",
          "with_special_tokens": true
        }
      },
      "results_field": "string",
      "max_answer_length": 42.0
    }
  },
  "input": {
    "field_names": [
      "string"
    ]
  },
  "license_level": "string",
  "metadata": {
    "model_aliases": [
      "string"
    ],
    "feature_importance_baseline": {
      "additionalProperty1": "string",
      "additionalProperty2": "string"
    },
    "hyperparameters": [
      {
        "absolute_importance": 42.0,
        "name": "string",
        "relative_importance": 42.0,
        "supplied": true,
        "value": 42.0
      }
    ],
    "total_feature_importance": [
      {
        "feature_name": "string",
        "importance": [
          {
            "mean_magnitude": 42.0,
            "max": 42.0,
            "min": 42.0
          }
        ],
        "classes": [
          {
            "class_name": "string",
            "importance": [
              {}
            ]
          }
        ]
      }
    ]
  },
  "model_package": {
    "": 42.0,
    "description": "string",
    "inference_config": {
      "additionalProperty1": {},
      "additionalProperty2": {}
    },
    "metadata": {
      "additionalProperty1": {},
      "additionalProperty2": {}
    },
    "minimum_version": "string",
    "model_repository": "string",
    "model_type": "string",
    "packaged_model_id": "string",
    "platform_architecture": "string",
    "prefix_strings": {
      "ingest": "string",
      "search": "string"
    },
    "sha256": "string",
    "tags": [
      "string"
    ],
    "vocabulary_file": "string"
  },
  "location": {
    "index": {
      "name": "string"
    }
  },
  "platform_architecture": "string",
  "prefix_strings": {
    "ingest": "string",
    "search": "string"
  }
}

Rollup

Run a scrolling search

GET /_search/scroll
Api key auth Basic auth Bearer auth

IMPORTANT: The scroll API is no longer recommend for deep pagination. If you need to preserve the index state while paging through more than 10,000 hits, use the search_after parameter with a point in time (PIT).

The scroll API gets large sets of results from a single scrolling search request. To get the necessary scroll ID, submit a search API request that includes an argument for the scroll query parameter. The scroll parameter indicates how long Elasticsearch should retain the search context for the request. The search response returns a scroll ID in the _scroll_id response body parameter. You can then use the scroll ID with the scroll API to retrieve the next batch of results for the request. If the Elasticsearch security features are enabled, the access to the results of a specific scroll ID is restricted to the user or API key that submitted the search.

You can also use the scroll API to specify a new scroll parameter that extends or shortens the retention period for the search context.

IMPORTANT: Results from a scrolling search reflect the state of the index at the time of the initial search request. Subsequent indexing or document changes only affect later search and scroll requests.

External documentation

Query parameters

  • scroll string

    The period to retain the search context for scrolling.

  • scroll_id string Deprecated

    The scroll ID for scrolled search

  • rest_total_hits_as_int boolean

    If true, the API response’s hit.total property is returned as an integer. If false, the API response’s hit.total property is returned as an object.

application/json

Body

  • scroll string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • scroll_id string Required

Responses

GET /_search/scroll 
curl \
 --request GET 'http://api.example.com/_search/scroll' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"scroll_id\" : \"DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ==\"\n}"'
Request example
Run `GET /_search/scroll` to get the next batch of results for a scrolling search. 
{
  "scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ=="
}
Response examples (200) 
{
  "took": 42.0,
  "timed_out": true,
  "_shards": {
    "failed": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "failures": [
      {
        "index": "string",
        "node": "string",
        "reason": {
          "type": "string",
          "reason": "string",
          "stack_trace": "string",
          "caused_by": {},
          "root_cause": [
            {}
          ],
          "suppressed": [
            {}
          ]
        },
        "shard": 42.0,
        "status": "string"
      }
    ],
    "skipped": 42.0
  },
  "hits": {
    "total": {
      "relation": "eq",
      "value": 42.0
    },
    "hits": [
      {
        "_index": "string",
        "_id": "string",
        "_score": 42.0,
        "_explanation": {
          "description": "string",
          "details": [
            {}
          ],
          "value": 42.0
        },
        "fields": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "highlight": {
          "additionalProperty1": [
            "string"
          ],
          "additionalProperty2": [
            "string"
          ]
        },
        "inner_hits": {
          "additionalProperty1": {
            "hits": {}
          },
          "additionalProperty2": {
            "hits": {}
          }
        },
        "matched_queries": [
          "string"
        ],
        "_nested": {
          "field": "string",
          "offset": 42.0,
          "_nested": {}
        },
        "_ignored": [
          "string"
        ],
        "ignored_field_values": {
          "additionalProperty1": [
            {}
          ],
          "additionalProperty2": [
            {}
          ]
        },
        "_shard": "string",
        "_node": "string",
        "_routing": "string",
        "_source": {},
        "_rank": 42.0,
        "_seq_no": 42.0,
        "_primary_term": 42.0,
        "_version": 42.0,
        "sort": [
          42.0
        ]
      }
    ],
    "max_score": 42.0
  },
  "aggregations": {},
  "_clusters": {
    "skipped": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "running": 42.0,
    "partial": 42.0,
    "failed": 42.0,
    "details": {
      "additionalProperty1": {
        "status": "running",
        "indices": "string",
        "": 42.0,
        "timed_out": true,
        "_shards": {
          "failed": 42.0,
          "successful": 42.0,
          "total": 42.0,
          "failures": [
            {}
          ],
          "skipped": 42.0
        },
        "failures": [
          {
            "index": "string",
            "node": "string",
            "reason": {},
            "shard": 42.0,
            "status": "string"
          }
        ]
      },
      "additionalProperty2": {
        "status": "running",
        "indices": "string",
        "": 42.0,
        "timed_out": true,
        "_shards": {
          "failed": 42.0,
          "successful": 42.0,
          "total": 42.0,
          "failures": [
            {}
          ],
          "skipped": 42.0
        },
        "failures": [
          {
            "index": "string",
            "node": "string",
            "reason": {},
            "shard": 42.0,
            "status": "string"
          }
        ]
      }
    }
  },
  "fields": {
    "additionalProperty1": {},
    "additionalProperty2": {}
  },
  "max_score": 42.0,
  "num_reduce_phases": 42.0,
  "profile": {
    "shards": [
      {
        "aggregations": [
          {
            "breakdown": {},
            "description": "string",
            "type": "string",
            "debug": {},
            "children": [
              {}
            ]
          }
        ],
        "cluster": "string",
        "dfs": {
          "statistics": {
            "type": "string",
            "description": "string",
            "time": "string",
            "breakdown": {},
            "debug": {},
            "children": [
              {}
            ]
          },
          "knn": [
            {}
          ]
        },
        "fetch": {
          "type": "string",
          "description": "string",
          "": 42.0,
          "breakdown": {
            "load_source": 42.0,
            "load_source_count": 42.0,
            "load_stored_fields": 42.0,
            "load_stored_fields_count": 42.0,
            "next_reader": 42.0,
            "next_reader_count": 42.0,
            "process_count": 42.0,
            "process": 42.0
          },
          "debug": {
            "stored_fields": [
              "string"
            ],
            "fast_path": 42.0
          },
          "children": [
            {}
          ]
        },
        "id": "string",
        "index": "string",
        "node_id": "string",
        "searches": [
          {
            "collector": [
              {}
            ],
            "query": [
              {}
            ],
            "rewrite_time": 42.0
          }
        ],
        "shard_id": 42.0
      }
    ]
  },
  "pit_id": "string",
  "_scroll_id": "string",
  "suggest": {
    "additionalProperty1": [
      {
        "length": 42.0,
        "offset": 42.0,
        "text": "string"
      }
    ],
    "additionalProperty2": [
      {
        "length": 42.0,
        "offset": 42.0,
        "text": "string"
      }
    ]
  },
  "terminated_early": true
}

Clear the cache Technical preview

POST /_searchable_snapshots/cache/clear
Api key auth Basic auth Bearer auth

Clear indices and data streams from the shared cache for partially mounted indices.

External documentation

Query parameters

  • expand_wildcards string | array[string]

    Whether to expand wildcard expression to concrete indices that are open, closed or both.

  • allow_no_indices boolean

    Whether to ignore if a wildcard indices expression resolves into no concrete indices. (This includes _all string or when no indices have been specified)

  • ignore_unavailable boolean

    Whether specified concrete indices should be ignored when unavailable (missing or closed)

Responses

POST /_searchable_snapshots/cache/clear 
curl \
 --request POST 'http://api.example.com/_searchable_snapshots/cache/clear' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{}

Clear the user cache

POST /_security/realm/{realms}/_clear_cache
Api key auth Basic auth Bearer auth

Evict users from the user cache. You can completely clear the cache or evict specific users.

User credentials are cached in memory on each node to avoid connecting to a remote authentication service or hitting the disk for every incoming request. There are realm settings that you can use to configure the user cache. For more information, refer to the documentation about controlling the user cache.

External documentation

Path parameters

  • realms string | array[string] Required

    A comma-separated list of realms. To clear all realms, use an asterisk (*). It does not support other wildcard patterns.

Query parameters

  • usernames array[string]

    A comma-separated list of the users to clear from the cache. If you do not specify this parameter, the API evicts all users from the user cache.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object Required
      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]
        Hide failures attributes Show failures attributes object
      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attribute Show * attribute object
POST /_security/realm/{realms}/_clear_cache 
curl \
 --request POST 'http://api.example.com/_security/realm/{realms}/_clear_cache' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "_nodes": {
    "failures": [
      {
        "type": "string",
        "reason": "string",
        "stack_trace": "string",
        "caused_by": {},
        "root_cause": [
          {}
        ],
        "suppressed": [
          {}
        ]
      }
    ],
    "total": 42.0,
    "successful": 42.0,
    "failed": 42.0
  },
  "cluster_name": "string",
  "nodes": {
    "additionalProperty1": {
      "name": "string"
    },
    "additionalProperty2": {
      "name": "string"
    }
  }
}

Create a cross-cluster API key

POST /_security/cross_cluster/api_key
Api key auth Basic auth Bearer auth

Create an API key of the cross_cluster type for the API key based remote cluster access. A cross_cluster API key cannot be used to authenticate through the REST interface.

IMPORTANT: To authenticate this request you must use a credential that is not an API key. Even if you use an API key that has the required privilege, the API returns an error.

Cross-cluster API keys are created by the Elasticsearch API key service, which is automatically enabled.

NOTE: Unlike REST API keys, a cross-cluster API key does not capture permissions of the authenticated user. The API key’s effective permission is exactly as specified with the access property.

A successful request returns a JSON structure that contains the API key, its unique ID, and its name. If applicable, it also returns expiration information for the API key in milliseconds.

By default, API keys never expire. You can specify expiration information when you create the API keys.

Cross-cluster API keys can only be updated with the update cross-cluster API key API. Attempting to update them with the update REST API key API or the bulk update REST API keys API will result in an error.

External documentation
application/json

Body Required

  • access object Required
    Hide access attributes Show access attributes object
    • replication array[object]

      A list of indices permission entries for cross-cluster replication.

      Hide replication attributes Show replication attributes object
    • search array[object]

      A list of indices permission entries for cross-cluster search.

      Hide search attributes Show search attributes object
  • expiration string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • metadata object
    Hide metadata attribute Show metadata attribute object
    • * object Additional properties
  • name string Required

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • api_key string Required

      Generated API key.

    • expiration number

      Time unit for milliseconds

    • id string Required
    • name string Required
    • encoded string Required

      API key credentials which is the base64-encoding of the UTF-8 representation of id and api_key joined by a colon (:).
POST /_security/cross_cluster/api_key 
curl \
 --request POST 'http://api.example.com/_security/cross_cluster/api_key' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"name\": \"my-cross-cluster-api-key\",\n  \"expiration\": \"1d\",   \n  \"access\": {\n    \"search\": [  \n      {\n        \"names\": [\"logs*\"]\n      }\n    ],\n    \"replication\": [  \n      {\n        \"names\": [\"archive*\"]\n      }\n    ]\n  },\n  \"metadata\": {\n    \"description\": \"phase one\",\n    \"environment\": {\n      \"level\": 1,\n      \"trusted\": true,\n      \"tags\": [\"dev\", \"staging\"]\n    }\n  }\n}"'
Request example
Run `POST /_security/cross_cluster/api_key` to create a cross-cluster API key. 
{
  "name": "my-cross-cluster-api-key",
  "expiration": "1d",   
  "access": {
    "search": [  
      {
        "names": ["logs*"]
      }
    ],
    "replication": [  
      {
        "names": ["archive*"]
      }
    ]
  },
  "metadata": {
    "description": "phase one",
    "environment": {
      "level": 1,
      "trusted": true,
      "tags": ["dev", "staging"]
    }
  }
}
Response examples (200)
A successful response from `POST /_security/service/elastic/fleet-server/credential/token`. 
{
  "created": true,
  "token": {
    "name": "Jk5J1HgBuyBK5TpDrdo4",
    "value": "AAEAAWVsYXN0aWM...vZmxlZXQtc2VydmVyL3Rva2VuMTo3TFdaSDZ"
  }
}

Delete users

DELETE /_security/user/{username}
Api key auth Basic auth Bearer auth

Delete users from the native realm.

Path parameters

  • username string Required

    An identifier for the user.

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • found boolean Required

      If the user is successfully deleted, the request returns {"found": true}. Otherwise, found is set to false.
DELETE /_security/user/{username} 
curl \
 --request DELETE 'http://api.example.com/_security/user/{username}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `DELETE /_security/user/jacknich`. 
{
  "found" : true
}

Create or update application privileges Added in 6.4.0

PUT /_security/privilege
Api key auth Basic auth Bearer auth

To use this API, you must have one of the following privileges:

  • The manage_security cluster privilege (or a greater privilege such as all).
  • The "Manage Application Privileges" global privilege for the application being referenced in the request.

Application names are formed from a prefix, with an optional suffix that conform to the following rules:

  • The prefix must begin with a lowercase ASCII letter.
  • The prefix must contain only ASCII letters or digits.
  • The prefix must be at least 3 characters long.
  • If the suffix exists, it must begin with either a dash - or _.
  • The suffix cannot contain any of the following characters: \, /, *, ?, ", <, >, |, ,, *.
  • No part of the name can contain whitespace.

Privilege names must begin with a lowercase ASCII letter and must contain only ASCII letters and digits along with the characters _, -, and ..

Action names can contain any number of printable ASCII characters and must contain at least one of the following characters: /, *, :.

External documentation

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

application/json

Body Required

  • * object Additional properties
    Hide * attribute Show * attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • * object Additional properties
      Hide * attribute Show * attribute object
      • * object
        Hide * attribute Show * attribute object
PUT /_security/privilege 
curl \
 --request PUT 'http://api.example.com/_security/privilege' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"myapp\": {\n    \"read\": {\n      \"actions\": [ \n        \"data:read/*\" , \n        \"action:login\" ],\n        \"metadata\": { \n          \"description\": \"Read access to myapp\"\n        }\n      }\n    }\n}"'
Request examples
Run `PUT /_security/privilege` to add a single application privilege. The wildcard (`*`) means that this privilege grants access to all actions that start with `data:read/`. Elasticsearch does not assign any meaning to these actions. However, if the request includes an application privilege such as `data:read/users` or `data:read/settings`, the has privileges API respects the use of a wildcard and returns `true`. 
{
  "myapp": {
    "read": {
      "actions": [ 
        "data:read/*" , 
        "action:login" ],
        "metadata": { 
          "description": "Read access to myapp"
        }
      }
    }
}
Run `PUT /_security/privilege` to add multiple application privileges. 
{
  "app01": {
    "read": {
      "actions": [ "action:login", "data:read/*" ]
    },
    "write": {
      "actions": [ "action:login", "data:write/*" ]
    }
  },
  "app02": {
    "all": {
      "actions": [ "*" ]
    }
  }
}
Response examples (200)
A successful response from `PUT /_security/privilege`. 
{
  "myapp": {
    "read": {
      "created": true 
    }
  }
}
A successful response from `PUT /_security/privilege`. The `created` property indicates whether the privileges have been created or updated. 
{
  "app02": {
    "all": {
      "created": true
    }
  },
  "app01": {
    "read": {
      "created": true
    },
    "write": {
      "created": true
    }
  }
}

Get service account credentials Added in 7.13.0

GET /_security/service/{namespace}/{service}/credential
Api key auth Basic auth Bearer auth

To use this API, you must have at least the read_security cluster privilege (or a greater privilege such as manage_service_account or manage_security).

The response includes service account tokens that were created with the create service account tokens API as well as file-backed tokens from all nodes of the cluster.

NOTE: For tokens backed by the service_tokens file, the API collects them from all nodes of the cluster. Tokens with the same name from different nodes are assumed to be the same token and are only counted once towards the total number of service tokens.

External documentation

Path parameters

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • service_account string Required
    • count number Required
    • tokens object Required
      Hide tokens attribute Show tokens attribute object
      • * object
        Hide * attribute Show * attribute object
        • * object Additional properties
    • nodes_credentials object Required
      Hide nodes_credentials attributes Show nodes_credentials attributes object
      • _nodes object Required
        Hide _nodes attributes Show _nodes attributes object
        • failures array[object]
          Hide failures attributes Show failures attributes object
        • total number Required

          Total number of nodes selected by the request.

        • successful number Required

          Number of nodes that responded successfully to the request.

        • failed number Required

          Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

      • file_tokens object Required

        File-backed tokens collected from all nodes

        Hide file_tokens attribute Show file_tokens attribute object
        • * object Additional properties
          Hide * attribute Show * attribute object
GET /_security/service/{namespace}/{service}/credential 
curl \
 --request GET 'http://api.example.com/_security/service/{namespace}/{service}/credential' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_security/service/elastic/fleet-server/credential`. The response contains all credentials for the `elastic/fleet-server` service account. 
{
  "service_account": "elastic/fleet-server",
  "count": 3,
  "tokens": {
    "token1": {},        
    "token42": {}        
  },
  "nodes_credentials": { 
    "_nodes": {          
      "total": 3,
      "successful": 3,
      "failed": 0
    },
    "file_tokens": {      
      "my-token": {
        "nodes": [ "node0", "node1" ] 
      }
    }
  }
}

Check user profile privileges Added in 8.3.0

GET /_security/profile/_has_privileges
Api key auth Basic auth Bearer auth

Determine whether the users associated with the specified user profile IDs have all the requested privileges.

NOTE: The user profile feature is designed only for use by Kibana and Elastic's Observability, Enterprise Search, and Elastic Security solutions. Individual users and external applications should not call this API directly. Elastic reserves the right to change or remove this feature in future releases without prior notice.

External documentation
application/json

Body Required

  • uids array[string] Required

    A list of profile IDs. The privileges are checked for associated users of the profiles.

  • privileges object Required
    Hide privileges attributes Show privileges attributes object
    • application array[object]
      Hide application attributes Show application attributes object
      • application string Required

        The name of the application.

      • privileges array[string] Required

        A list of the privileges that you want to check for the specified resources. It may be either application privilege names or the names of actions that are granted by those privileges

      • resources array[string] Required

        A list of resource names against which the privileges should be checked.

    • cluster array[string]

      A list of the cluster privileges that you want to check.

    • index array[object]
      Hide index attributes Show index attributes object
      • names string | array[string] Required
      • privileges array[string] Required

        A list of the privileges that you want to check for the specified indices.

      • allow_restricted_indices boolean

        This needs to be set to true (default is false) if using wildcards or regexps for patterns that cover restricted indices. Implicitly, restricted indices do not match index patterns because restricted indices usually have limited privileges and including them in pattern tests would render most such tests false. If restricted indices are explicitly included in the names list, privileges will be checked against them regardless of the value of allow_restricted_indices.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • has_privilege_uids array[string] Required

      The subset of the requested profile IDs of the users that have all the requested privileges.

    • errors object
      Hide errors attributes Show errors attributes object
      • count number Required
      • details object Required
        Hide details attribute Show details attribute object
        • * object
          Hide * attributes Show * attributes object
GET /_security/profile/_has_privileges 
curl \
 --request GET 'http://api.example.com/_security/profile/_has_privileges' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"uids\": [\n    \"u_LQPnxDxEjIH0GOUoFkZr5Y57YUwSkL9Joiq-g4OCbPc_0\",\n    \"u_rzRnxDgEHIH0GOUoFkZr5Y27YUwSk19Joiq=g4OCxxB_1\",\n    \"u_does-not-exist_0\"\n  ],\n  \"privileges\": {\n    \"cluster\": [ \"monitor\", \"create_snapshot\", \"manage_ml\" ],\n    \"index\" : [\n      {\n        \"names\": [ \"suppliers\", \"products\" ],\n        \"privileges\": [ \"create_doc\"]\n      },\n      {\n        \"names\": [ \"inventory\" ],\n        \"privileges\" : [ \"read\", \"write\" ]\n      }\n    ],\n    \"application\": [\n      {\n        \"application\": \"inventory_manager\",\n        \"privileges\" : [ \"read\", \"data:write/inventory\" ],\n        \"resources\" : [ \"product/1852563\" ]\n      }\n    ]\n  }\n}"'
Request example
Run `POST /_security/profile/_has_privileges` to check whether the two users associated with the specified profiles have all the requested set of cluster, index, and application privileges. 
{
  "uids": [
    "u_LQPnxDxEjIH0GOUoFkZr5Y57YUwSkL9Joiq-g4OCbPc_0",
    "u_rzRnxDgEHIH0GOUoFkZr5Y27YUwSk19Joiq=g4OCxxB_1",
    "u_does-not-exist_0"
  ],
  "privileges": {
    "cluster": [ "monitor", "create_snapshot", "manage_ml" ],
    "index" : [
      {
        "names": [ "suppliers", "products" ],
        "privileges": [ "create_doc"]
      },
      {
        "names": [ "inventory" ],
        "privileges" : [ "read", "write" ]
      }
    ],
    "application": [
      {
        "application": "inventory_manager",
        "privileges" : [ "read", "data:write/inventory" ],
        "resources" : [ "product/1852563" ]
      }
    ]
  }
}
Response examples (200)
A response from `POST /_security/profile/_has_privileges` that indicates only one of the three users has all the privileges and one of them is not found. 
{
  "has_privilege_uids": ["u_rzRnxDgEHIH0GOUoFkZr5Y27YUwSk19Joiq=g4OCxxB_1"],
  "errors": {
    "count": 1,
    "details": {
      "u_does-not-exist_0": {
        "type": "resource_not_found_exception",
        "reason": "profile document not found"
      }
    }
  }
}

Invalidate SAML Added in 7.5.0

POST /_security/saml/invalidate
Api key auth Basic auth Bearer auth

Submit a SAML LogoutRequest message to Elasticsearch for consumption.

NOTE: This API is intended for use by custom web applications other than Kibana. If you are using Kibana, refer to the documentation for configuring SAML single-sign-on on the Elastic Stack.

The logout request comes from the SAML IdP during an IdP initiated Single Logout. The custom web application can use this API to have Elasticsearch process the LogoutRequest. After successful validation of the request, Elasticsearch invalidates the access token and refresh token that corresponds to that specific SAML principal and provides a URL that contains a SAML LogoutResponse message. Thus the user can be redirected back to their IdP.

External documentation
application/json

Body Required

  • acs string

    The Assertion Consumer Service URL that matches the one of the SAML realm in Elasticsearch that should be used. You must specify either this parameter or the realm parameter.

  • query_string string Required

    The query part of the URL that the user was redirected to by the SAML IdP to initiate the Single Logout. This query should include a single parameter named SAMLRequest that contains a SAML logout request that is deflated and Base64 encoded. If the SAML IdP has signed the logout request, the URL should include two extra parameters named SigAlg and Signature that contain the algorithm used for the signature and the signature value itself. In order for Elasticsearch to be able to verify the IdP's signature, the value of the query_string field must be an exact match to the string provided by the browser. The client application must not attempt to parse or process the string in any way.

  • realm string

    The name of the SAML realm in Elasticsearch the configuration. You must specify either this parameter or the acs parameter.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • invalidated number Required

      The number of tokens that were invalidated as part of this logout.

    • realm string Required

      The realm name of the SAML realm in Elasticsearch that authenticated the user.

    • redirect string Required

      A SAML logout response as a parameter so that the user can be redirected back to the SAML IdP.
POST /_security/saml/invalidate 
curl \
 --request POST 'http://api.example.com/_security/saml/invalidate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query_string\" : \"SAMLRequest=nZFda4MwFIb%2FiuS%2BmviRpqFaClKQdbvo2g12M2KMraCJ9cRR9utnW4Wyi13sMie873MeznJ1aWrnS3VQGR0j4mLkKC1NUeljjA77zYyhVbIE0dR%2By7fmaHq7U%2BdegXWGpAZ%2B%2F4pR32luBFTAtWgUcCv56%2Fp5y30X87Yz1khTIycdgpUW9kY7WdsC9zxoXTvMvWuVV98YyMnSGH2SYE5pwALBIr9QKiwDGpW0oGVUznGeMyJZKFkQ4jBf5HnhUymjIhzCAL3KNFihbYx8TBYzzGaY7EnIyZwHzCWMfiDnbRIftkSjJr%2BFu0e9v%2B0EgOquRiiZjKpiVFp6j50T4WXoyNJ%2FEWC9fdqc1t%2F1%2B2F3aUpjzhPiXpqMz1%2FHSn4A\u0026SigAlg=http%3A%2F%2Fwww.w3.org%2F2001%2F04%2Fxmldsig-more%23rsa-sha256\u0026Signature=MsAYz2NFdovMG2mXf6TSpu5vlQQyEJAg%2B4KCwBqJTmrb3yGXKUtIgvjqf88eCAK32v3eN8vupjPC8LglYmke1ZnjK0%2FKxzkvSjTVA7mMQe2AQdKbkyC038zzRq%2FYHcjFDE%2Bz0qISwSHZY2NyLePmwU7SexEXnIz37jKC6NMEhus%3D\",\n  \"realm\" : \"saml1\"\n}"'
Request example
Run `POST /_security/saml/invalidate` to invalidate all the tokens for realm `saml1` pertaining to the user that is identified in the SAML Logout Request. 
{
  "query_string" : "SAMLRequest=nZFda4MwFIb%2FiuS%2BmviRpqFaClKQdbvo2g12M2KMraCJ9cRR9utnW4Wyi13sMie873MeznJ1aWrnS3VQGR0j4mLkKC1NUeljjA77zYyhVbIE0dR%2By7fmaHq7U%2BdegXWGpAZ%2B%2F4pR32luBFTAtWgUcCv56%2Fp5y30X87Yz1khTIycdgpUW9kY7WdsC9zxoXTvMvWuVV98YyMnSGH2SYE5pwALBIr9QKiwDGpW0oGVUznGeMyJZKFkQ4jBf5HnhUymjIhzCAL3KNFihbYx8TBYzzGaY7EnIyZwHzCWMfiDnbRIftkSjJr%2BFu0e9v%2B0EgOquRiiZjKpiVFp6j50T4WXoyNJ%2FEWC9fdqc1t%2F1%2B2F3aUpjzhPiXpqMz1%2FHSn4A&SigAlg=http%3A%2F%2Fwww.w3.org%2F2001%2F04%2Fxmldsig-more%23rsa-sha256&Signature=MsAYz2NFdovMG2mXf6TSpu5vlQQyEJAg%2B4KCwBqJTmrb3yGXKUtIgvjqf88eCAK32v3eN8vupjPC8LglYmke1ZnjK0%2FKxzkvSjTVA7mMQe2AQdKbkyC038zzRq%2FYHcjFDE%2Bz0qISwSHZY2NyLePmwU7SexEXnIz37jKC6NMEhus%3D",
  "realm" : "saml1"
}
Response examples (200)
A successful response from `POST /_security/saml/invalidate`. 
{
  "redirect" : "https://my-idp.org/logout/SAMLResponse=....",
  "invalidated" : 2,
  "realm" : "saml1"
}

Logout of SAML Added in 7.5.0

POST /_security/saml/logout
Api key auth Basic auth Bearer auth

Submits a request to invalidate an access token and refresh token.

NOTE: This API is intended for use by custom web applications other than Kibana. If you are using Kibana, refer to the documentation for configuring SAML single-sign-on on the Elastic Stack.

This API invalidates the tokens that were generated for a user by the SAML authenticate API. If the SAML realm in Elasticsearch is configured accordingly and the SAML IdP supports this, the Elasticsearch response contains a URL to redirect the user to the IdP that contains a SAML logout request (starting an SP-initiated SAML Single Logout).

External documentation
application/json

Body Required

  • token string Required

    The access token that was returned as a response to calling the SAML authenticate API. Alternatively, the most recent token that was received after refreshing the original one by using a refresh_token.

  • refresh_token string

    The refresh token that was returned as a response to calling the SAML authenticate API. Alternatively, the most recent refresh token that was received after refreshing the original access token.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • redirect string Required

      A URL that contains a SAML logout request as a parameter. You can use this URL to be redirected back to the SAML IdP and to initiate Single Logout.
POST /_security/saml/logout 
curl \
 --request POST 'http://api.example.com/_security/saml/logout' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"token\" : \"46ToAxZVaXVVZTVKOVF5YU04ZFJVUDVSZlV3\",\n  \"refresh_token\" : \"mJdXLtmvTUSpoLwMvdBt_w\"\n}"'
Request example
Run `POST /_security/saml/logout` to invalidate the pair of tokens that were generated by calling the SAML authenticate API with a successful SAML response. 
{
  "token" : "46ToAxZVaXVVZTVKOVF5YU04ZFJVUDVSZlV3",
  "refresh_token" : "mJdXLtmvTUSpoLwMvdBt_w"
}
Response examples (200)
A successful response from `POST /_security/saml/logout`. 
{
  "redirect" : "https://my-idp.org/logout/SAMLRequest=...."
}

Update a cross-cluster API key

PUT /_security/cross_cluster/api_key/{id}
Api key auth Basic auth Bearer auth

Update the attributes of an existing cross-cluster API key, which is used for API key based remote cluster access.

To use this API, you must have at least the manage_security cluster privilege. Users can only update API keys that they created. To update another user's API key, use the run_as feature to submit a request on behalf of another user.

IMPORTANT: It's not possible to use an API key as the authentication credential for this API. To update an API key, the owner user's credentials are required.

It's not possible to update expired API keys, or API keys that have been invalidated by the invalidate API key API.

This API supports updates to an API key's access scope, metadata, and expiration. The owner user's information, such as the username and realm, is also updated automatically on every call.

NOTE: This API cannot update REST API keys, which should be updated by either the update API key or bulk update API keys API.

External documentation

Path parameters

  • id string Required

    The ID of the cross-cluster API key to update.

application/json

Body Required

  • access object Required
    Hide access attributes Show access attributes object
    • replication array[object]

      A list of indices permission entries for cross-cluster replication.

      Hide replication attributes Show replication attributes object
    • search array[object]

      A list of indices permission entries for cross-cluster search.

      Hide search attributes Show search attributes object
  • expiration string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • metadata object
    Hide metadata attribute Show metadata attribute object
    • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • updated boolean Required

      If true, the API key was updated. If false, the API key didn’t change because no change was detected.
PUT /_security/cross_cluster/api_key/{id} 
curl \
 --request PUT 'http://api.example.com/_security/cross_cluster/api_key/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"access\": {\n    \"replication\": [\n      {\n        \"names\": [\"archive\"]\n      }\n    ]\n  },\n  \"metadata\": {\n    \"application\": \"replication\"\n  }\n}"'
Request example
Run `PUT /_security/cross_cluster/api_key/VuaCfGcBCdbkQm-e5aOx` to update a cross-cluster API key, assigning it new access scope and metadata. 
{
  "access": {
    "replication": [
      {
        "names": ["archive"]
      }
    ]
  },
  "metadata": {
    "application": "replication"
  }
}
Response examples (200)
A successful response from `PUT /_security/cross_cluster/api_key/VuaCfGcBCdbkQm-e5aOx` that indicates that the API key was updated. 
{
  "updated": true
}

Run a retention policy Added in 7.5.0

POST /_slm/_execute_retention
Api key auth Basic auth Bearer auth

Manually apply the retention policy to force immediate removal of snapshots that are expired according to the snapshot lifecycle policy retention rules. The retention policy is normally applied according to its schedule.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
POST /_slm/_execute_retention 
curl \
 --request POST 'http://api.example.com/_slm/_execute_retention' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "acknowledged": true
}