Get the autoscaling capacity Added in 7.11.0

GET /_autoscaling/capacity
Api key auth Basic auth Bearer auth

NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.

This API gets the current autoscaling capacity based on the configured autoscaling policy. It will return information to size the cluster appropriately to the current workload.

The required_capacity is calculated as the maximum of the required_capacity result of all individual deciders that are enabled for the policy.

The operator should verify that the current_nodes match the operator’s knowledge of the cluster to avoid making autoscaling decisions based on stale or incomplete information.

The response contains decider-specific information you can use to diagnose how and why autoscaling determined a certain capacity was required. This information is provided for diagnosis only. Do not use this information to make autoscaling decisions.

External documentation

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • policies object Required
      Hide policies attribute Show policies attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • required_capacity object Required
          Hide required_capacity attributes Show required_capacity attributes object
          • node object Required
            Hide node attributes Show node attributes object
          • total object Required
            Hide total attributes Show total attributes object
        • current_capacity object Required
          Hide current_capacity attributes Show current_capacity attributes object
          • node object Required
            Hide node attributes Show node attributes object
          • total object Required
            Hide total attributes Show total attributes object
        • current_nodes array[object] Required
          Hide current_nodes attribute Show current_nodes attribute object
        • deciders object Required
          Hide deciders attribute Show deciders attribute object
GET /_autoscaling/capacity 
curl \
 --request GET 'http://api.example.com/_autoscaling/capacity' \
 --header "Authorization: $API_KEY"
Response examples (200)
This may be a response to `GET /_autoscaling/capacity`. 
{
  policies: {}
}

Get feature usage information Added in 6.0.0

GET /_nodes/{node_id}/usage/{metric}
Api key auth Basic auth Bearer auth

Path parameters

  • node_id string | array[string] Required

    A comma-separated list of node IDs or names to limit the returned information; use _local to return information from the node you're connecting to, leave empty to get information from all nodes

  • metric string | array[string] Required

    Limits the information returned to the specific metrics. A comma-separated list of the following options: _all, rest_actions.

Query parameters

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object
      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]
        Hide failures attributes Show failures attributes object
      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • rest_actions object Required
          Hide rest_actions attribute Show rest_actions attribute object
          • * number Additional properties

        • Time unit for milliseconds

        • Time unit for milliseconds

        • aggregations object Required
          Hide aggregations attribute Show aggregations attribute object
          • * object Additional properties
GET /_nodes/{node_id}/usage/{metric} 
curl \
 --request GET 'http://api.example.com/_nodes/{node_id}/usage/{metric}' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "_nodes": {
    "failures": [
      {
        "type": "string",
        "reason": "string",
        "stack_trace": "string",
        "caused_by": {},
        "root_cause": [
          {}
        ],
        "suppressed": [
          {}
        ]
      }
    ],
    "total": 42.0,
    "successful": 42.0,
    "failed": 42.0
  },
  "cluster_name": "string",
  "nodes": {
    "additionalProperty1": {
      "rest_actions": {
        "additionalProperty1": 42.0,
        "additionalProperty2": 42.0
      },
      "": 42.0,
      "aggregations": {
        "additionalProperty1": {},
        "additionalProperty2": {}
      }
    },
    "additionalProperty2": {
      "rest_actions": {
        "additionalProperty1": 42.0,
        "additionalProperty2": 42.0
      },
      "": 42.0,
      "aggregations": {
        "additionalProperty1": {},
        "additionalProperty2": {}
      }
    }
  }
}

Connector

The connector and sync jobs APIs provide a convenient way to create and manage Elastic connectors and sync jobs in an internal index. Connectors are Elasticsearch integrations that bring content from third-party data sources, which can be deployed on Elastic Cloud or hosted on your own infrastructure:

  • Elastic managed connectors (Native connectors) are a managed service on Elastic Cloud
  • Self-managed connectors (Connector clients) are self-managed on your infrastructure.

This API provides an alternative to relying solely on Kibana UI for connector and sync job management. The API comes with a set of validations and assertions to ensure that the state representation in the internal index remains valid.

Check out the connector API tutorial

Update the connector name and description Beta

PUT /_connector/{connector_id}/_name
Api key auth Basic auth Bearer auth

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.
PUT /_connector/{connector_id}/_name 
curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_name' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"name\": \"Custom connector\",\n    \"description\": \"This is my customized connector\"\n}"'
Request example 
{
    "name": "Custom connector",
    "description": "This is my customized connector"
}
Response examples (200) 
{
  "result": "updated"
}

Cross-cluster replication

Create or update auto-follow patterns Added in 6.5.0

PUT /_ccr/auto_follow/{name}
Api key auth Basic auth Bearer auth

Create a collection of cross-cluster replication auto-follow patterns for a remote cluster. Newly created indices on the remote cluster that match any of the patterns are automatically configured as follower indices. Indices on the remote cluster that were created before the auto-follow pattern was created will not be auto-followed even if they match the pattern.

This API can also be used to update auto-follow patterns. NOTE: Follower indices that were configured automatically before updating an auto-follow pattern will remain unchanged even if they do not match against the new patterns.

External documentation

Path parameters

  • name string Required

    The name of the collection of auto-follow patterns.

Query parameters

application/json

Body Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
PUT /_ccr/auto_follow/{name} 
curl \
 --request PUT 'http://api.example.com/_ccr/auto_follow/{name}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"remote_cluster\" : \"remote_cluster\",\n  \"leader_index_patterns\" :\n  [\n    \"leader_index*\"\n  ],\n  \"follow_index_pattern\" : \"{{leader_index}}-follower\",\n  \"settings\": {\n    \"index.number_of_replicas\": 0\n  },\n  \"max_read_request_operation_count\" : 1024,\n  \"max_outstanding_read_requests\" : 16,\n  \"max_read_request_size\" : \"1024k\",\n  \"max_write_request_operation_count\" : 32768,\n  \"max_write_request_size\" : \"16k\",\n  \"max_outstanding_write_requests\" : 8,\n  \"max_write_buffer_count\" : 512,\n  \"max_write_buffer_size\" : \"512k\",\n  \"max_retry_delay\" : \"10s\",\n  \"read_poll_timeout\" : \"30s\"\n}"'
Request example
Run `PUT /_ccr/auto_follow/my_auto_follow_pattern` to creates an auto-follow pattern. 
{
  "remote_cluster" : "remote_cluster",
  "leader_index_patterns" :
  [
    "leader_index*"
  ],
  "follow_index_pattern" : "{{leader_index}}-follower",
  "settings": {
    "index.number_of_replicas": 0
  },
  "max_read_request_operation_count" : 1024,
  "max_outstanding_read_requests" : 16,
  "max_read_request_size" : "1024k",
  "max_write_request_operation_count" : 32768,
  "max_write_request_size" : "16k",
  "max_outstanding_write_requests" : 8,
  "max_write_buffer_count" : 512,
  "max_write_buffer_size" : "512k",
  "max_retry_delay" : "10s",
  "read_poll_timeout" : "30s"
}
Response examples (200)
A successful response for creating an auto-follow pattern. 
{
  "acknowledged": true
}

Pause a follower Added in 6.5.0

POST /{index}/_ccr/pause_follow
Api key auth Basic auth Bearer auth

Pause a cross-cluster replication follower index. The follower index will not fetch any additional operations from the leader index. You can resume following with the resume follower API. You can pause and resume a follower index to change the configuration of the following task.

Path parameters

  • index string Required

    The name of the follower index.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
POST /{index}/_ccr/pause_follow 
curl \
 --request POST 'http://api.example.com/{index}/_ccr/pause_follow' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `POST /follower_index/_ccr/pause_follow`, which pauses a follower index. 
{
  "acknowledged" : true
}

Resume a follower Added in 6.5.0

POST /{index}/_ccr/resume_follow
Api key auth Basic auth Bearer auth

Resume a cross-cluster replication follower index that was paused. The follower index could have been paused with the pause follower API. Alternatively it could be paused due to replication that cannot be retried due to failures during following tasks. When this API returns, the follower index will resume fetching operations from the leader index.

External documentation

Path parameters

  • index string Required

    The name of the follow index to resume following.

Query parameters

application/json

Body

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
POST /{index}/_ccr/resume_follow 
curl \
 --request POST 'http://api.example.com/{index}/_ccr/resume_follow' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"max_read_request_operation_count\" : 1024,\n  \"max_outstanding_read_requests\" : 16,\n  \"max_read_request_size\" : \"1024k\",\n  \"max_write_request_operation_count\" : 32768,\n  \"max_write_request_size\" : \"16k\",\n  \"max_outstanding_write_requests\" : 8,\n  \"max_write_buffer_count\" : 512,\n  \"max_write_buffer_size\" : \"512k\",\n  \"max_retry_delay\" : \"10s\",\n  \"read_poll_timeout\" : \"30s\"\n}"'
Request example
Run `POST /follower_index/_ccr/resume_follow` to resume the follower index. 
{
  "max_read_request_operation_count" : 1024,
  "max_outstanding_read_requests" : 16,
  "max_read_request_size" : "1024k",
  "max_write_request_operation_count" : 32768,
  "max_write_request_size" : "16k",
  "max_outstanding_write_requests" : 8,
  "max_write_buffer_count" : 512,
  "max_write_buffer_size" : "512k",
  "max_retry_delay" : "10s",
  "read_poll_timeout" : "30s"
}
Response examples (200)
A successful response from resuming a folower index. 
{
  "acknowledged" : true
}

Get data stream lifecycle stats Added in 8.12.0

GET /_lifecycle/stats
Api key auth Basic auth Bearer auth

Get statistics about the data streams that are managed by a data stream lifecycle.

Responses

GET /_lifecycle/stats 
curl \
 --request GET 'http://api.example.com/_lifecycle/stats' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for `GET _lifecycle/stats?human&pretty` 
{
  "last_run_duration_in_millis": 2,
  "last_run_duration": "2ms",
  "time_between_starts_in_millis": 9998,
  "time_between_starts": "9.99s",
  "data_streams_count": 2,
  "data_streams": [
    {
      "name": "my-data-stream",
      "backing_indices_in_total": 2,
      "backing_indices_in_error": 0
    },
    {
      "name": "my-other-stream",
      "backing_indices_in_total": 2,
      "backing_indices_in_error": 1
    }
  ]
}

Convert an index alias to a data stream Added in 7.9.0

POST /_data_stream/_migrate/{name}
Api key auth Basic auth Bearer auth

Converts an index alias to a data stream. You must have a matching index template that is data stream enabled. The alias must meet the following criteria: The alias must have a write index; All indices for the alias must have a @timestamp field mapping of a date or date_nanos field type; The alias must not have any filters; The alias must not use custom routing. If successful, the request removes the alias and creates a data stream with the same name. The indices for the alias become hidden backing indices for the stream. The write index for the alias becomes the write index for the stream.

Path parameters

  • name string Required

    Name of the index alias to convert to a data stream.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
POST /_data_stream/_migrate/{name} 
curl \
 --request POST 'http://api.example.com/_data_stream/_migrate/{name}' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "acknowledged": true
}

Create or update a document in an index

PUT /{index}/_doc/{id}
Api key auth Basic auth Bearer auth

Add a JSON document to the specified data stream or index and make it searchable. If the target is an index and the document already exists, the request updates the document and increments its version.

NOTE: You cannot use this API to send update requests for existing documents in a data stream.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:

  • To add or overwrite a document using the PUT /<target>/_doc/<_id> request format, you must have the create, index, or write index privilege.
  • To add a document using the POST /<target>/_doc/ request format, you must have the create_doc, create, index, or write index privilege.
  • To automatically create a data stream or index with this API request, you must have the auto_configure, create_index, or manage index privilege.

Automatic data stream creation requires a matching index template with data stream enabled.

NOTE: Replica shards might not all be started when an indexing operation returns successfully. By default, only the primary is required. Set wait_for_active_shards to change this default behavior.

Automatically create data streams and indices

If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.

If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.

NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.

If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.

Automatic index creation is controlled by the action.auto_create_index setting. If it is true, any index can be created automatically. You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely. Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked. When a list is specified, the default behaviour is to disallow.

NOTE: The action.auto_create_index setting affects the automatic creation of indices only. It does not affect the creation of data streams.

Optimistic concurrency control

Index operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no and if_primary_term parameters. If a mismatch is detected, the operation will result in a VersionConflictException and a status code of 409.

Routing

By default, shard placement — or routing — is controlled by using a hash of the document's ID value. For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.

When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself. This does come at the (very minimal) cost of an additional document parsing pass. If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.

NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.

Distributed

The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.

Active shards

To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation. If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs. By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1). This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards. To alter this behavior per operation, use the wait_for_active_shards request parameter.

Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1). Specifying a negative value or a number greater than the number of shard copies will throw an error.

For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes). If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding. This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data. If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding. This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard. However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index. The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.

It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts. After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary. The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.

No operation (noop) updates

When updating a document by using this API, a new version of the document is always created even if the document hasn't changed. If this isn't acceptable use the _update API with detect_noop set to true. The detect_noop option isn't available on this API because it doesn’t fetch the old source and isn't able to compare it against the new source.

There isn't a definitive rule for when noop updates aren't acceptable. It's a combination of lots of factors like how frequently your data source sends updates that are actually noops and how many queries per second Elasticsearch runs on the shard receiving the updates.

Versioning

Each indexed document is given a version number. By default, internal versioning is used that starts at 1 and increments with each update, deletes included. Optionally, the version number can be set to an external value (for example, if maintained in a database). To enable this functionality, version_type should be set to external. The value provided must be a numeric, long value greater than or equal to 0, and less than around 9.2e+18.

NOTE: Versioning is completely real time, and is not affected by the near real time aspects of search operations. If no version is provided, the operation runs without any version checks.

When using the external version type, the system checks to see if the version number passed to the index request is greater than the version of the currently stored document. If true, the document will be indexed and the new version number used. If the value provided is less than or equal to the stored document's version number, a version conflict will occur and the index operation will fail. For example:

PUT my-index-000001/_doc/1?version=2&version_type=external
{
  "user": {
    "id": "elkbee"
  }
}

In this example, the operation will succeed since the supplied version of 2 is higher than the current document version of 1.
If the document was already updated and its version was set to 2 or higher, the indexing command will fail and result in a conflict (409 HTTP status code).

A nice side effect is that there is no need to maintain strict ordering of async indexing operations run as a result of changes to a source database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from a database is simplified if external versioning is used, as only the latest version will be used if the index operations arrive out of order.
External documentation

Path parameters

  • index string Required

    The name of the data stream or index to target. If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream. If the target doesn't exist and doesn't match a data stream template, this request creates the index. You can check for existing targets with the resolve index API.

  • id string Required

    A unique identifier for the document. To automatically generate a document ID, use the POST /<target>/_doc/ request format and omit this parameter.

Query parameters

  • if_primary_term number

    Only perform the operation if the document has this primary term.

  • if_seq_no number

    Only perform the operation if the document has this sequence number.

  • include_source_on_error boolean

    True or false if to include the document source in the error message in case of parsing errors.

  • op_type string

    Set to create to only index the document if it does not already exist (put if absent). If a document with the specified _id already exists, the indexing operation will fail. The behavior is the same as using the <index>/_create endpoint. If a document ID is specified, this paramater defaults to index. Otherwise, it defaults to create. If the request targets a data stream, an op_type of create is required.

    Values are index or create.

  • pipeline string

    The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, then setting the value to _none disables the default ingest pipeline for this request. If a final pipeline is configured it will always run, regardless of the value of this parameter.

  • refresh string

    If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, it waits for a refresh to make this operation visible to search. If false, it does nothing with refreshes.

    Values are true, false, or wait_for.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • timeout string

    The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards.

    This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.

  • version number

    An explicit version number for concurrency control. It must be a non-negative long number.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default value of 1 means it waits for each primary shard to be active.

  • require_alias boolean

    If true, the destination must be an index alias.

application/json

Body Required

object object

Responses

PUT /{index}/_doc/{id} 
curl \
 --request PUT 'http://api.example.com/{index}/_doc/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"@timestamp\": \"2099-11-15T13:12:00\",\n  \"message\": \"GET /search HTTP/1.1 200 1070000\",\n  \"user\": {\n    \"id\": \"kimchy\"\n  }\n}"'
Request examples
Run `POST my-index-000001/_doc/` to index a document. When you use the `POST /<target>/_doc/` request format, the `op_type` is automatically set to `create` and the index operation generates a unique ID for the document. 
{
  "@timestamp": "2099-11-15T13:12:00",
  "message": "GET /search HTTP/1.1 200 1070000",
  "user": {
    "id": "kimchy"
  }
}
Run `PUT my-index-000001/_doc/1` to insert a JSON document into the `my-index-000001` index with an `_id` of 1. 
{
  "@timestamp": "2099-11-15T13:12:00",
  "message": "GET /search HTTP/1.1 200 1070000",
  "user": {
    "id": "kimchy"
  }
}
Response examples (200)
A successful response from `POST my-index-000001/_doc/`, which contains an automated document ID. 
{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "W0tpsmIBdwcYyG50zbta",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "result": "created"
}
A successful response from `PUT my-index-000001/_doc/1`. 
{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "result": "created"
}

Create or update a document in an index

POST /{index}/_doc
Api key auth Basic auth Bearer auth

Add a JSON document to the specified data stream or index and make it searchable. If the target is an index and the document already exists, the request updates the document and increments its version.

NOTE: You cannot use this API to send update requests for existing documents in a data stream.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:

  • To add or overwrite a document using the PUT /<target>/_doc/<_id> request format, you must have the create, index, or write index privilege.
  • To add a document using the POST /<target>/_doc/ request format, you must have the create_doc, create, index, or write index privilege.
  • To automatically create a data stream or index with this API request, you must have the auto_configure, create_index, or manage index privilege.

Automatic data stream creation requires a matching index template with data stream enabled.

NOTE: Replica shards might not all be started when an indexing operation returns successfully. By default, only the primary is required. Set wait_for_active_shards to change this default behavior.

Automatically create data streams and indices

If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.

If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.

NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.

If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.

Automatic index creation is controlled by the action.auto_create_index setting. If it is true, any index can be created automatically. You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely. Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked. When a list is specified, the default behaviour is to disallow.

NOTE: The action.auto_create_index setting affects the automatic creation of indices only. It does not affect the creation of data streams.

Optimistic concurrency control

Index operations can be made conditional and only be performed if the last modification to the document was assigned the sequence number and primary term specified by the if_seq_no and if_primary_term parameters. If a mismatch is detected, the operation will result in a VersionConflictException and a status code of 409.

Routing

By default, shard placement — or routing — is controlled by using a hash of the document's ID value. For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.

When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself. This does come at the (very minimal) cost of an additional document parsing pass. If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.

NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.

Distributed

The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.

Active shards

To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation. If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs. By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1). This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards. To alter this behavior per operation, use the wait_for_active_shards request parameter.

Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1). Specifying a negative value or a number greater than the number of shard copies will throw an error.

For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes). If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding. This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data. If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding. This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard. However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index. The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.

It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts. After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary. The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.

No operation (noop) updates

When updating a document by using this API, a new version of the document is always created even if the document hasn't changed. If this isn't acceptable use the _update API with detect_noop set to true. The detect_noop option isn't available on this API because it doesn’t fetch the old source and isn't able to compare it against the new source.

There isn't a definitive rule for when noop updates aren't acceptable. It's a combination of lots of factors like how frequently your data source sends updates that are actually noops and how many queries per second Elasticsearch runs on the shard receiving the updates.

Versioning

Each indexed document is given a version number. By default, internal versioning is used that starts at 1 and increments with each update, deletes included. Optionally, the version number can be set to an external value (for example, if maintained in a database). To enable this functionality, version_type should be set to external. The value provided must be a numeric, long value greater than or equal to 0, and less than around 9.2e+18.

NOTE: Versioning is completely real time, and is not affected by the near real time aspects of search operations. If no version is provided, the operation runs without any version checks.

When using the external version type, the system checks to see if the version number passed to the index request is greater than the version of the currently stored document. If true, the document will be indexed and the new version number used. If the value provided is less than or equal to the stored document's version number, a version conflict will occur and the index operation will fail. For example:

PUT my-index-000001/_doc/1?version=2&version_type=external
{
  "user": {
    "id": "elkbee"
  }
}

In this example, the operation will succeed since the supplied version of 2 is higher than the current document version of 1.
If the document was already updated and its version was set to 2 or higher, the indexing command will fail and result in a conflict (409 HTTP status code).

A nice side effect is that there is no need to maintain strict ordering of async indexing operations run as a result of changes to a source database, as long as version numbers from the source database are used.
Even the simple case of updating the Elasticsearch index using data from a database is simplified if external versioning is used, as only the latest version will be used if the index operations arrive out of order.
External documentation

Path parameters

  • index string Required

    The name of the data stream or index to target. If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream. If the target doesn't exist and doesn't match a data stream template, this request creates the index. You can check for existing targets with the resolve index API.

Query parameters

  • if_primary_term number

    Only perform the operation if the document has this primary term.

  • if_seq_no number

    Only perform the operation if the document has this sequence number.

  • include_source_on_error boolean

    True or false if to include the document source in the error message in case of parsing errors.

  • op_type string

    Set to create to only index the document if it does not already exist (put if absent). If a document with the specified _id already exists, the indexing operation will fail. The behavior is the same as using the <index>/_create endpoint. If a document ID is specified, this paramater defaults to index. Otherwise, it defaults to create. If the request targets a data stream, an op_type of create is required.

    Values are index or create.

  • pipeline string

    The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, then setting the value to _none disables the default ingest pipeline for this request. If a final pipeline is configured it will always run, regardless of the value of this parameter.

  • refresh string

    If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, it waits for a refresh to make this operation visible to search. If false, it does nothing with refreshes.

    Values are true, false, or wait_for.

  • routing string

    A custom value that is used to route operations to a specific shard.

  • timeout string

    The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards.

    This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.

  • version number

    An explicit version number for concurrency control. It must be a non-negative long number.

  • version_type string

    The version type.

    Values are internal, external, external_gte, or force.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default value of 1 means it waits for each primary shard to be active.

  • require_alias boolean

    If true, the destination must be an index alias.

application/json

Body Required

object object

Responses

POST /{index}/_doc 
curl \
 --request POST 'http://api.example.com/{index}/_doc' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"@timestamp\": \"2099-11-15T13:12:00\",\n  \"message\": \"GET /search HTTP/1.1 200 1070000\",\n  \"user\": {\n    \"id\": \"kimchy\"\n  }\n}"'
Request examples
Run `POST my-index-000001/_doc/` to index a document. When you use the `POST /<target>/_doc/` request format, the `op_type` is automatically set to `create` and the index operation generates a unique ID for the document. 
{
  "@timestamp": "2099-11-15T13:12:00",
  "message": "GET /search HTTP/1.1 200 1070000",
  "user": {
    "id": "kimchy"
  }
}
Run `PUT my-index-000001/_doc/1` to insert a JSON document into the `my-index-000001` index with an `_id` of 1. 
{
  "@timestamp": "2099-11-15T13:12:00",
  "message": "GET /search HTTP/1.1 200 1070000",
  "user": {
    "id": "kimchy"
  }
}
Response examples (200)
A successful response from `POST my-index-000001/_doc/`, which contains an automated document ID. 
{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "W0tpsmIBdwcYyG50zbta",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "result": "created"
}
A successful response from `PUT my-index-000001/_doc/1`. 
{
  "_shards": {
    "total": 2,
    "failed": 0,
    "successful": 2
  },
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 1,
  "_seq_no": 0,
  "_primary_term": 1,
  "result": "created"
}

Stop async ES|QL query Added in 8.18.0

POST /_query/async/{id}/stop
Api key auth Basic auth Bearer auth

This API interrupts the query execution and returns the results so far. If the Elasticsearch security features are enabled, only the user who first submitted the ES|QL query can stop it.

External documentation

Path parameters

  • id string Required

    The unique identifier of the query. A query ID is provided in the ES|QL async query API response for a query that does not complete in the designated time. A query ID is also provided when the request was submitted with the keep_on_completion parameter set to true.

Query parameters

  • drop_null_columns boolean

    Indicates whether columns that are entirely null will be removed from the columns and values portion of the results. If true, the response will include an extra section under the name all_columns which has the name of all the columns.

Responses

POST /_query/async/{id}/stop 
curl \
 --request POST 'http://api.example.com/_query/async/{id}/stop' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{}

Executes several [fleet searches](https://www.elastic.co/guide/en/elasticsearch/reference/current/fleet-search.html) with a single API request Technical preview

GET /{index}/_fleet/_fleet_msearch
Api key auth Basic auth Bearer auth

The API follows the same structure as the multi search API. However, similar to the fleet search API, it supports the wait_for_checkpoints parameter.

Path parameters

  • index string Required

    A single target to search. If the target is an index alias, it must resolve to a single index.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • ccs_minimize_roundtrips boolean

    If true, network roundtrips between the coordinating node and remote clusters are minimized for cross-cluster search requests.

  • expand_wildcards string | array[string]

    Type of index that wildcard expressions can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams.

  • ignore_throttled boolean

    If true, concrete, expanded or aliased indices are ignored when frozen.

  • ignore_unavailable boolean

    If true, missing or closed indices are not included in the response.

  • max_concurrent_searches number

    Maximum number of concurrent searches the multi search API can execute.

  • max_concurrent_shard_requests number

    Maximum number of concurrent shard requests that each sub-search request executes per node.

  • pre_filter_shard_size number

    Defines a threshold that enforces a pre-filter roundtrip to prefilter search shards based on query rewriting if the number of shards the search request expands to exceeds the threshold. This filter roundtrip can limit the number of shards significantly if for instance a shard can not match any documents based on its rewrite method i.e., if date filters are mandatory to match but the shard bounds and the query are disjoint.

  • search_type string

    Indicates whether global term and document frequencies should be used when scoring returned documents.

    Values are query_then_fetch or dfs_query_then_fetch.

  • rest_total_hits_as_int boolean

    If true, hits.total are returned as an integer in the response. Defaults to false, which returns an object.

  • typed_keys boolean

    Specifies whether aggregation and suggester names should be prefixed by their respective types in the response.

  • wait_for_checkpoints array[number]

    A comma separated list of checkpoints. When configured, the search API will only be executed on a shard after the relevant checkpoint has become visible for search. Defaults to an empty list which will cause Elasticsearch to immediately execute the search.

  • allow_partial_search_results boolean

    If true, returns partial results if there are shard request timeouts or shard failures. If false, returns an error with no partial results. Defaults to the configured cluster setting search.default_allow_partial_results which is true by default.

application/json

Body object Required

One of:
MultisearchHeader object MultisearchBody object

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
GET /{index}/_fleet/_fleet_msearch 
curl \
 --request GET 'http://api.example.com/{index}/_fleet/_fleet_msearch' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '[{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"index":"string","preference":"string","request_cache":true,"routing":"string","search_type":"query_then_fetch","ccs_minimize_roundtrips":true,"allow_partial_search_results":true,"ignore_throttled":true}]'
Request examples 
[
  {
    "allow_no_indices": true,
    "expand_wildcards": "string",
    "ignore_unavailable": true,
    "index": "string",
    "preference": "string",
    "request_cache": true,
    "routing": "string",
    "search_type": "query_then_fetch",
    "ccs_minimize_roundtrips": true,
    "allow_partial_search_results": true,
    "ignore_throttled": true
  }
]
Response examples (200) 
{
  "docs": [
    {
      "took": 42.0,
      "timed_out": true,
      "_shards": {
        "failed": 42.0,
        "successful": 42.0,
        "total": 42.0,
        "failures": [
          {}
        ],
        "skipped": 42.0
      },
      "hits": {
        "hits": [
          {}
        ]
      },
      "aggregations": {},
      "_clusters": {
        "skipped": 42.0,
        "successful": 42.0,
        "total": 42.0,
        "running": 42.0,
        "partial": 42.0,
        "failed": 42.0,
        "details": {}
      },
      "fields": {
        "additionalProperty1": {},
        "additionalProperty2": {}
      },
      "max_score": 42.0,
      "num_reduce_phases": 42.0,
      "profile": {
        "shards": [
          {}
        ]
      },
      "pit_id": "string",
      "_scroll_id": "string",
      "suggest": {
        "additionalProperty1": [
          {}
        ],
        "additionalProperty2": [
          {}
        ]
      },
      "terminated_early": true,
      "status": 42.0
    }
  ]
}

The purpose of the fleet search api is to provide a search api where the search will only be executed Technical preview

GET /{index}/_fleet/_fleet_search
Api key auth Basic auth Bearer auth

after provided checkpoint has been processed and is visible for searches inside of Elasticsearch.

Path parameters

  • index string Required

    A single target to search. If the target is an index alias, it must resolve to a single index.

Query parameters

application/json

Body

  • aggregations object
  • collapse object
    External documentation
  • explain boolean

    If true, returns detailed information about score computation as part of a hit.

  • ext object

    Configuration of search extensions defined by Elasticsearch plugins.

    Hide ext attribute Show ext attribute object
    • * object Additional properties
  • from number

    Starting document offset. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • highlight object
    Hide highlight attributes Show highlight attributes object

    • type string

      Any of:
      HighlighterType string HighlighterType string

      Values are plain, fvh, or unified.

    • boundary_chars string

      A string that contains each boundary character.

    • boundary_max_scan number

      How far to scan for boundary characters.

    • boundary_scanner string

      Values are chars, sentence, or word.

    • boundary_scanner_locale string

      Controls which locale is used to search for sentence and word boundaries. This parameter takes a form of a language tag, for example: "en-US", "fr-FR", "ja-JP".

    • force_source boolean Deprecated
    • fragmenter string

      Values are simple or span.

    • fragment_size number

      The size of the highlighted fragment in characters.

    • highlight_filter boolean
    • highlight_query object

      An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

      External documentation
    • max_fragment_length number
    • max_analyzed_offset number

      If set to a non-negative value, highlighting stops at this defined maximum limit. The rest of the text is not processed, thus not highlighted and no error is returned The max_analyzed_offset query setting does not override the index.highlight.max_analyzed_offset setting, which prevails when it’s set to lower value than the query setting.

    • no_match_size number

      The amount of text you want to return from the beginning of the field if there are no matching fragments to highlight.

    • number_of_fragments number

      The maximum number of fragments to return. If the number of fragments is set to 0, no fragments are returned. Instead, the entire field contents are highlighted and returned. This can be handy when you need to highlight short texts such as a title or address, but fragmentation is not required. If number_of_fragments is 0, fragment_size is ignored.

    • options object
      Hide options attribute Show options attribute object
      • * object Additional properties
    • order string

      Value is score.

    • phrase_limit number

      Controls the number of matching phrases in a document that are considered. Prevents the fvh highlighter from analyzing too many phrases and consuming too much memory. When using matched_fields, phrase_limit phrases per matched field are considered. Raising the limit increases query time and consumes more memory. Only supported by the fvh highlighter.

    • post_tags array[string]

      Use in conjunction with pre_tags to define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in <em> and </em> tags.

    • pre_tags array[string]

      Use in conjunction with post_tags to define the HTML tags to use for the highlighted text. By default, highlighted text is wrapped in <em> and </em> tags.

    • require_field_match boolean

      By default, only fields that contains a query match are highlighted. Set to false to highlight all fields.

    • tags_schema string

      Value is styled.

    • encoder string

      Values are default or html.

    • fields object Required
  • track_total_hits boolean | number

    Number of hits matching the query to count accurately. If true, the exact number of hits is returned at the cost of some performance. If false, the response does not include the total number of hits matching the query. Defaults to 10,000 hits.

  • indices_boost array[object]

    Boosts the _score of documents from specified indices.

    Hide indices_boost attribute Show indices_boost attribute object
    • * number Additional properties
  • docvalue_fields array[object]

    Array of wildcard (*) patterns. The request returns doc values for field names matching these patterns in the hits.fields property of the response.

    Hide docvalue_fields attributes Show docvalue_fields attributes object
    • field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • format string

      The format in which the values are returned.

    • include_unmapped boolean
  • min_score number

    Minimum _score for matching documents. Documents with a lower _score are not included in the search results.

  • post_filter object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation
  • profile boolean
  • query object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    External documentation

  • rescore object | array[object]

    One of:
    Rescore object array-2 array[object]
    Hide attributes Show attributes
    • window_size number
    • query object
      Hide query attributes Show query attributes object
    • learning_to_rank object
      Hide learning_to_rank attributes Show learning_to_rank attributes object
      • model_id string Required

        The unique identifier of the trained model uploaded to Elasticsearch

      • params object

        Named parameters to be passed to the query templates used for feature

        Hide params attribute Show params attribute object
        • * object Additional properties
  • script_fields object

    Retrieve a script evaluation (based on different fields) for each hit.

    Hide script_fields attribute Show script_fields attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • script object Required
        Hide script attributes Show script attributes object
        • source string

          The script source.

        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          Hide params attribute Show params attribute object
          • * object Additional properties

        • lang string

          Any of:
          ScriptLanguage string ScriptLanguage string

          Values are painless, expression, mustache, or java.

        • options object
          Hide options attribute Show options attribute object
          • * string Additional properties
      • ignore_failure boolean
  • search_after array[number | string | boolean | null | object]

    A field value.

    A field value.

    One of:
    FieldValue number FieldValue number FieldValue string FieldValue boolean FieldValue string | null FieldValue object
  • size number

    The number of hits to return. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • slice object
    Hide slice attributes Show slice attributes object
    • field string

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • id string Required
    • max number Required

  • sort string | object | array[string | object]

    One of:
    Field string SortOptions object Sort array[string | object]

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • _source boolean | object

    Defines how to fetch a source. Fetching can be disabled entirely, or the source can be filtered.

    One of:
    SourceConfig boolean SourceFilter object
  • fields array[object]

    Array of wildcard (*) patterns. The request returns values for field names matching these patterns in the hits.fields property of the response.

    Hide fields attributes Show fields attributes object
    • field string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • format string

      The format in which the values are returned.

    • include_unmapped boolean
  • suggest object
    Hide suggest attribute Show suggest attribute object
    • text string

      Global suggest text, to avoid repetition when the same text is used in several suggesters

  • terminate_after number

    Maximum number of documents to collect for each shard. If a query reaches this limit, Elasticsearch terminates the query early. Elasticsearch collects documents before sorting. Defaults to 0, which does not terminate query execution early.

  • timeout string

    Specifies the period of time to wait for a response from each shard. If no response is received before the timeout expires, the request fails and returns an error. Defaults to no timeout.

  • track_scores boolean

    If true, calculate and return document scores, even if the scores are not used for sorting.

  • version boolean

    If true, returns document version as part of a hit.

  • seq_no_primary_term boolean

    If true, returns sequence number and primary term of the last modification of each hit. See Optimistic concurrency control.

  • stored_fields string | array[string]
  • pit object
    Hide pit attributes Show pit attributes object
    • id string Required
    • keep_alive string

      A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • runtime_mappings object
    Hide runtime_mappings attribute Show runtime_mappings attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • fields object

        For type composite

        Hide fields attribute Show fields attribute object
        • * object Additional properties
          Hide * attribute Show * attribute object
          • type string Required

            Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

      • fetch_fields array[object]

        For type lookup

        Hide fetch_fields attributes Show fetch_fields attributes object
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • format string
      • format string

        A custom format for date type runtime fields.

      • input_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_field string

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • target_index string
      • script object
        Hide script attributes Show script attributes object
        • source string

          The script source.

        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          Hide params attribute Show params attribute object
          • * object Additional properties

        • lang string

          Any of:
          ScriptLanguage string ScriptLanguage string

          Values are painless, expression, mustache, or java.

        • options object
          Hide options attribute Show options attribute object
          • * string Additional properties
      • type string Required

        Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

  • stats array[string]

    Stats groups to associate with the search. Each group maintains a statistics aggregation for its associated searches. You can retrieve these stats using the indices stats API.

Responses

GET /{index}/_fleet/_fleet_search 
curl \
 --request GET 'http://api.example.com/{index}/_fleet/_fleet_search' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"aggregations":{},"collapse":{},"explain":true,"ext":{"additionalProperty1":{},"additionalProperty2":{}},"from":42.0,"highlight":{"":"plain","boundary_chars":"string","boundary_max_scan":42.0,"boundary_scanner":"chars","boundary_scanner_locale":"string","force_source":true,"fragmenter":"simple","fragment_size":42.0,"highlight_filter":true,"highlight_query":{},"max_fragment_length":42.0,"max_analyzed_offset":42.0,"no_match_size":42.0,"number_of_fragments":42.0,"options":{"additionalProperty1":{},"additionalProperty2":{}},"order":"score","phrase_limit":42.0,"post_tags":["string"],"pre_tags":["string"],"require_field_match":true,"tags_schema":"styled","encoder":"default","fields":{}},"track_total_hits":true,"indices_boost":[{"additionalProperty1":42.0,"additionalProperty2":42.0}],"docvalue_fields":[{"field":"string","format":"string","include_unmapped":true}],"min_score":42.0,"post_filter":{},"profile":true,"query":{},"rescore":{"window_size":42.0,"query":{"rescore_query":{},"query_weight":42.0,"rescore_query_weight":42.0,"score_mode":"avg"},"learning_to_rank":{"model_id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}}}},"script_fields":{"additionalProperty1":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"search_after":[42.0],"size":42.0,"slice":{"field":"string","id":"string","max":42.0},"":true,"fields":[{"field":"string","format":"string","include_unmapped":true}],"suggest":{"text":"string"},"terminate_after":42.0,"timeout":"string","track_scores":true,"version":true,"seq_no_primary_term":true,"stored_fields":"string","pit":{"id":"string","keep_alive":"string"},"runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"stats":["string"]}'
Request examples 
{
  "aggregations": {},
  "collapse": {},
  "explain": true,
  "ext": {
    "additionalProperty1": {},
    "additionalProperty2": {}
  },
  "from": 42.0,
  "highlight": {
    "": "plain",
    "boundary_chars": "string",
    "boundary_max_scan": 42.0,
    "boundary_scanner": "chars",
    "boundary_scanner_locale": "string",
    "force_source": true,
    "fragmenter": "simple",
    "fragment_size": 42.0,
    "highlight_filter": true,
    "highlight_query": {},
    "max_fragment_length": 42.0,
    "max_analyzed_offset": 42.0,
    "no_match_size": 42.0,
    "number_of_fragments": 42.0,
    "options": {
      "additionalProperty1": {},
      "additionalProperty2": {}
    },
    "order": "score",
    "phrase_limit": 42.0,
    "post_tags": [
      "string"
    ],
    "pre_tags": [
      "string"
    ],
    "require_field_match": true,
    "tags_schema": "styled",
    "encoder": "default",
    "fields": {}
  },
  "track_total_hits": true,
  "indices_boost": [
    {
      "additionalProperty1": 42.0,
      "additionalProperty2": 42.0
    }
  ],
  "docvalue_fields": [
    {
      "field": "string",
      "format": "string",
      "include_unmapped": true
    }
  ],
  "min_score": 42.0,
  "post_filter": {},
  "profile": true,
  "query": {},
  "rescore": {
    "window_size": 42.0,
    "query": {
      "rescore_query": {},
      "query_weight": 42.0,
      "rescore_query_weight": 42.0,
      "score_mode": "avg"
    },
    "learning_to_rank": {
      "model_id": "string",
      "params": {
        "additionalProperty1": {},
        "additionalProperty2": {}
      }
    }
  },
  "script_fields": {
    "additionalProperty1": {
      "script": {
        "source": "string",
        "id": "string",
        "params": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "": "painless",
        "options": {
          "additionalProperty1": "string",
          "additionalProperty2": "string"
        }
      },
      "ignore_failure": true
    },
    "additionalProperty2": {
      "script": {
        "source": "string",
        "id": "string",
        "params": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "": "painless",
        "options": {
          "additionalProperty1": "string",
          "additionalProperty2": "string"
        }
      },
      "ignore_failure": true
    }
  },
  "search_after": [
    42.0
  ],
  "size": 42.0,
  "slice": {
    "field": "string",
    "id": "string",
    "max": 42.0
  },
  "": true,
  "fields": [
    {
      "field": "string",
      "format": "string",
      "include_unmapped": true
    }
  ],
  "suggest": {
    "text": "string"
  },
  "terminate_after": 42.0,
  "timeout": "string",
  "track_scores": true,
  "version": true,
  "seq_no_primary_term": true,
  "stored_fields": "string",
  "pit": {
    "id": "string",
    "keep_alive": "string"
  },
  "runtime_mappings": {
    "additionalProperty1": {
      "fields": {
        "additionalProperty1": {
          "type": "boolean"
        },
        "additionalProperty2": {
          "type": "boolean"
        }
      },
      "fetch_fields": [
        {
          "field": "string",
          "format": "string"
        }
      ],
      "format": "string",
      "input_field": "string",
      "target_field": "string",
      "target_index": "string",
      "script": {
        "source": "string",
        "id": "string",
        "params": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "": "painless",
        "options": {
          "additionalProperty1": "string",
          "additionalProperty2": "string"
        }
      },
      "type": "boolean"
    },
    "additionalProperty2": {
      "fields": {
        "additionalProperty1": {
          "type": "boolean"
        },
        "additionalProperty2": {
          "type": "boolean"
        }
      },
      "fetch_fields": [
        {
          "field": "string",
          "format": "string"
        }
      ],
      "format": "string",
      "input_field": "string",
      "target_field": "string",
      "target_index": "string",
      "script": {
        "source": "string",
        "id": "string",
        "params": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "": "painless",
        "options": {
          "additionalProperty1": "string",
          "additionalProperty2": "string"
        }
      },
      "type": "boolean"
    }
  },
  "stats": [
    "string"
  ]
}
Response examples (200) 
{
  "took": 42.0,
  "timed_out": true,
  "_shards": {
    "failed": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "failures": [
      {
        "index": "string",
        "node": "string",
        "reason": {
          "type": "string",
          "reason": "string",
          "stack_trace": "string",
          "caused_by": {},
          "root_cause": [
            {}
          ],
          "suppressed": [
            {}
          ]
        },
        "shard": 42.0,
        "status": "string"
      }
    ],
    "skipped": 42.0
  },
  "hits": {
    "total": {
      "relation": "eq",
      "value": 42.0
    },
    "hits": [
      {
        "_index": "string",
        "_id": "string",
        "_score": 42.0,
        "_explanation": {
          "description": "string",
          "details": [
            {}
          ],
          "value": 42.0
        },
        "fields": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "highlight": {
          "additionalProperty1": [
            "string"
          ],
          "additionalProperty2": [
            "string"
          ]
        },
        "inner_hits": {
          "additionalProperty1": {
            "hits": {}
          },
          "additionalProperty2": {
            "hits": {}
          }
        },
        "matched_queries": [
          "string"
        ],
        "_nested": {
          "field": "string",
          "offset": 42.0,
          "_nested": {}
        },
        "_ignored": [
          "string"
        ],
        "ignored_field_values": {
          "additionalProperty1": [
            {}
          ],
          "additionalProperty2": [
            {}
          ]
        },
        "_shard": "string",
        "_node": "string",
        "_routing": "string",
        "_source": {},
        "_rank": 42.0,
        "_seq_no": 42.0,
        "_primary_term": 42.0,
        "_version": 42.0,
        "sort": [
          42.0
        ]
      }
    ],
    "max_score": 42.0
  },
  "aggregations": {},
  "_clusters": {
    "skipped": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "running": 42.0,
    "partial": 42.0,
    "failed": 42.0,
    "details": {
      "additionalProperty1": {
        "status": "running",
        "indices": "string",
        "": 42.0,
        "timed_out": true,
        "_shards": {
          "failed": 42.0,
          "successful": 42.0,
          "total": 42.0,
          "failures": [
            {}
          ],
          "skipped": 42.0
        },
        "failures": [
          {
            "index": "string",
            "node": "string",
            "reason": {},
            "shard": 42.0,
            "status": "string"
          }
        ]
      },
      "additionalProperty2": {
        "status": "running",
        "indices": "string",
        "": 42.0,
        "timed_out": true,
        "_shards": {
          "failed": 42.0,
          "successful": 42.0,
          "total": 42.0,
          "failures": [
            {}
          ],
          "skipped": 42.0
        },
        "failures": [
          {
            "index": "string",
            "node": "string",
            "reason": {},
            "shard": 42.0,
            "status": "string"
          }
        ]
      }
    }
  },
  "fields": {
    "additionalProperty1": {},
    "additionalProperty2": {}
  },
  "max_score": 42.0,
  "num_reduce_phases": 42.0,
  "profile": {
    "shards": [
      {
        "aggregations": [
          {
            "breakdown": {},
            "description": "string",
            "type": "string",
            "debug": {},
            "children": [
              {}
            ]
          }
        ],
        "cluster": "string",
        "dfs": {
          "statistics": {
            "type": "string",
            "description": "string",
            "time": "string",
            "breakdown": {},
            "debug": {},
            "children": [
              {}
            ]
          },
          "knn": [
            {}
          ]
        },
        "fetch": {
          "type": "string",
          "description": "string",
          "": 42.0,
          "breakdown": {
            "load_source": 42.0,
            "load_source_count": 42.0,
            "load_stored_fields": 42.0,
            "load_stored_fields_count": 42.0,
            "next_reader": 42.0,
            "next_reader_count": 42.0,
            "process_count": 42.0,
            "process": 42.0
          },
          "debug": {
            "stored_fields": [
              "string"
            ],
            "fast_path": 42.0
          },
          "children": [
            {}
          ]
        },
        "id": "string",
        "index": "string",
        "node_id": "string",
        "searches": [
          {
            "collector": [
              {}
            ],
            "query": [
              {}
            ],
            "rewrite_time": 42.0
          }
        ],
        "shard_id": 42.0
      }
    ]
  },
  "pit_id": "string",
  "_scroll_id": "string",
  "suggest": {
    "additionalProperty1": [
      {
        "length": 42.0,
        "offset": 42.0,
        "text": "string"
      }
    ],
    "additionalProperty2": [
      {
        "length": 42.0,
        "offset": 42.0,
        "text": "string"
      }
    ]
  },
  "terminated_early": true
}

Get tokens from text analysis

POST /_analyze
Api key auth Basic auth Bearer auth

The analyze API performs analysis on a text string and returns the resulting tokens.

Generating excessive amount of tokens may cause a node to run out of memory. The index.analyze.max_token_count setting enables you to limit the number of tokens that can be produced. If more than this limit of tokens gets generated, an error occurs. The _analyze endpoint without a specified index will always use 10000 as its limit.

External documentation

Query parameters

  • index string

    Index used to derive the analyzer. If specified, the analyzer or field parameter overrides this value. If no index is specified or the index does not have a default analyzer, the analyze API uses the standard analyzer.

application/json

Body

Responses

POST /_analyze 
curl \
 --request POST 'http://api.example.com/_analyze' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"analyzer\": \"standard\",\n  \"text\": \"this is a test\"\n}"'
Request examples
You can apply any of the built-in analyzers to the text string without specifying an index. 
{
  "analyzer": "standard",
  "text": "this is a test"
}
If the text parameter is provided as array of strings, it is analyzed as a multi-value field. 
{
  "analyzer": "standard",
  "text": [
    "this is a test",
    "the second text"
  ]
}
You can test a custom transient analyzer built from tokenizers, token filters, and char filters. Token filters use the filter parameter. 
{
  "tokenizer": "keyword",
  "filter": [
    "lowercase"
  ],
  "char_filter": [
    "html_strip"
  ],
  "text": "this is a <b>test</b>"
}
Custom tokenizers, token filters, and character filters can be specified in the request body. 
{
  "tokenizer": "whitespace",
  "filter": [
    "lowercase",
    {
      "type": "stop",
      "stopwords": [
        "a",
        "is",
        "this"
      ]
    }
  ],
  "text": "this is a test"
}
Run `GET /analyze_sample/_analyze` to run an analysis on the text using the default index analyzer associated with the `analyze_sample` index. Alternatively, the analyzer can be derived based on a field mapping. 
{
  "field": "obj1.field1",
  "text": "this is a test"
}
Run `GET /analyze_sample/_analyze` and supply a normalizer for a keyword field if there is a normalizer associated with the specified index. 
{
  "normalizer": "my_normalizer",
  "text": "BaR"
}
If you want to get more advanced details, set `explain` to `true`. It will output all token attributes for each token. You can filter token attributes you want to output by setting the `attributes` option. NOTE: The format of the additional detail information is labelled as experimental in Lucene and it may change in the future. 
{
  "tokenizer": "standard",
  "filter": [
    "snowball"
  ],
  "text": "detailed output",
  "explain": true,
  "attributes": [
    "keyword"
  ]
}
Response examples (200)
A successful response for an analysis with `explain` set to `true`. 
{
  "detail": {
    "custom_analyzer": true,
    "charfilters": [],
    "tokenizer": {
      "name": "standard",
      "tokens": [
        {
          "token": "detailed",
          "start_offset": 0,
          "end_offset": 8,
          "type": "<ALPHANUM>",
          "position": 0
        },
        {
          "token": "output",
          "start_offset": 9,
          "end_offset": 15,
          "type": "<ALPHANUM>",
          "position": 1
        }
      ]
    },
    "tokenfilters": [
      {
        "name": "snowball",
        "tokens": [
          {
            "token": "detail",
            "start_offset": 0,
            "end_offset": 8,
            "type": "<ALPHANUM>",
            "position": 0,
            "keyword": false
          },
          {
            "token": "output",
            "start_offset": 9,
            "end_offset": 15,
            "type": "<ALPHANUM>",
            "position": 1,
            "keyword": false
          }
        ]
      }
    ]
  }
}

Close an index

POST /{index}/_close
Api key auth Basic auth Bearer auth

A closed index is blocked for read or write operations and does not allow all operations that opened indices allow. It is not possible to index documents or to search for documents in a closed index. Closed indices do not have to maintain internal data structures for indexing or searching documents, which results in a smaller overhead on the cluster.

When opening or closing an index, the master node is responsible for restarting the index shards to reflect the new state of the index. The shards will then go through the normal recovery process. The data of opened and closed indices is automatically replicated by the cluster to ensure that enough shard copies are safely kept around at all times.

You can open and close multiple indices. An error is thrown if the request explicitly refers to a missing index. This behaviour can be turned off using the ignore_unavailable=true parameter.

By default, you must explicitly name the indices you are opening or closing. To open or close indices with _all, *, or other wildcard expressions, change theaction.destructive_requires_name setting to false. This setting can also be changed with the cluster update settings API.

Closed indices consume a significant amount of disk-space which can cause problems in managed environments. Closing indices can be turned off with the cluster settings API by setting cluster.indices.close.enable to false.

Path parameters

  • index string | array[string] Required

    Comma-separated list or wildcard expression of index names used to limit the request.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required
    • indices object Required
      Hide indices attribute Show indices attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • closed boolean Required
        • shards object
          Hide shards attribute Show shards attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
    • shards_acknowledged boolean Required
POST /{index}/_close 
curl \
 --request POST 'http://api.example.com/{index}/_close' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response for closing an index. 
{
  "acknowledged": true,
  "shards_acknowledged": true,
  "indices": {
    "my-index-000001": {
      "closed": true
    }
  }
}

Flush data streams or indices

GET /{index}/_flush
Api key auth Basic auth Bearer auth

Flushing a data stream or index is the process of making sure that any data that is currently only stored in the transaction log is also permanently stored in the Lucene index. When restarting, Elasticsearch replays any unflushed operations from the transaction log into the Lucene index to bring it back into the state that it was in before the restart. Elasticsearch automatically triggers flushes as needed, using heuristics that trade off the size of the unflushed transaction log against the cost of performing each flush.

After each operation has been flushed it is permanently stored in the Lucene index. This may mean that there is no need to maintain an additional copy of it in the transaction log. The transaction log is made up of multiple files, called generations, and Elasticsearch will delete any generation files when they are no longer needed, freeing up disk space.

It is also possible to trigger a flush on one or more indices using the flush API, although it is rare for users to need to call this API directly. If you call the flush API after indexing some documents then a successful response indicates that Elasticsearch has flushed all the documents that were indexed before the flush API was called.

Path parameters

  • index string | array[string] Required

    Comma-separated list of data streams, indices, and aliases to flush. Supports wildcards (*). To flush all data streams and indices, omit this parameter or use * or _all.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

  • force boolean

    If true, the request forces a flush even if there are no changes to commit to the index.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • wait_if_ongoing boolean

    If true, the flush operation blocks until execution when another flush operation is running. If false, Elasticsearch returns an error if you request a flush when another flush operation is running.

Responses

GET /{index}/_flush 
curl \
 --request GET 'http://api.example.com/{index}/_flush' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "_shards": {
    "failed": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "failures": [
      {
        "index": "string",
        "node": "string",
        "reason": {
          "type": "string",
          "reason": "string",
          "stack_trace": "string",
          "caused_by": {},
          "root_cause": [
            {}
          ],
          "suppressed": [
            {}
          ]
        },
        "shard": 42.0,
        "status": "string"
      }
    ],
    "skipped": 42.0
  }
}

Shrink an index Added in 5.0.0

POST /{index}/_shrink/{target}
Api key auth Basic auth Bearer auth

Shrink an index into a new index with fewer primary shards.

Before you can shrink an index:

  • The index must be read-only.
  • A copy of every shard in the index must reside on the same node.
  • The index must have a green health status.

To make shard allocation easier, we recommend you also remove the index's replica shards. You can later re-add replica shards as part of the shrink operation.

The requested number of primary shards in the target index must be a factor of the number of shards in the source index. For example an index with 8 primary shards can be shrunk into 4, 2 or 1 primary shards or an index with 15 primary shards can be shrunk into 5, 3 or 1. If the number of shards in the index is a prime number it can only be shrunk into a single primary shard Before shrinking, a (primary or replica) copy of every shard in the index must be present on the same node.

The current write index on a data stream cannot be shrunk. In order to shrink the current write index, the data stream must first be rolled over so that a new write index is created and then the previous write index can be shrunk.

A shrink operation:

  • Creates a new target index with the same definition as the source index, but with a smaller number of primary shards.
  • Hard-links segments from the source index into the target index. If the file system does not support hard-linking, then all segments are copied into the new index, which is a much more time consuming process. Also if using multiple data paths, shards on different data paths require a full copy of segment files if they are not on the same disk since hardlinks do not work across disks.
  • Recovers the target index as though it were a closed index which had just been re-opened. Recovers shards to the .routing.allocation.initial_recovery._id index setting.

IMPORTANT: Indices can only be shrunk if they satisfy the following requirements:

  • The target index must not exist.
  • The source index must have more primary shards than the target index.
  • The number of primary shards in the target index must be a factor of the number of primary shards in the source index. The source index must have more primary shards than the target index.
  • The index must not contain more than 2,147,483,519 documents in total across all shards that will be shrunk into a single shard on the target index as this is the maximum number of docs that can fit into a single shard.
  • The node handling the shrink process must have sufficient free disk space to accommodate a second copy of the existing index.

Path parameters

  • index string Required

    Name of the source index to shrink.

  • target string Required

    Name of the target index to create.

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

  • wait_for_active_shards number | string

    The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1).

application/json

Body

  • aliases object

    The key is the alias name. Index alias names support date math.

    Hide aliases attribute Show aliases attribute object
  • settings object

    Configuration options for the target index.

    Hide settings attribute Show settings attribute object
    • * object Additional properties

Responses

POST /{index}/_shrink/{target} 
curl \
 --request POST 'http://api.example.com/{index}/_shrink/{target}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"settings\": {\n    \"index.routing.allocation.require._name\": null,\n    \"index.blocks.write\": null\n  }\n}"'
Request example 
{
  "settings": {
    "index.routing.allocation.require._name": null,
    "index.blocks.write": null
  }
}
Response examples (200) 
{
  "acknowledged": true,
  "shards_acknowledged": true,
  "index": "string"
}

Create an Azure AI studio inference endpoint Added in 8.14.0

PUT /_inference/{task_type}/{azureaistudio_inference_id}
Api key auth Basic auth Bearer auth

Create an inference endpoint to perform an inference task with the azureaistudio service.

When you create an inference endpoint, the associated machine learning model is automatically deployed if it is not already running. After creating the endpoint, wait for the model deployment to complete before using it. To verify the deployment status, use the get trained model statistics API. Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count". Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.

Path parameters

  • task_type string Required

    The type of the inference task that the model will perform.

    Values are completion or text_embedding.

  • azureaistudio_inference_id string Required

    The unique identifier of the inference endpoint.

application/json

Body

  • chunking_settings object
    Hide chunking_settings attributes Show chunking_settings attributes object
    • max_chunk_size number

      The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

    • overlap number

      The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

    • sentence_overlap number

      The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

    • strategy string

      The chunking strategy: sentence or word.

  • service string Required

    Value is azureaistudio.

  • service_settings object Required
    Hide service_settings attributes Show service_settings attributes object
    • api_key string Required

      A valid API key of your Azure AI Studio model deployment. This key can be found on the overview page for your deployment in the management section of your Azure AI Studio account.

      IMPORTANT: You need to provide the API key only once, during the inference model creation. The get inference endpoint API does not retrieve your API key. After creating the inference model, you cannot change the associated API key. If you want to use a different API key, delete the inference model and recreate it with the same name and the updated API key.

      External documentation
    • endpoint_type string Required

      The type of endpoint that is available for deployment through Azure AI Studio: token or realtime. The token endpoint type is for "pay as you go" endpoints that are billed per token. The realtime endpoint type is for "real-time" endpoints that are billed per hour of usage.

      External documentation
    • target string Required

      The target URL of your Azure AI Studio model deployment. This can be found on the overview page for your deployment in the management section of your Azure AI Studio account.

    • provider string Required

      The model provider for your deployment. Note that some providers may support only certain task types. Supported providers include:

      • cohere - available for text_embedding and completion task types
      • databricks - available for completion task type only
      • meta - available for completion task type only
      • microsoft_phi - available for completion task type only
      • mistral - available for completion task type only
      • openai - available for text_embedding and completion task types
    • rate_limit object
      Hide rate_limit attribute Show rate_limit attribute object
  • task_settings object
    Hide task_settings attributes Show task_settings attributes object
    • do_sample number

      For a completion task, instruct the inference process to perform sampling. It has no effect unless temperature or top_p is specified.

    • max_new_tokens number

      For a completion task, provide a hint for the maximum number of output tokens to be generated.

    • temperature number

      For a completion task, control the apparent creativity of generated completions with a sampling temperature. It must be a number in the range of 0.0 to 2.0. It should not be used if top_p is specified.

    • top_p number

      For a completion task, make the model consider the results of the tokens with nucleus sampling probability. It is an alternative value to temperature and must be a number in the range of 0.0 to 2.0. It should not be used if temperature is specified.

    • user string

      For a text_embedding task, specify the user issuing the request. This information can be used for abuse detection.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • chunking_settings object
      Hide chunking_settings attributes Show chunking_settings attributes object
      • max_chunk_size number

        The maximum size of a chunk in words. This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).

      • overlap number

        The number of overlapping words for chunks. It is applicable only to a word chunking strategy. This value cannot be higher than half the max_chunk_size value.

      • sentence_overlap number

        The number of overlapping sentences for chunks. It is applicable only for a sentence chunking strategy. It can be either 1 or 0.

      • strategy string

        The chunking strategy: sentence or word.

    • service string Required

      The service type

    • service_settings object Required
    • task_settings object
    • inference_id string Required

      The inference Id

    • task_type string Required

      Values are sparse_embedding, text_embedding, rerank, completion, or chat_completion.
PUT /_inference/{task_type}/{azureaistudio_inference_id} 
curl \
 --request PUT 'http://api.example.com/_inference/{task_type}/{azureaistudio_inference_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n    \"service\": \"azureaistudio\",\n    \"service_settings\": {\n        \"api_key\": \"Azure-AI-Studio-API-key\",\n        \"target\": \"Target-Uri\",\n        \"provider\": \"openai\",\n        \"endpoint_type\": \"token\"\n    }\n}"'
Request examples
Run `PUT _inference/text_embedding/azure_ai_studio_embeddings` to create an inference endpoint that performs a text_embedding task. Note that you do not specify a model here, as it is defined already in the Azure AI Studio deployment. 
{
    "service": "azureaistudio",
    "service_settings": {
        "api_key": "Azure-AI-Studio-API-key",
        "target": "Target-Uri",
        "provider": "openai",
        "endpoint_type": "token"
    }
}
Run `PUT _inference/completion/azure_ai_studio_completion` to create an inference endpoint that performs a completion task. 
{
    "service": "azureaistudio",
    "service_settings": {
        "api_key": "Azure-AI-Studio-API-key",
        "target": "Target-URI",
        "provider": "databricks",
        "endpoint_type": "realtime"
    }
}
Response examples (200) 
{
  "chunking_settings": {
    "max_chunk_size": 42.0,
    "overlap": 42.0,
    "sentence_overlap": 42.0,
    "strategy": "string"
  },
  "service": "string",
  "service_settings": {},
  "task_settings": {},
  "inference_id": "string",
  "task_type": "sparse_embedding"
}

Get IP geolocation database configurations Added in 8.15.0

GET /_ingest/ip_location/database
Api key auth Basic auth Bearer auth

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. A value of -1 indicates that the request should never time out.

Responses

GET /_ingest/ip_location/database 
curl \
 --request GET 'http://api.example.com/_ingest/ip_location/database' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "databases": [
    {
      "id": "string",
      "version": 42.0,
      "": 42.0,
      "database": {
        "name": "string",
        "web": {},
        "local": {
          "type": "string"
        },
        "maxmind": {
          "account_id": "string"
        },
        "ipinfo": {}
      }
    }
  ]
}

Simulate data ingestion Technical preview

POST /_ingest/_simulate
Api key auth Basic auth Bearer auth

Run ingest pipelines against a set of provided documents, optionally with substitute pipeline definitions, to simulate ingesting data into an index.

This API is meant to be used for troubleshooting or pipeline development, as it does not actually index any data into Elasticsearch.

The API runs the default and final pipeline for that index against a set of documents provided in the body of the request. If a pipeline contains a reroute processor, it follows that reroute processor to the new index, running that index's pipelines as well the same way that a non-simulated ingest would. No data is indexed into Elasticsearch. Instead, the transformed document is returned, along with the list of pipelines that have been run and the name of the index where the document would have been indexed if this were not a simulation. The transformed document is validated against the mappings that would apply to this index, and any validation error is reported in the result.

This API differs from the simulate pipeline API in that you specify a single pipeline for that API, and it runs only that one pipeline. The simulate pipeline API is more useful for developing a single pipeline, while the simulate ingest API is more useful for troubleshooting the interaction of the various pipelines that get applied when ingesting into an index.

By default, the pipeline definitions that are currently in the system are used. However, you can supply substitute pipeline definitions in the body of the request. These will be used in place of the pipeline definitions that are already in the system. This can be used to replace existing pipeline definitions or to create new ones. The pipeline substitutions are used only within this request.

Query parameters

  • pipeline string

    The pipeline to use as the default pipeline. This value can be used to override the default pipeline of the index.

application/json

Body Required

  • docs array[object] Required

    Sample documents to test in the pipeline.

    Hide docs attributes Show docs attributes object
  • component_template_substitutions object

    A map of component template names to substitute component template definition objects.

    Hide component_template_substitutions attribute Show component_template_substitutions attribute object
  • index_template_substitutions object

    A map of index template names to substitute index template definition objects.

    Hide index_template_substitutions attribute Show index_template_substitutions attribute object
  • mapping_addition object
    Hide mapping_addition attributes Show mapping_addition attributes object
  • pipeline_substitutions object

    Pipelines to test. If you don’t specify the pipeline request path parameter, this parameter is required. If you specify both this and the request path parameter, the API only uses the request path parameter.

    Hide pipeline_substitutions attribute Show pipeline_substitutions attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • description string

        Description of the ingest pipeline.

      • on_failure array[object]

        Processors to run immediately after a processor failure.

        Hide on_failure attributes Show on_failure attributes object
        • append object
          Hide append attributes Show append attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • value
          • allow_duplicates boolean

            If false, the processor does not append values already present in the field.

        • attachment object
          Hide attachment attributes Show attachment attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • indexed_chars number

            The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

          • indexed_chars_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • remove_binary boolean

            If true, the binary field will be removed from the document

          • resource_name string

            Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

        • bytes object
          Hide bytes attributes Show bytes attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • circle object
          Hide circle attributes Show circle attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • error_distance number Required

            The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • shape_type string Required

            Values are geo_shape or shape.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • community_id object
          Hide community_id attributes Show community_id attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • source_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • iana_number string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_type string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_code string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • transport string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • seed number

            Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • convert object
          Hide convert attributes Show convert attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • type string Required

            Values are integer, long, double, float, boolean, ip, string, or auto.

        • csv object
          Hide csv attributes Show csv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • empty_value object

            Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • quote string

            Quote used in CSV, has to be single character string.

          • separator string

            Separator used in CSV, has to be single character string.

          • target_fields string | array[string] Required
          • trim boolean

            Trim whitespaces in unquoted fields.

        • date object
          Hide date attributes Show date attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • formats array[string] Required

            An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • locale string

            The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • timezone string

            The timezone to use when parsing the date. Supports template snippets.

          • output_format string

            The format to use when writing the date to target_field. Must be a valid java time pattern.

        • date_index_name object
          Hide date_index_name attributes Show date_index_name attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • date_formats array[string] Required

            An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • date_rounding string Required

            How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • index_name_format string

            The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

          • index_name_prefix string

            A prefix of the index name to be prepended before the printed date. Supports template snippets.

          • locale string

            The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

          • timezone string

            The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

        • dissect object
          Hide dissect attributes Show dissect attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • append_separator string

            The character(s) that separate the appended fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to apply to the field.

        • dot_expander object
          Hide dot_expander attributes Show dot_expander attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • override boolean

            Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

          • path string

            The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

        • drop object
          Hide drop attributes Show drop attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • enrich object
          Hide enrich attributes Show enrich attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • max_matches number

            The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

          • override boolean

            If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • policy_name string Required

            The name of the enrich policy to use.

          • shape_relation string

            Values are intersects, disjoint, within, or contains.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • fail object
          Hide fail attributes Show fail attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • message string Required

            The error message thrown by the processor. Supports template snippets.

        • fingerprint object
          Hide fingerprint attributes Show fingerprint attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • fields string | array[string] Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • salt string

            Salt value for the hash function.

          • method string

            Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

          • ignore_missing boolean

            If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

        • foreach object
          Hide foreach attributes Show foreach attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true, the processor silently exits without changing the document if the field is null or missing.

          • processor object Required
        • ip_location object
          Hide ip_location attributes Show ip_location attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found IP location data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the IP location lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • geo_grid object
          Hide geo_grid attributes Show geo_grid attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            The field to interpret as a geo-tile.= The field format is determined by the tile_type.

          • tile_type string Required

            Values are geotile, geohex, or geohash.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • parent_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • non_children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • precision_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_format string

            Values are geojson or wkt.

        • geoip object
          Hide geoip attributes Show geoip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found geoip data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the geoip lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • grok object
          Hide grok attributes Show grok attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • ecs_compatibility string

            Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern_definitions object

            A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          • patterns array[string] Required

            An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

          • trace_match boolean

            When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

        • gsub object
          Hide gsub attributes Show gsub attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to be replaced.

          • replacement string Required

            The string to replace the matching patterns with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • html_strip object
          Hide html_strip attributes Show html_strip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document,

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • inference object
          Hide inference attributes Show inference attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • model_id string Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_map object

            Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          • inference_config object
          • input_output
          • ignore_missing boolean

            If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

        • join object
          Hide join attributes Show join attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • separator string Required

            The separator character.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • json object
          Hide json attributes Show json attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • add_to_root boolean

            Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

          • add_to_root_conflict_strategy string

            Values are replace or merge.

          • allow_duplicate_keys boolean

            When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • kv object
          Hide kv attributes Show kv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • exclude_keys array[string]

            List of keys to exclude from document.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_split string Required

            Regex pattern to use for splitting key-value pairs.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • include_keys array[string]

            List of keys to filter and insert into document. Defaults to including all keys.

          • prefix string

            Prefix to be added to extracted keys.

          • strip_brackets boolean

            If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • trim_key string

            String of characters to trim from extracted keys.

          • trim_value string

            String of characters to trim from extracted values.

          • value_split string Required

            Regex pattern to use for splitting the key from the value within a key-value pair.

        • lowercase object
          Hide lowercase attributes Show lowercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • network_direction object
          Hide network_direction attributes Show network_direction attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • internal_networks array[string]

            List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

          • internal_networks_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • pipeline object
          Hide pipeline attributes Show pipeline attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • name string Required
          • ignore_missing_pipeline boolean

            Whether to ignore missing pipelines instead of failing.

        • redact object
          Hide redact attributes Show redact attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • patterns array[string] Required

            A list of grok expressions to match and redact named captures with

          • pattern_definitions object
          • prefix string

            Start a redacted section with this token

          • suffix string

            End a redacted section with this token

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • skip_if_unlicensed boolean

            If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

          • trace_redact boolean

            If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

        • registered_domain object
          Hide registered_domain attributes Show registered_domain attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • remove object
          Hide remove attributes Show remove attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string | array[string] Required
          • keep string | array[string]
          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • rename object
          Hide rename attributes Show rename attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • reroute object
          Hide reroute attributes Show reroute attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • destination string

            A static value for the target. Can’t be set when the dataset or namespace option is set.

          • dataset
          • namespace
        • script object
          Hide script attributes Show script attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • id string
          • lang string

            Script language.

          • params object

            Object containing parameters for the script.

          • source string

            Inline script. If no id is specified, this parameter is required.

        • set object
          Hide set attributes Show set attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • copy_from string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_empty_value boolean

            If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

          • media_type string

            The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

          • override boolean

            If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • value object

            The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

        • set_security_user object
          Hide set_security_user attributes Show set_security_user attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what user related properties are added to the field.

        • sort object
          Hide sort attributes Show sort attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • order string

            Values are asc or desc.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • split object
          Hide split attributes Show split attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • preserve_trailing boolean

            Preserves empty trailing fields, if any.

          • separator string Required

            A regex which matches the separator, for example, , or \s+.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • terminate object
          Hide terminate attributes Show terminate attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • trim object
          Hide trim attributes Show trim attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uppercase object
          Hide uppercase attributes Show uppercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • urldecode object
          Hide urldecode attributes Show urldecode attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uri_parts object
          Hide uri_parts attributes Show uri_parts attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • keep_original boolean

            If true, the processor copies the unparsed URI to <target_field>.original.

          • remove_if_successful boolean

            If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • user_agent object
          Hide user_agent attributes Show user_agent attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • regex_file string

            The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what properties are added to target_field.

            Values are name, os, device, original, or version.

          • extract_device_type boolean Beta

            Extracts device type from the user agent string on a best-effort basis.

      • processors array[object]

        Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.

        Hide processors attributes Show processors attributes object
        • append object
          Hide append attributes Show append attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • value
          • allow_duplicates boolean

            If false, the processor does not append values already present in the field.

        • attachment object
          Hide attachment attributes Show attachment attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • indexed_chars number

            The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

          • indexed_chars_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • remove_binary boolean

            If true, the binary field will be removed from the document

          • resource_name string

            Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

        • bytes object
          Hide bytes attributes Show bytes attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • circle object
          Hide circle attributes Show circle attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • error_distance number Required

            The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • shape_type string Required

            Values are geo_shape or shape.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • community_id object
          Hide community_id attributes Show community_id attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • source_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_port string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • iana_number string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_type string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • icmp_code string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • transport string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • seed number

            Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • convert object
          Hide convert attributes Show convert attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • type string Required

            Values are integer, long, double, float, boolean, ip, string, or auto.

        • csv object
          Hide csv attributes Show csv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • empty_value object

            Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • quote string

            Quote used in CSV, has to be single character string.

          • separator string

            Separator used in CSV, has to be single character string.

          • target_fields string | array[string] Required
          • trim boolean

            Trim whitespaces in unquoted fields.

        • date object
          Hide date attributes Show date attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • formats array[string] Required

            An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • locale string

            The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • timezone string

            The timezone to use when parsing the date. Supports template snippets.

          • output_format string

            The format to use when writing the date to target_field. Must be a valid java time pattern.

        • date_index_name object
          Hide date_index_name attributes Show date_index_name attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • date_formats array[string] Required

            An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

          • date_rounding string Required

            How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • index_name_format string

            The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

          • index_name_prefix string

            A prefix of the index name to be prepended before the printed date. Supports template snippets.

          • locale string

            The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

          • timezone string

            The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

        • dissect object
          Hide dissect attributes Show dissect attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • append_separator string

            The character(s) that separate the appended fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to apply to the field.

        • dot_expander object
          Hide dot_expander attributes Show dot_expander attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • override boolean

            Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

          • path string

            The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

        • drop object
          Hide drop attributes Show drop attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • enrich object
          Hide enrich attributes Show enrich attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • max_matches number

            The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

          • override boolean

            If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • policy_name string Required

            The name of the enrich policy to use.

          • shape_relation string

            Values are intersects, disjoint, within, or contains.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • fail object
          Hide fail attributes Show fail attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • message string Required

            The error message thrown by the processor. Supports template snippets.

        • fingerprint object
          Hide fingerprint attributes Show fingerprint attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • fields string | array[string] Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • salt string

            Salt value for the hash function.

          • method string

            Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

          • ignore_missing boolean

            If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

        • foreach object
          Hide foreach attributes Show foreach attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true, the processor silently exits without changing the document if the field is null or missing.

          • processor object Required
        • ip_location object
          Hide ip_location attributes Show ip_location attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found IP location data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the IP location lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • geo_grid object
          Hide geo_grid attributes Show geo_grid attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            The field to interpret as a geo-tile.= The field format is determined by the tile_type.

          • tile_type string Required

            Values are geotile, geohex, or geohash.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • parent_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • non_children_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • precision_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_format string

            Values are geojson or wkt.

        • geoip object
          Hide geoip attributes Show geoip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • database_file string

            The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • first_only boolean

            If true, only the first found geoip data will be returned, even if the field contains an array.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • properties array[string]

            Controls what properties are added to the target_field based on the geoip lookup.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • download_database_on_pipeline_creation boolean

            If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

        • grok object
          Hide grok attributes Show grok attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • ecs_compatibility string

            Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern_definitions object

            A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          • patterns array[string] Required

            An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

          • trace_match boolean

            When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

        • gsub object
          Hide gsub attributes Show gsub attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • pattern string Required

            The pattern to be replaced.

          • replacement string Required

            The string to replace the matching patterns with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • html_strip object
          Hide html_strip attributes Show html_strip attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document,

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • inference object
          Hide inference attributes Show inference attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • model_id string Required
          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_map object

            Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          • inference_config object
          • input_output
          • ignore_missing boolean

            If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.

        • join object
          Hide join attributes Show join attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • separator string Required

            The separator character.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • json object
          Hide json attributes Show json attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • add_to_root boolean

            Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

          • add_to_root_conflict_strategy string

            Values are replace or merge.

          • allow_duplicate_keys boolean

            When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • kv object
          Hide kv attributes Show kv attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • exclude_keys array[string]

            List of keys to exclude from document.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field_split string Required

            Regex pattern to use for splitting key-value pairs.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • include_keys array[string]

            List of keys to filter and insert into document. Defaults to including all keys.

          • prefix string

            Prefix to be added to extracted keys.

          • strip_brackets boolean

            If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • trim_key string

            String of characters to trim from extracted keys.

          • trim_value string

            String of characters to trim from extracted values.

          • value_split string Required

            Regex pattern to use for splitting the key from the value within a key-value pair.

        • lowercase object
          Hide lowercase attributes Show lowercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • network_direction object
          Hide network_direction attributes Show network_direction attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • source_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • destination_ip string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • internal_networks array[string]

            List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

          • internal_networks_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • pipeline object
          Hide pipeline attributes Show pipeline attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • name string Required
          • ignore_missing_pipeline boolean

            Whether to ignore missing pipelines instead of failing.

        • redact object
          Hide redact attributes Show redact attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • patterns array[string] Required

            A list of grok expressions to match and redact named captures with

          • pattern_definitions object
          • prefix string

            Start a redacted section with this token

          • suffix string

            End a redacted section with this token

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • skip_if_unlicensed boolean

            If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

          • trace_redact boolean

            If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

        • registered_domain object
          Hide registered_domain attributes Show registered_domain attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and any required fields are missing, the processor quietly exits without modifying the document.

        • remove object
          Hide remove attributes Show remove attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string | array[string] Required
          • keep string | array[string]
          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • rename object
          Hide rename attributes Show rename attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • reroute object
          Hide reroute attributes Show reroute attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • destination string

            A static value for the target. Can’t be set when the dataset or namespace option is set.

          • dataset
          • namespace
        • script object
          Hide script attributes Show script attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • id string
          • lang string

            Script language.

          • params object

            Object containing parameters for the script.

          • source string

            Inline script. If no id is specified, this parameter is required.

        • set object
          Hide set attributes Show set attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • copy_from string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_empty_value boolean

            If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

          • media_type string

            The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

          • override boolean

            If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

          • value object

            The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

        • set_security_user object
          Hide set_security_user attributes Show set_security_user attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what user related properties are added to the field.

        • sort object
          Hide sort attributes Show sort attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • order string

            Values are asc or desc.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • split object
          Hide split attributes Show split attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • preserve_trailing boolean

            Preserves empty trailing fields, if any.

          • separator string Required

            A regex which matches the separator, for example, , or \s+.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • terminate object
          Hide terminate attributes Show terminate attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

        • trim object
          Hide trim attributes Show trim attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uppercase object
          Hide uppercase attributes Show uppercase attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • urldecode object
          Hide urldecode attributes Show urldecode attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist or is null, the processor quietly exits without modifying the document.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • uri_parts object
          Hide uri_parts attributes Show uri_parts attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • keep_original boolean

            If true, the processor copies the unparsed URI to <target_field>.original.

          • remove_if_successful boolean

            If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • user_agent object
          Hide user_agent attributes Show user_agent attributes object
          • description string

            Description of the processor. Useful for describing the purpose of the processor or its configuration.

          • if string

            Conditionally execute the processor.

          • ignore_failure boolean

            Ignore failures for the processor.

          • on_failure array[object]

            Handle failures for the processor.

          • tag string

            Identifier for the processor. Useful for debugging and metrics.

          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • ignore_missing boolean

            If true and field does not exist, the processor quietly exits without modifying the document.

          • regex_file string

            The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

          • target_field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • properties array[string]

            Controls what properties are added to target_field.

            Values are name, os, device, original, or version.

          • extract_device_type boolean Beta

            Extracts device type from the user agent string on a best-effort basis.

      • version number
      • deprecated boolean

        Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required
      Hide docs attribute Show docs attribute object
      • doc object
        Hide doc attributes Show doc attributes object
        • _id string Required
        • _index string Required
        • _source object Required

          JSON body for the document.

          Hide _source attribute Show _source attribute object
          • * object Additional properties

        • _version number | string Required

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          VersionNumber number StringifiedVersionNumber string
        • executed_pipelines array[string] Required

          A list of the names of the pipelines executed on this document.

        • ignored_fields array[object]

          A list of the fields that would be ignored at the indexing step. For example, a field whose value is larger than the allowed limit would make it through all of the pipelines, but would not be indexed into Elasticsearch.

          Hide ignored_fields attribute Show ignored_fields attribute object
          • * string Additional properties
        • error object
          Hide error attributes Show error attributes object
POST /_ingest/_simulate 
curl \
 --request POST 'http://api.example.com/_ingest/_simulate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"docs\": [\n    {\n      \"_id\": 123,\n      \"_index\": \"my-index\",\n      \"_source\": {\n        \"foo\": \"bar\"\n      }\n    },\n    {\n      \"_id\": 456,\n      \"_index\": \"my-index\",\n      \"_source\": {\n        \"foo\": \"rab\"\n      }\n    }\n  ]\n}"'
Request examples
In this example the index `my-index` has a default pipeline called `my-pipeline` and a final pipeline called `my-final-pipeline`. Since both documents are being ingested into `my-index`, both pipelines are run using the pipeline definitions that are already in the system. 
{
  "docs": [
    {
      "_id": 123,
      "_index": "my-index",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_id": 456,
      "_index": "my-index",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}
In this example the index `my-index` has a default pipeline called `my-pipeline` and a final pipeline called `my-final-pipeline`. But a substitute definition of `my-pipeline` is provided in `pipeline_substitutions`. The substitute `my-pipeline` will be used in place of the `my-pipeline` that is in the system, and then the `my-final-pipeline` that is already defined in the system will run. 
{
  "docs": [
    {
      "_index": "my-index",
      "_id": 123,
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "my-index",
      "_id": 456,
      "_source": {
        "foo": "rab"
      }
    }
  ],
  "pipeline_substitutions": {
    "my-pipeline": {
      "processors": [
        {
          "uppercase": {
            "field": "foo"
          }
        }
      ]
    }
  }
}
In this example, imagine that the index `my-index` has a strict mapping with only the `foo` keyword field defined. Say that field mapping came from a component template named `my-mappings-template`. You want to test adding a new field, `bar`. So a substitute definition of `my-mappings-template` is provided in `component_template_substitutions`. The substitute `my-mappings-template` will be used in place of the existing mapping for `my-index` and in place of the `my-mappings-template` that is in the system. 
{
  "docs": [
    {
      "_index": "my-index",
      "_id": "123",
      "_source": {
        "foo": "foo"
      }
    },
    {
      "_index": "my-index",
      "_id": "456",
      "_source": {
        "bar": "rab"
      }
    }
  ],
  "component_template_substitutions": {
    "my-mappings_template": {
      "template": {
        "mappings": {
          "dynamic": "strict",
          "properties": {
            "foo": {
              "type": "keyword"
            },
            "bar": {
              "type": "keyword"
            }
          }
        }
      }
    }
  }
}
The pipeline, component template, and index template substitutions replace the existing pipeline details for the duration of this request. 
{
  "docs": [
    {
      "_id": "id",
      "_index": "my-index",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_id": "id",
      "_index": "my-index",
      "_source": {
        "foo": "rab"
      }
    }
  ],
  "pipeline_substitutions": {
    "my-pipeline": {
      "processors": [
        {
          "set": {
            "field": "field3",
            "value": "value3"
          }
        }
      ]
    }
  },
  "component_template_substitutions": {
    "my-component-template": {
      "template": {
        "mappings": {
          "dynamic": true,
          "properties": {
            "field3": {
              "type": "keyword"
            }
          }
        },
        "settings": {
          "index": {
            "default_pipeline": "my-pipeline"
          }
        }
      }
    }
  },
  "index_template_substitutions": {
    "my-index-template": {
      "index_patterns": [
        "my-index-*"
      ],
      "composed_of": [
        "component_template_1",
        "component_template_2"
      ]
    }
  },
  "mapping_addition": {
    "dynamic": "strict",
    "properties": {
      "foo": {
        "type": "keyword"
      }
    }
  }
}
Response examples (200)
A successful response when the simulation uses pipeline definitions that are already in the system. 
{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "bar"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": "-3,",
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "rab"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}
A successful response when the simulation uses pipeline substitutions. 
{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "BAR"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "RAB"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}
A successful response when the simulation uses pipeline substitutions. 
{
  "docs": [
    {
      "doc": {
        "_id": "123",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "foo": "foo"
        },
        "executed_pipelines": []
      }
    },
    {
      "doc": {
        "_id": "456",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "bar": "rab"
        },
      "executed_pipelines": []
      }
    }
  ]
}

Get the trial status Added in 6.1.0

GET /_license/trial_status
Api key auth Basic auth Bearer auth

Responses

GET /_license/trial_status 
curl \
 --request GET 'http://api.example.com/_license/trial_status' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_license/trial_status`. 
{
  "eligible_to_start_trial": true
}

Set upgrade_mode for ML indices Added in 6.7.0

POST /_ml/set_upgrade_mode
Api key auth Basic auth Bearer auth

Sets a cluster wide upgrade_mode setting that prepares machine learning indices for an upgrade. When upgrading your cluster, in some circumstances you must restart your nodes and reindex your machine learning indices. In those circumstances, there must be no machine learning jobs running. You can close the machine learning jobs, do the upgrade, then open all the jobs again. Alternatively, you can use this API to temporarily halt tasks associated with the jobs and datafeeds and prevent new jobs from opening. You can also use this API during upgrades that do not require you to reindex your machine learning indices, though stopping jobs is not a requirement in that case. You can see the current value for the upgrade_mode setting by using the get machine learning info API.

Query parameters

  • enabled boolean

    When true, it enables upgrade_mode which temporarily halts all job and datafeed tasks and prohibits new job and datafeed tasks from starting.

  • timeout string

    The time to wait for the request to be completed.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
POST /_ml/set_upgrade_mode 
curl \
 --request POST 'http://api.example.com/_ml/set_upgrade_mode' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "acknowledged": true
}

Get info about events in calendars Added in 6.2.0

GET /_ml/calendars/{calendar_id}/events
Api key auth Basic auth Bearer auth

Path parameters

  • calendar_id string Required

    A string that uniquely identifies a calendar. You can get information for multiple calendars by using a comma-separated list of ids or a wildcard expression. You can get information for all calendars by using _all or * or by omitting the calendar identifier.

Query parameters

  • end string | number

    Specifies to get events with timestamps earlier than this time.

  • from number

    Skips the specified number of events.

  • job_id string

    Specifies to get events for a specific anomaly detection job identifier or job group. It must be used with a calendar identifier of _all or *.

  • size number

    Specifies the maximum number of events to obtain.

  • start string | number

    Specifies to get events with timestamps after this time.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • events array[object] Required
      Hide events attributes Show events attributes object
      • calendar_id string
      • event_id string
      • description string Required

        A description of the scheduled event.

      • end_time string | number Required

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
        DateTime string UnitMillis number

      • start_time string | number Required

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
        DateTime string UnitMillis number
GET /_ml/calendars/{calendar_id}/events 
curl \
 --request GET 'http://api.example.com/_ml/calendars/{calendar_id}/events' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "count": 42.0,
  "events": [
    {
      "calendar_id": "string",
      "event_id": "string",
      "description": "string",
      "": "string"
    }
  ]
}

Get anomaly detection job results for categories Added in 5.4.0

GET /_ml/anomaly_detectors/{job_id}/results/categories/{category_id}
Api key auth Basic auth Bearer auth

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

  • category_id string Required

    Identifier for the category, which is unique in the job. If you specify neither the category ID nor the partition_field_value, the API returns information about all categories. If you specify only the partition_field_value, it returns information about all categories for the specified partition.

Query parameters

  • from number

    Skips the specified number of categories.

  • partition_field_value string

    Only return categories for the specified partition.

  • size number

    Specifies the maximum number of categories to obtain.

application/json

Body

  • page object
    Hide page attributes Show page attributes object
    • from number

      Skips the specified number of items.

    • size number

      Specifies the maximum number of items to obtain.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • categories array[object] Required
      Hide categories attributes Show categories attributes object
      • category_id number Required
      • examples array[string] Required

        A list of examples of actual values that matched the category.

      • grok_pattern string
      • job_id string Required
      • max_matching_length number Required
      • partition_field_name string

        If per-partition categorization is enabled, this property identifies the field used to segment the categorization. It is not present when per-partition categorization is disabled.

      • partition_field_value string

        If per-partition categorization is enabled, this property identifies the value of the partition_field_name for the category. It is not present when per-partition categorization is disabled.

      • regex string Required

        A regular expression that is used to search for values that match the category.

      • terms string Required

        A space separated list of the common tokens that are matched in values of the category.

      • num_matches number

        The number of messages that have been matched by this category. This is only guaranteed to have the latest accurate count after a job _flush or _close

      • preferred_to_categories array[string]

        A list of category_id entries that this current category encompasses. Any new message that is processed by the categorizer will match against this category and not any of the categories in this list. This is only guaranteed to have the latest accurate list of categories after a job _flush or _close

      • p string
      • result_type string Required
      • mlcategory string Required
    • count number Required
GET /_ml/anomaly_detectors/{job_id}/results/categories/{category_id} 
curl \
 --request GET 'http://api.example.com/_ml/anomaly_detectors/{job_id}/results/categories/{category_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"page":{"from":42.0,"size":42.0}}'
Request examples 
{
  "page": {
    "from": 42.0,
    "size": 42.0
  }
}
Response examples (200) 
{
  "categories": [
    {
      "category_id": 42.0,
      "examples": [
        "string"
      ],
      "grok_pattern": "string",
      "job_id": "string",
      "max_matching_length": 42.0,
      "partition_field_name": "string",
      "partition_field_value": "string",
      "regex": "string",
      "terms": "string",
      "num_matches": 42.0,
      "preferred_to_categories": [
        "string"
      ],
      "p": "string",
      "result_type": "string",
      "mlcategory": "string"
    }
  ],
  "count": 42.0
}

Get datafeeds usage info Added in 5.5.0

GET /_ml/datafeeds/{datafeed_id}/_stats
Api key auth Basic auth Bearer auth

You can get statistics for multiple datafeeds in a single API request by using a comma-separated list of datafeeds or a wildcard expression. You can get statistics for all datafeeds by using _all, by specifying * as the <feed_id>, or by omitting the <feed_id>. If the datafeed is stopped, the only information you receive is the datafeed_id and the state. This API returns a maximum of 10,000 datafeeds.

Path parameters

  • datafeed_id string | array[string] Required

    Identifier for the datafeed. It can be a datafeed identifier or a wildcard expression. If you do not specify one of these options, the API returns information about all datafeeds.

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    1. Contains wildcard expressions and there are no datafeeds that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    The default value is true, which returns an empty datafeeds array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
GET /_ml/datafeeds/{datafeed_id}/_stats 
curl \
 --request GET 'http://api.example.com/_ml/datafeeds/{datafeed_id}/_stats' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "count": 42.0,
  "datafeeds": [
    {
      "assignment_explanation": "string",
      "datafeed_id": "string",
      "node": {
        "name": "string",
        "ephemeral_id": "string",
        "id": "string",
        "transport_address": "string",
        "attributes": {
          "additionalProperty1": "string",
          "additionalProperty2": "string"
        }
      },
      "state": "started",
      "timing_stats": {
        "bucket_count": 42.0,
        "": 42.0,
        "exponential_average_calculation_context": {
          "": 42.0
        },
        "job_id": "string",
        "search_count": 42.0
      },
      "running_state": {
        "real_time_configured": true,
        "real_time_running": true,
        "search_interval": {
          "end": "string",
          "": 42.0,
          "start": "string"
        }
      }
    }
  ]
}

Revert to a snapshot Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_revert
Api key auth Basic auth Bearer auth

The machine learning features react quickly to anomalous input, learning new behaviors in data. Highly anomalous input increases the variance in the models whilst the system learns whether this is a new step-change in behavior or a one-off event. In the case where this anomalous input is known to be a one-off, then it might be appropriate to reset the model state to a time before this event. For example, you might consider reverting to a saved snapshot after Black Friday or a critical system failure.

Path parameters

  • job_id string Required

    Identifier for the anomaly detection job.

  • snapshot_id string Required

    You can specify empty as the . Reverting to the empty snapshot means the anomaly detection job starts learning a new model from scratch when it is started.

Query parameters

  • delete_intervening_results boolean

    If true, deletes the results in the time period between the latest results and the time of the reverted snapshot. It also resets the model to accept records for this time period. If you choose not to delete intervening results when reverting a snapshot, the job will not accept input data that is older than the current time. If you want to resend data, then delete the intervening results.

application/json

Body

Responses

POST /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_revert 
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}/_revert' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"delete_intervening_results":true}'
Request examples 
{
  "delete_intervening_results": true
}
Response examples (200) 
{
  "model": {
    "description": "string",
    "job_id": "string",
    "latest_record_time_stamp": 42.0,
    "latest_result_time_stamp": 42.0,
    "min_version": "string",
    "model_size_stats": {
      "bucket_allocation_failures_count": 42.0,
      "job_id": "string",
      "": 42.0,
      "memory_status": "ok",
      "assignment_memory_basis": "string",
      "result_type": "string",
      "total_by_field_count": 42.0,
      "total_over_field_count": 42.0,
      "total_partition_field_count": 42.0,
      "categorization_status": "ok",
      "categorized_doc_count": 42.0,
      "dead_category_count": 42.0,
      "failed_category_count": 42.0,
      "frequent_category_count": 42.0,
      "rare_category_count": 42.0,
      "total_category_count": 42.0,
      "timestamp": 42.0
    },
    "retain": true,
    "snapshot_doc_count": 42.0,
    "snapshot_id": "string",
    "timestamp": 42.0
  }
}

Validate an anomaly detection job Added in 5.4.0

POST /_ml/anomaly_detectors/_validate/detector
Api key auth Basic auth Bearer auth
application/json

Body Required

  • by_field_name string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • custom_rules array[object]

    Custom rules enable you to customize the way detectors operate. For example, a rule may dictate conditions under which results should be skipped. Kibana refers to custom rules as job rules.

    Hide custom_rules attributes Show custom_rules attributes object
    • actions array[string]

      The set of actions to be triggered when the rule applies. If more than one action is specified the effects of all actions are combined.

      Values are skip_result or skip_model_update.

    • conditions array[object]

      An array of numeric conditions when the rule applies. A rule must either have a non-empty scope or at least one condition. Multiple conditions are combined together with a logical AND.

      Hide conditions attributes Show conditions attributes object
      • applies_to string Required

        Values are actual, typical, diff_from_typical, or time.

      • operator string Required

        Values are gt, gte, lt, or lte.

      • value number Required

        The value that is compared against the applies_to field using the operator.

    • scope object

      A scope of series where the rule applies. A rule must either have a non-empty scope or at least one condition. By default, the scope includes all series. Scoping is allowed for any of the fields that are also specified in by_field_name, over_field_name, or partition_field_name.

      Hide scope attribute Show scope attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
  • detector_description string

    A description of the detector.

  • detector_index number

    A unique identifier for the detector. This identifier is based on the order of the detectors in the analysis_config, starting at zero. If you specify a value for this property, it is ignored.

  • exclude_frequent string

    Values are all, none, by, or over.

  • field_name string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • function string

    The analysis function that is used. For example, count, rare, mean, min, max, or sum.

  • over_field_name string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • partition_field_name string

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • use_null boolean

    Defines whether a new series is used as the null series when there is no value for the by or partition fields.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
POST /_ml/anomaly_detectors/_validate/detector 
curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/_validate/detector' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"by_field_name":"string","custom_rules":[{"actions":["skip_result"],"conditions":[{"applies_to":"actual","operator":"gt","value":42.0}],"scope":{"additionalProperty1":{"filter_id":"string","filter_type":"include"},"additionalProperty2":{"filter_id":"string","filter_type":"include"}}}],"detector_description":"string","detector_index":42.0,"exclude_frequent":"all","field_name":"string","function":"string","over_field_name":"string","partition_field_name":"string","use_null":true}'
Request examples 
{
  "by_field_name": "string",
  "custom_rules": [
    {
      "actions": [
        "skip_result"
      ],
      "conditions": [
        {
          "applies_to": "actual",
          "operator": "gt",
          "value": 42.0
        }
      ],
      "scope": {
        "additionalProperty1": {
          "filter_id": "string",
          "filter_type": "include"
        },
        "additionalProperty2": {
          "filter_id": "string",
          "filter_type": "include"
        }
      }
    }
  ],
  "detector_description": "string",
  "detector_index": 42.0,
  "exclude_frequent": "all",
  "field_name": "string",
  "function": "string",
  "over_field_name": "string",
  "partition_field_name": "string",
  "use_null": true
}
Response examples (200) 
{
  "acknowledged": true
}

Get trained model configuration info Added in 7.10.0

GET /_ml/trained_models
Api key auth Basic auth Bearer auth

Query parameters

  • allow_no_match boolean

    Specifies what to do when the request:

    • Contains wildcard expressions and there are no models that match.
    • Contains the _all string or no identifiers and there are no matches.
    • Contains wildcard expressions and there are only partial matches.

    If true, it returns an empty array when there are no matches and the subset of results when there are partial matches.

  • decompress_definition boolean

    Specifies whether the included model definition should be returned as a JSON map (true) or in a custom compressed format (false).

  • exclude_generated boolean

    Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.

  • from number

    Skips the specified number of models.

  • include string

    A comma delimited string of optional fields to include in the response body.

    Values are definition, feature_importance_baseline, hyperparameters, total_feature_importance, or definition_status.

  • include_model_definition boolean Deprecated

    parameter is deprecated! Use [include=definition] instead

  • size number

    Specifies the maximum number of models to obtain.

  • tags string | array[string]

    A comma delimited string of tags. A trained model can have many tags, or none. When supplied, only trained models that contain all the supplied tags are returned.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • trained_model_configs array[object] Required

      An array of trained model resources, which are sorted by the model_id value in ascending order.

      Hide trained_model_configs attributes Show trained_model_configs attributes object
      • model_id string Required
      • model_type string

        Values are tree_ensemble, lang_ident, or pytorch.

      • tags array[string] Required

        A comma delimited string of tags. A trained model can have many tags, or none.

      • version string
      • compressed_definition string
      • created_by string

        Information on the creator of the trained model.

      • create_time string | number

        A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

        One of:
        DateTime string UnitMillis number
      • default_field_map object

        Any field map described in the inference configuration takes precedence.

        Hide default_field_map attribute Show default_field_map attribute object
        • * string Additional properties
      • description string

        The free-text description of the trained model.

      • estimated_heap_memory_usage_bytes number

        The estimated heap usage in bytes to keep the trained model in memory.

      • estimated_operations number

        The estimated number of operations to use the trained model.

      • fully_defined boolean

        True if the full model definition is present.

      • inference_config object

        Inference configuration provided when storing the model config

        Hide inference_config attributes Show inference_config attributes object
        • regression object
          Hide regression attributes Show regression attributes object
        • classification object
          Hide classification attributes Show classification attributes object
          • num_top_classes number

            Specifies the number of top class predictions to return. Defaults to 0.

          • num_top_feature_importance_values number

            Specifies the maximum number of feature importance values per document.

          • prediction_field_type string

            Specifies the type of the predicted field to write. Acceptable values are: string, number, boolean. When boolean is provided 1.0 is transformed to true and 0.0 to false.

          • results_field string

            The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

          • top_classes_results_field string

            Specifies the field to which the top classes are written. Defaults to top_classes.

        • text_classification object
          Hide text_classification attributes Show text_classification attributes object
          • num_top_classes number

            Specifies the number of top class predictions to return. Defaults to 0.

          • tokenization object

            Tokenization options stored in inference configuration

            Hide tokenization attributes Show tokenization attributes object
          • results_field string

            The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

          • classification_labels array[string]

            Classification labels to apply other than the stored labels. Must have the same deminsions as the default configured labels

          • vocabulary object
            Hide vocabulary attribute Show vocabulary attribute object
        • zero_shot_classification object
          Hide zero_shot_classification attributes Show zero_shot_classification attributes object
          • tokenization object

            Tokenization options stored in inference configuration

            Hide tokenization attributes Show tokenization attributes object
          • hypothesis_template string

            Hypothesis template used when tokenizing labels for prediction

          • classification_labels array[string] Required

            The zero shot classification labels indicating entailment, neutral, and contradiction Must contain exactly and only entailment, neutral, and contradiction

          • results_field string

            The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

          • multi_label boolean

            Indicates if more than one true label exists.

          • labels array[string]

            The labels to predict.

        • fill_mask object
          Hide fill_mask attributes Show fill_mask attributes object
          • mask_token string

            The string/token which will be removed from incoming documents and replaced with the inference prediction(s). In a response, this field contains the mask token for the specified model/tokenizer. Each model and tokenizer has a predefined mask token which cannot be changed. Thus, it is recommended not to set this value in requests. However, if this field is present in a request, its value must match the predefined value for that model/tokenizer, otherwise the request will fail.

          • num_top_classes number

            Specifies the number of top class predictions to return. Defaults to 0.

          • tokenization object

            Tokenization options stored in inference configuration

            Hide tokenization attributes Show tokenization attributes object
          • results_field string

            The field that is added to incoming documents to contain the inference prediction. Defaults to predicted_value.

          • vocabulary object Required
            Hide vocabulary attribute Show vocabulary attribute object
        • learning_to_rank object
          Hide learning_to_rank attributes Show learning_to_rank attributes object
        • ner object
          Hide ner attributes Show ner attributes object
        • pass_through object
          Hide pass_through attributes Show pass_through attributes object
        • text_embedding object
          Hide text_embedding attributes Show text_embedding attributes object
        • text_expansion object
          Hide text_expansion attributes Show text_expansion attributes object
        • question_answering object
          Hide question_answering attributes Show question_answering attributes object
      • input object Required
        Hide input attribute Show input attribute object
        • field_names array[string] Required

          An array of input field names for the model.

      • license_level string

        The license level of the trained model.

      • metadata object
        Hide metadata attributes Show metadata attributes object
        • model_aliases array[string]
        • feature_importance_baseline object

          An object that contains the baseline for feature importance values. For regression analysis, it is a single value. For classification analysis, there is a value for each class.

          Hide feature_importance_baseline attribute Show feature_importance_baseline attribute object
          • * string Additional properties
        • hyperparameters array[object]

          List of the available hyperparameters optimized during the fine_parameter_tuning phase as well as specified by the user.

          Hide hyperparameters attributes Show hyperparameters attributes object
          • absolute_importance number

            A positive number showing how much the parameter influences the variation of the loss function. For hyperparameters with values that are not specified by the user but tuned during hyperparameter optimization.

          • name string Required
          • relative_importance number

            A number between 0 and 1 showing the proportion of influence on the variation of the loss function among all tuned hyperparameters. For hyperparameters with values that are not specified by the user but tuned during hyperparameter optimization.

          • supplied boolean Required

            Indicates if the hyperparameter is specified by the user (true) or optimized (false).

          • value number Required

            The value of the hyperparameter, either optimized or specified by the user.

        • total_feature_importance array[object]

          An array of the total feature importance for each feature used from the training data set. This array of objects is returned if data frame analytics trained the model and the request includes total_feature_importance in the include request parameter.

          Hide total_feature_importance attributes Show total_feature_importance attributes object
          • feature_name string Required
          • importance array[object] Required

            A collection of feature importance statistics related to the training data set for this particular feature.

          • classes array[object] Required

            If the trained model is a classification model, feature importance statistics are gathered per target class value.

      • model_size_bytes number | string

        One of:
        ByteSize number ByteSize string
      • model_package object
        Hide model_package attributes Show model_package attributes object
      • location object
        Hide location attribute Show location attribute object
        • index object Required
          Hide index attribute Show index attribute object
      • platform_architecture string
      • prefix_strings object
        Hide prefix_strings attributes Show prefix_strings attributes object
        • ingest string

          String prepended to input at ingest

        • search string

          String prepended to input at search
GET /_ml/trained_models 
curl \
 --request GET 'http://api.example.com/_ml/trained_models' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "count": 42.0,
  "trained_model_configs": [
    {
      "model_id": "string",
      "model_type": "tree_ensemble",
      "tags": [
        "string"
      ],
      "version": "string",
      "compressed_definition": "string",
      "created_by": "string",
      "": 42.0,
      "default_field_map": {
        "additionalProperty1": "string",
        "additionalProperty2": "string"
      },
      "description": "string",
      "estimated_heap_memory_usage_bytes": 42.0,
      "estimated_operations": 42.0,
      "fully_defined": true,
      "inference_config": {
        "regression": {
          "results_field": "string",
          "num_top_feature_importance_values": 42.0
        },
        "classification": {
          "num_top_classes": 42.0,
          "num_top_feature_importance_values": 42.0,
          "prediction_field_type": "string",
          "results_field": "string",
          "top_classes_results_field": "string"
        },
        "text_classification": {
          "num_top_classes": 42.0,
          "tokenization": {},
          "results_field": "string",
          "classification_labels": [
            "string"
          ],
          "vocabulary": {
            "index": "string"
          }
        },
        "zero_shot_classification": {
          "tokenization": {},
          "hypothesis_template": "string",
          "classification_labels": [
            "string"
          ],
          "results_field": "string",
          "multi_label": true,
          "labels": [
            "string"
          ]
        },
        "fill_mask": {
          "mask_token": "string",
          "num_top_classes": 42.0,
          "tokenization": {},
          "results_field": "string",
          "vocabulary": {
            "index": "string"
          }
        },
        "learning_to_rank": {
          "default_params": {
            "additionalProperty1": {},
            "additionalProperty2": {}
          },
          "feature_extractors": [
            {}
          ],
          "num_top_feature_importance_values": 42.0
        },
        "ner": {
          "tokenization": {},
          "results_field": "string",
          "classification_labels": [
            "string"
          ],
          "vocabulary": {
            "index": "string"
          }
        },
        "pass_through": {
          "tokenization": {},
          "results_field": "string",
          "vocabulary": {
            "index": "string"
          }
        },
        "text_embedding": {
          "embedding_size": 42.0,
          "tokenization": {},
          "results_field": "string",
          "vocabulary": {
            "index": "string"
          }
        },
        "text_expansion": {
          "tokenization": {},
          "results_field": "string",
          "vocabulary": {
            "index": "string"
          }
        },
        "question_answering": {
          "num_top_classes": 42.0,
          "tokenization": {},
          "results_field": "string",
          "max_answer_length": 42.0
        }
      },
      "input": {
        "field_names": [
          "string"
        ]
      },
      "license_level": "string",
      "metadata": {
        "model_aliases": [
          "string"
        ],
        "feature_importance_baseline": {
          "additionalProperty1": "string",
          "additionalProperty2": "string"
        },
        "hyperparameters": [
          {
            "absolute_importance": 42.0,
            "name": "string",
            "relative_importance": 42.0,
            "supplied": true,
            "value": 42.0
          }
        ],
        "total_feature_importance": [
          {
            "feature_name": "string",
            "importance": [
              {}
            ],
            "classes": [
              {}
            ]
          }
        ]
      },
      "model_package": {
        "": 42.0,
        "description": "string",
        "inference_config": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "metadata": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "minimum_version": "string",
        "model_repository": "string",
        "model_type": "string",
        "packaged_model_id": "string",
        "platform_architecture": "string",
        "prefix_strings": {
          "ingest": "string",
          "search": "string"
        },
        "sha256": "string",
        "tags": [
          "string"
        ],
        "vocabulary_file": "string"
      },
      "location": {
        "index": {
          "name": "string"
        }
      },
      "platform_architecture": "string",
      "prefix_strings": {
        "ingest": "string",
        "search": "string"
      }
    }
  ]
}

Get the shutdown status Added in 7.13.0

GET /_nodes/{node_id}/shutdown
Api key auth Basic auth Bearer auth

Get information about nodes that are ready to be shut down, have shut down preparations still in progress, or have stalled. The API returns status information for each part of the shut down process.

NOTE: This feature is designed for indirect use by Elasticsearch Service, Elastic Cloud Enterprise, and Elastic Cloud on Kubernetes. Direct use is not supported.

If the operator privileges feature is enabled, you must be an operator to use this API.

Path parameters

  • node_id string | array[string] Required

    Which node for which to retrieve the shutdown status

Query parameters

  • master_timeout string

    Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

    Values are nanos, micros, ms, s, m, h, or d.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • nodes array[object] Required
      Hide nodes attributes Show nodes attributes object
      • node_id string Required
      • type string Required

        Values are remove or restart.

      • reason string Required
      • shutdown_startedmillis number

        Time unit for milliseconds

      • status string Required

        Values are not_started, in_progress, stalled, or complete.

      • shard_migration object Required
        Hide shard_migration attribute Show shard_migration attribute object
        • status string Required

          Values are not_started, in_progress, stalled, or complete.

      • persistent_tasks object Required
        Hide persistent_tasks attribute Show persistent_tasks attribute object
        • status string Required

          Values are not_started, in_progress, stalled, or complete.

      • plugins object Required
        Hide plugins attribute Show plugins attribute object
        • status string Required

          Values are not_started, in_progress, stalled, or complete.
GET /_nodes/{node_id}/shutdown 
curl \
 --request GET 'http://api.example.com/_nodes/{node_id}/shutdown' \
 --header "Authorization: $API_KEY"
Response examples (200)
Get the status of shutdown preparations with `GET /_nodes/USpTGYaBSIKbgSUJR2Z9lg/shutdown`. The response shows information about the shutdown preparations, including the status of shard migration, task migration, and plugin cleanup 
{
    "nodes": [
        {
            "node_id": "USpTGYaBSIKbgSUJR2Z9lg",
            "type": "RESTART",
            "reason": "Demonstrating how the node shutdown API works",
            "shutdown_startedmillis": 1624406108685,
            "allocation_delay": "10m",
            "status": "COMPLETE",
            "shard_migration": {
                "status": "COMPLETE",
                "shard_migrations_remaining": 0,
                "explanation": "no shard relocation is necessary for a node restart"
            },
            "persistent_tasks": {
                "status": "COMPLETE"
            },
            "plugins": {
                "status": "COMPLETE"
            }
        }
    ]
}

Get rollup job information Deprecated Technical preview

GET /_rollup/job/{id}
Api key auth Basic auth Bearer auth

Get the configuration, stats, and status of rollup jobs.

NOTE: This API returns only active (both STARTED and STOPPED) jobs. If a job was created, ran for a while, then was deleted, the API does not return any details about it. For details about a historical rollup job, the rollup capabilities API may be more useful.

Path parameters

  • id string Required

    Identifier for the rollup job. If it is _all or omitted, the API returns all rollup jobs.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • jobs array[object] Required
      Hide jobs attributes Show jobs attributes object
      • config object Required
        Hide config attributes Show config attributes object
        • cron string Required
        • groups object Required
          Hide groups attributes Show groups attributes object
          • date_histogram object
            Hide date_histogram attributes Show date_histogram attributes object
            • delay string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • field string Required

              Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

            • format string
            • interval string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • calendar_interval string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • fixed_interval string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • time_zone string
          • histogram object
            Hide histogram attributes Show histogram attributes object
            • fields string | array[string] Required
            • interval number Required

              The interval of histogram buckets to be generated when rolling up. For example, a value of 5 creates buckets that are five units wide (0-5, 5-10, etc). Note that only one interval can be specified in the histogram group, meaning that all fields being grouped via the histogram must share the same interval.

          • terms object
            Hide terms attribute Show terms attribute object
            • fields string | array[string] Required
        • id string Required
        • index_pattern string Required
        • metrics array[object] Required
          Hide metrics attributes Show metrics attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • metrics array[string] Required

            An array of metrics to collect for the field. At least one metric must be configured.

            Values are min, max, sum, avg, or value_count.

        • page_size number Required
        • rollup_index string Required
        • timeout string Required

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • stats object Required
        Hide stats attributes Show stats attributes object
      • status object Required
        Hide status attributes Show status attributes object
GET /_rollup/job/{id} 
curl \
 --request GET 'http://api.example.com/_rollup/job/{id}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET _rollup/job/sensor`. 
{
  "jobs": [
    {
      "config": {
        "id": "sensor",
        "index_pattern": "sensor-*",
        "rollup_index": "sensor_rollup",
        "cron": "*/30 * * * * ?",
        "groups": {
          "date_histogram": {
            "fixed_interval": "1h",
            "delay": "7d",
            "field": "timestamp",
            "time_zone": "UTC"
          },
          "terms": {
            "fields": [
              "node"
            ]
          }
        },
        "metrics": [
          {
            "field": "temperature",
            "metrics": [
              "min",
              "max",
              "sum"
            ]
          },
          {
            "field": "voltage",
            "metrics": [
              "avg"
            ]
          }
        ],
        "timeout": "20s",
        "page_size": 1000
      },
      "status": {
        "job_state": "stopped"
      },
      "stats": {
        "pages_processed": 0,
        "documents_processed": 0,
        "rollups_indexed": 0,
        "trigger_count": 0,
        "index_failures": 0,
        "index_time_in_ms": 0,
        "index_total": 0,
        "search_failures": 0,
        "search_time_in_ms": 0,
        "search_total": 0,
        "processing_time_in_ms": 0,
        "processing_total": 0
      }
    }
  ]
}

Delete a script or search template

DELETE /_scripts/{id}
Api key auth Basic auth Bearer auth

Deletes a stored script or search template.

Path parameters

  • id string Required

    The identifier for the stored script or search template.

Query parameters

  • master_timeout string

    The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

  • timeout string

    The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. It can also be set to -1 to indicate that the request should never timeout.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
DELETE /_scripts/{id} 
curl \
 --request DELETE 'http://api.example.com/_scripts/{id}' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "acknowledged": true
}

Run a scrolling search

POST /_search/scroll
Api key auth Basic auth Bearer auth

IMPORTANT: The scroll API is no longer recommend for deep pagination. If you need to preserve the index state while paging through more than 10,000 hits, use the search_after parameter with a point in time (PIT).

The scroll API gets large sets of results from a single scrolling search request. To get the necessary scroll ID, submit a search API request that includes an argument for the scroll query parameter. The scroll parameter indicates how long Elasticsearch should retain the search context for the request. The search response returns a scroll ID in the _scroll_id response body parameter. You can then use the scroll ID with the scroll API to retrieve the next batch of results for the request. If the Elasticsearch security features are enabled, the access to the results of a specific scroll ID is restricted to the user or API key that submitted the search.

You can also use the scroll API to specify a new scroll parameter that extends or shortens the retention period for the search context.

IMPORTANT: Results from a scrolling search reflect the state of the index at the time of the initial search request. Subsequent indexing or document changes only affect later search and scroll requests.

External documentation

Query parameters

  • scroll string

    The period to retain the search context for scrolling.

  • scroll_id string Deprecated

    The scroll ID for scrolled search

  • rest_total_hits_as_int boolean

    If true, the API response’s hit.total property is returned as an integer. If false, the API response’s hit.total property is returned as an object.

application/json

Body

  • scroll string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • scroll_id string Required

Responses

POST /_search/scroll 
curl \
 --request POST 'http://api.example.com/_search/scroll' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"scroll_id\" : \"DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ==\"\n}"'
Request example
Run `GET /_search/scroll` to get the next batch of results for a scrolling search. 
{
  "scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ=="
}
Response examples (200) 
{
  "took": 42.0,
  "timed_out": true,
  "_shards": {
    "failed": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "failures": [
      {
        "index": "string",
        "node": "string",
        "reason": {
          "type": "string",
          "reason": "string",
          "stack_trace": "string",
          "caused_by": {},
          "root_cause": [
            {}
          ],
          "suppressed": [
            {}
          ]
        },
        "shard": 42.0,
        "status": "string"
      }
    ],
    "skipped": 42.0
  },
  "hits": {
    "total": {
      "relation": "eq",
      "value": 42.0
    },
    "hits": [
      {
        "_index": "string",
        "_id": "string",
        "_score": 42.0,
        "_explanation": {
          "description": "string",
          "details": [
            {}
          ],
          "value": 42.0
        },
        "fields": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "highlight": {
          "additionalProperty1": [
            "string"
          ],
          "additionalProperty2": [
            "string"
          ]
        },
        "inner_hits": {
          "additionalProperty1": {
            "hits": {}
          },
          "additionalProperty2": {
            "hits": {}
          }
        },
        "matched_queries": [
          "string"
        ],
        "_nested": {
          "field": "string",
          "offset": 42.0,
          "_nested": {}
        },
        "_ignored": [
          "string"
        ],
        "ignored_field_values": {
          "additionalProperty1": [
            {}
          ],
          "additionalProperty2": [
            {}
          ]
        },
        "_shard": "string",
        "_node": "string",
        "_routing": "string",
        "_source": {},
        "_rank": 42.0,
        "_seq_no": 42.0,
        "_primary_term": 42.0,
        "_version": 42.0,
        "sort": [
          42.0
        ]
      }
    ],
    "max_score": 42.0
  },
  "aggregations": {},
  "_clusters": {
    "skipped": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "running": 42.0,
    "partial": 42.0,
    "failed": 42.0,
    "details": {
      "additionalProperty1": {
        "status": "running",
        "indices": "string",
        "": 42.0,
        "timed_out": true,
        "_shards": {
          "failed": 42.0,
          "successful": 42.0,
          "total": 42.0,
          "failures": [
            {}
          ],
          "skipped": 42.0
        },
        "failures": [
          {
            "index": "string",
            "node": "string",
            "reason": {},
            "shard": 42.0,
            "status": "string"
          }
        ]
      },
      "additionalProperty2": {
        "status": "running",
        "indices": "string",
        "": 42.0,
        "timed_out": true,
        "_shards": {
          "failed": 42.0,
          "successful": 42.0,
          "total": 42.0,
          "failures": [
            {}
          ],
          "skipped": 42.0
        },
        "failures": [
          {
            "index": "string",
            "node": "string",
            "reason": {},
            "shard": 42.0,
            "status": "string"
          }
        ]
      }
    }
  },
  "fields": {
    "additionalProperty1": {},
    "additionalProperty2": {}
  },
  "max_score": 42.0,
  "num_reduce_phases": 42.0,
  "profile": {
    "shards": [
      {
        "aggregations": [
          {
            "breakdown": {},
            "description": "string",
            "type": "string",
            "debug": {},
            "children": [
              {}
            ]
          }
        ],
        "cluster": "string",
        "dfs": {
          "statistics": {
            "type": "string",
            "description": "string",
            "time": "string",
            "breakdown": {},
            "debug": {},
            "children": [
              {}
            ]
          },
          "knn": [
            {}
          ]
        },
        "fetch": {
          "type": "string",
          "description": "string",
          "": 42.0,
          "breakdown": {
            "load_source": 42.0,
            "load_source_count": 42.0,
            "load_stored_fields": 42.0,
            "load_stored_fields_count": 42.0,
            "next_reader": 42.0,
            "next_reader_count": 42.0,
            "process_count": 42.0,
            "process": 42.0
          },
          "debug": {
            "stored_fields": [
              "string"
            ],
            "fast_path": 42.0
          },
          "children": [
            {}
          ]
        },
        "id": "string",
        "index": "string",
        "node_id": "string",
        "searches": [
          {
            "collector": [
              {}
            ],
            "query": [
              {}
            ],
            "rewrite_time": 42.0
          }
        ],
        "shard_id": 42.0
      }
    ]
  },
  "pit_id": "string",
  "_scroll_id": "string",
  "suggest": {
    "additionalProperty1": [
      {
        "length": 42.0,
        "offset": 42.0,
        "text": "string"
      }
    ],
    "additionalProperty2": [
      {
        "length": 42.0,
        "offset": 42.0,
        "text": "string"
      }
    ]
  },
  "terminated_early": true
}

Render a search template

POST /_render/template
Api key auth Basic auth Bearer auth

Render a search template as a search request body.

application/json

Body

  • id string
  • file string
  • params object

    Key-value pairs used to replace Mustache variables in the template. The key is the variable name. The value is the variable value.

    Hide params attribute Show params attribute object
    • * object Additional properties
  • source string

    An inline search template. It supports the same parameters as the search API's request body. These parameters also support Mustache variables. If no id or <templated-id> is specified, this parameter is required.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • template_output object Required
      Hide template_output attribute Show template_output attribute object
      • * object Additional properties
POST /_render/template 
curl \
 --request POST 'http://api.example.com/_render/template' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"id\": \"my-search-template\",\n  \"params\": {\n    \"query_string\": \"hello world\",\n    \"from\": 20,\n    \"size\": 10\n  }\n}"'
Request example
Run `POST _render/template` 
{
  "id": "my-search-template",
  "params": {
    "query_string": "hello world",
    "from": 20,
    "size": 10
  }
}
Response examples (200) 
{
  "template_output": {
    "additionalProperty1": {},
    "additionalProperty2": {}
  }
}

Get the search shards

POST /_search_shards
Api key auth Basic auth Bearer auth

Get the indices and shards that a search request would be run against. This information can be useful for working out issues or planning optimizations with routing and shard preferences. When filtered aliases are used, the filter is returned as part of the indices section.

If the Elasticsearch security features are enabled, you must have the view_index_metadata or manage index privilege for the target data stream, index, or alias.

Query parameters

  • allow_no_indices boolean

    If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

  • ignore_unavailable boolean

    If false, the request returns an error if it targets a missing or closed index.

  • local boolean

    If true, the request retrieves information from the local node only.

  • master_timeout string

    The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. IT can also be set to -1 to indicate that the request should never timeout.

  • preference string

    The node or shard the operation should be performed on. It is random by default.

  • routing string

    A custom value used to route operations to a specific shard.

Responses

POST /_search_shards 
curl \
 --request POST 'http://api.example.com/_search_shards' \
 --header "Authorization: $API_KEY"
Response examples (200)
An abbreviated response from `GET /my-index-000001/_search_shards`. 
{
  "nodes": {},
  "indices": {
      "my-index-000001": { }
  },
  "shards": [
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 0,
          "state": "STARTED",
          "allocation_id": {"id":"0TvkCyF7TAmM1wHP4a42-A"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 1,
          "state": "STARTED",
          "allocation_id": {"id":"fMju3hd1QHWmWrIgFnI4Ww"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 2,
          "state": "STARTED",
          "allocation_id": {"id":"Nwl0wbMBTHCWjEEbGYGapg"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 3,
          "state": "STARTED",
          "allocation_id": {"id":"bU_KLGJISbW0RejwnwDPKw"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 4,
          "state": "STARTED",
          "allocation_id": {"id":"DMs7_giNSwmdqVukF7UydA"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ]
    ]
  }

Bulk create or update roles Added in 8.15.0

POST /_security/role
Api key auth Basic auth Bearer auth

The role management APIs are generally the preferred way to manage roles, rather than using file-based role management. The bulk create or update roles API cannot update roles that are defined in roles files.

Query parameters

  • refresh string

    If true (the default) then refresh the affected shards to make this operation visible to search, if wait_for then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

application/json

Body Required

  • roles object Required

    A dictionary of role name to RoleDescriptor objects to add or update

    Hide roles attribute Show roles attribute object
    • * object Additional properties
      Hide * attributes Show * attributes object
      • cluster array[string]

        A list of cluster privileges. These privileges define the cluster level actions that API keys are able to execute.

      • indices array[object]

        A list of indices permissions entries.

        Hide indices attributes Show indices attributes object
        • field_security object
          Hide field_security attributes Show field_security attributes object
        • names array[string] Required

          A list of indices (or index name patterns) to which the permissions in this entry apply.

        • privileges array[string] Required

          The index level privileges that owners of the role have on the specified indices.

        • query string | object

          While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.

          Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.

          One of:
          IndicesPrivilegesQuery string QueryContainer object RoleTemplateQuery object
        • allow_restricted_indices boolean

          Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

      • remote_indices array[object]

        A list of indices permissions for remote clusters.

        Hide remote_indices attributes Show remote_indices attributes object
        • clusters string | array[string] Required
        • field_security object
          Hide field_security attributes Show field_security attributes object
        • names array[string] Required

          A list of indices (or index name patterns) to which the permissions in this entry apply.

        • privileges array[string] Required

          The index level privileges that owners of the role have on the specified indices.

        • query string | object

          While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.

          Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.

          One of:
          IndicesPrivilegesQuery string QueryContainer object RoleTemplateQuery object
        • allow_restricted_indices boolean

          Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

      • remote_cluster array[object]

        A list of cluster permissions for remote clusters. NOTE: This is limited a subset of the cluster permissions.

        Hide remote_cluster attributes Show remote_cluster attributes object
        • clusters string | array[string] Required
        • privileges array[string] Required

          The cluster level privileges that owners of the role have on the remote cluster.

          Values are monitor_enrich or monitor_stats.

      • global array[object] | object

        An object defining global privileges. A global privilege is a form of cluster privilege that is request-aware. Support for global privileges is currently limited to the management of application privileges.

        One of:
        array-1 array[object] GlobalPrivilege object
        Hide attribute Show attribute object
        • application object Required
          Hide application attribute Show application attribute object
      • applications array[object]

        A list of application privilege entries

        Hide applications attributes Show applications attributes object
        • application string Required

          The name of the application to which this entry applies.

        • privileges array[string] Required

          A list of strings, where each element is the name of an application privilege or action.

        • resources array[string] Required

          A list resources to which the privileges are applied.

      • metadata object
        Hide metadata attribute Show metadata attribute object
        • * object Additional properties
      • run_as array[string]

        A list of users that the API keys can impersonate. NOTE: In Elastic Cloud Serverless, the run-as feature is disabled. For API compatibility, you can still specify an empty run_as field, but a non-empty list will be rejected.

      • description string

        Optional description of the role descriptor

      • restriction object
        Hide restriction attribute Show restriction attribute object
        • workflows array[string] Required

          A list of workflows to which the API key is restricted. NOTE: In order to use a role restriction, an API key must be created with a single role descriptor.

      • transient_metadata object
        Hide transient_metadata attribute Show transient_metadata attribute object
        • * object Additional properties

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • created array[string]

      Array of created roles

    • updated array[string]

      Array of updated roles

    • noop array[string]

      Array of role names without any changes

    • errors object
      Hide errors attributes Show errors attributes object
      • count number Required

        The number of errors

      • details object Required

        Details about the errors, keyed by role name

        Hide details attribute Show details attribute object
        • * object
          Hide * attributes Show * attributes object
POST /_security/role 
curl \
 --request POST 'http://api.example.com/_security/role' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"roles\": {\n      \"my_admin_role\": {\n          \"cluster\": [\n              \"all\"\n          ],\n          \"indices\": [\n              {\n                  \"names\": [\n                      \"index1\",\n                      \"index2\"\n                  ],\n                  \"privileges\": [\n                      \"all\"\n                  ],\n                  \"field_security\": {\n                      \"grant\": [\n                          \"title\",\n                          \"body\"\n                      ]\n                  },\n                  \"query\": \"{\\\"match\\\": {\\\"title\\\": \\\"foo\\\"}}\"\n              }\n          ],\n          \"applications\": [\n              {\n                  \"application\": \"myapp\",\n                  \"privileges\": [\n                      \"admin\",\n                      \"read\"\n                  ],\n                  \"resources\": [\n                      \"*\"\n                  ]\n              }\n          ],\n          \"run_as\": [\n              \"other_user\"\n          ],\n          \"metadata\": {\n              \"version\": 1\n          }\n      },\n      \"my_user_role\": {\n          \"cluster\": [\n              \"all\"\n          ],\n          \"indices\": [\n              {\n                  \"names\": [\n                      \"index1\"\n                  ],\n                  \"privileges\": [\n                      \"read\"\n                  ],\n                  \"field_security\": {\n                      \"grant\": [\n                          \"title\",\n                          \"body\"\n                      ]\n                  },\n                  \"query\": \"{\\\"match\\\": {\\\"title\\\": \\\"foo\\\"}}\"\n              }\n          ],\n          \"applications\": [\n              {\n                  \"application\": \"myapp\",\n                  \"privileges\": [\n                      \"admin\",\n                      \"read\"\n                  ],\n                  \"resources\": [\n                      \"*\"\n                  ]\n              }\n          ],\n          \"run_as\": [\n              \"other_user\"\n          ],\n          \"metadata\": {\n              \"version\": 1\n          }\n      }\n  }\n}"'
Request examples
Run `POST /_security/role` to add roles called `my_admin_role` and `my_user_role`. 
{
  "roles": {
      "my_admin_role": {
          "cluster": [
              "all"
          ],
          "indices": [
              {
                  "names": [
                      "index1",
                      "index2"
                  ],
                  "privileges": [
                      "all"
                  ],
                  "field_security": {
                      "grant": [
                          "title",
                          "body"
                      ]
                  },
                  "query": "{\"match\": {\"title\": \"foo\"}}"
              }
          ],
          "applications": [
              {
                  "application": "myapp",
                  "privileges": [
                      "admin",
                      "read"
                  ],
                  "resources": [
                      "*"
                  ]
              }
          ],
          "run_as": [
              "other_user"
          ],
          "metadata": {
              "version": 1
          }
      },
      "my_user_role": {
          "cluster": [
              "all"
          ],
          "indices": [
              {
                  "names": [
                      "index1"
                  ],
                  "privileges": [
                      "read"
                  ],
                  "field_security": {
                      "grant": [
                          "title",
                          "body"
                      ]
                  },
                  "query": "{\"match\": {\"title\": \"foo\"}}"
              }
          ],
          "applications": [
              {
                  "application": "myapp",
                  "privileges": [
                      "admin",
                      "read"
                  ],
                  "resources": [
                      "*"
                  ]
              }
          ],
          "run_as": [
              "other_user"
          ],
          "metadata": {
              "version": 1
          }
      }
  }
}
Because errors are handled individually for each role create or update, the API allows partial success. For example, `POST /_security/role` would throw an error for `my_admin_role` because the privilege `bad_cluster_privilege` doesn't exist, but would be successful for the `my_user_role`. 
{
  "roles": {
      "my_admin_role": {
          "cluster": [
              "bad_cluster_privilege"
          ],
          "indices": [
              {
                  "names": [
                      "index1",
                      "index2"
                  ],
                  "privileges": ["all"],
                  "field_security": {
                      "grant": [
                          "title",
                          "body"
                      ]
                  },
                  "query": "{\"match\": {\"title\": \"foo\"}}"
              }
          ],
          "applications": [
              {
                  "application": "myapp",
                  "privileges": [
                      "admin",
                      "read"
                  ],
                  "resources": [
                      "*"
                  ]
              }
          ],
          "run_as": [
              "other_user"
          ],
          "metadata": {
              "version": 1
          }
      },
      "my_user_role": {
          "cluster": [
              "all"
          ],
          "indices": [
              {
                  "names": [
                      "index1"
                  ],
                  "privileges": [
                      "read"
                  ],
                  "field_security": {
                      "grant": [
                          "title",
                          "body"
                      ]
                  },
                  "query": "{\"match\": {\"title\": \"foo\"}}"
              }
          ],
          "applications": [
              {
                  "application": "myapp",
                  "privileges": [
                      "admin",
                      "read"
                  ],
                  "resources": [
                      "*"
                  ]
              }
          ],
          "run_as": [
              "other_user"
          ],
          "metadata": {
              "version": 1
          }
      }
  }
}
Run `POST /_security/role/only_remote_access_role` to configure a role with remote indices and remote cluster privileges for a remote cluster. 
{
  "remote_indices": [
    {
      "clusters": ["my_remote"], 
      "names": ["logs*"], 
      "privileges": ["read", "read_cross_cluster", "view_index_metadata"] 
    }
  ],
  "remote_cluster": [
    {
      "clusters": ["my_remote"], 
      "privileges": ["monitor_stats"]  
    }
  ]
}
Response examples (200)
A successful response from `POST /_security/role/my_admin_role` returns a JSON structure that shows whether the role has been created, updated, or had no changes made. 
{
    "created": [ 
        "my_admin_role", 
        "my_user_role"
    ]
}
A partially successful response from `POST /_security/role`. Errors are handled individually for each role create or update, thus the API allows partial success. In this example, the creation of the `my_user_role` role succeeds and the `my_admin_role` role fails. 
{
  "created": [
      "my_user_role" 
  ],
  "errors": { 
      "count": 1, 
      "details": {
          "my_admin_role": { 
              "type": "action_request_validation_exception",
              "reason": "Validation Failed: 1: unknown cluster privilege [bad_cluster_privilege]. a privilege must be either one of the predefined cluster privilege names [manage_own_api_key,manage_data_stream_global_retention,monitor_data_stream_global_retention,none,cancel_task,cross_cluster_replication,cross_cluster_search,delegate_pki,grant_api_key,manage_autoscaling,manage_index_templates,manage_logstash_pipelines,manage_oidc,manage_saml,manage_search_application,manage_search_query_rules,manage_search_synonyms,manage_service_account,manage_token,manage_user_profile,monitor_connector,monitor_enrich,monitor_inference,monitor_ml,monitor_rollup,monitor_snapshot,monitor_stats,monitor_text_structure,monitor_watcher,post_behavioral_analytics_event,read_ccr,read_connector_secrets,read_fleet_secrets,read_ilm,read_pipeline,read_security,read_slm,transport_client,write_connector_secrets,write_fleet_secrets,create_snapshot,manage_behavioral_analytics,manage_ccr,manage_connector,manage_enrich,manage_ilm,manage_inference,manage_ml,manage_rollup,manage_slm,manage_watcher,monitor_data_frame_transforms,monitor_transform,manage_api_key,manage_ingest_pipelines,manage_pipeline,manage_data_frame_transforms,manage_transform,manage_security,monitor,manage,all] or a pattern over one of the available cluster actions;"
          }
      }
  }
}

Clear service account token caches

POST /_security/service/{namespace}/{service}/credential/token/{name}/_clear_cache
Api key auth Basic auth Bearer auth

Evict a subset of all entries from the service account token caches. Two separate caches exist for service account tokens: one cache for tokens backed by the service_tokens file, and another for tokens backed by the .security index. This API clears matching entries from both caches.

The cache for service account tokens backed by the .security index is cleared automatically on state changes of the security index. The cache for tokens backed by the service_tokens file is cleared automatically on file changes.

External documentation

Path parameters

  • namespace string Required

    The namespace, which is a top-level grouping of service accounts.

  • service string Required

    The name of the service, which must be unique within its namespace.

  • name string | array[string] Required

    A comma-separated list of token names to evict from the service account token caches. Use a wildcard (*) to evict all tokens that belong to a service account. It does not support other wildcard patterns.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • _nodes object Required
      Hide _nodes attributes Show _nodes attributes object
      • failures array[object]
        Hide failures attributes Show failures attributes object
      • total number Required

        Total number of nodes selected by the request.

      • successful number Required

        Number of nodes that responded successfully to the request.

      • failed number Required

        Number of nodes that rejected the request or failed to respond. If this value is not 0, a reason for the rejection or failure is included in the response.

    • cluster_name string Required
    • nodes object Required
      Hide nodes attribute Show nodes attribute object
      • * object Additional properties
        Hide * attribute Show * attribute object
POST /_security/service/{namespace}/{service}/credential/token/{name}/_clear_cache 
curl \
 --request POST 'http://api.example.com/_security/service/{namespace}/{service}/credential/token/{name}/_clear_cache' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "_nodes": {
    "failures": [
      {
        "type": "string",
        "reason": "string",
        "stack_trace": "string",
        "caused_by": {},
        "root_cause": [
          {}
        ],
        "suppressed": [
          {}
        ]
      }
    ],
    "total": 42.0,
    "successful": 42.0,
    "failed": 42.0
  },
  "cluster_name": "string",
  "nodes": {
    "additionalProperty1": {
      "name": "string"
    },
    "additionalProperty2": {
      "name": "string"
    }
  }
}

Create a service account token

POST /_security/service/{namespace}/{service}/credential/token
Api key auth Basic auth Bearer auth

Create a service accounts token for access without requiring basic authentication.

NOTE: Service account tokens never expire. You must actively delete them if they are no longer needed.

External documentation

Path parameters

  • namespace string Required

    The name of the namespace, which is a top-level grouping of service accounts.

  • service string Required

    The name of the service.

Query parameters

  • refresh string

    If true then refresh the affected shards to make this operation visible to search, if wait_for (the default) then wait for a refresh to make this operation visible to search, if false then do nothing with refreshes.

    Values are true, false, or wait_for.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
POST /_security/service/{namespace}/{service}/credential/token 
curl \
 --request POST 'http://api.example.com/_security/service/{namespace}/{service}/credential/token' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `POST /_security/service/elastic/fleet-server/credential/token/token1`. The response includes the service account token, its name, and its secret value as a bearer token. 
{
  "created": true,
  "token": {
    "name": "token1",
    "value": "AAEAAWVsYXN0aWM...vZmxlZXQtc2VydmVyL3Rva2VuMTo3TFdaSDZ" 
  }
}

Disable a user profile Added in 8.2.0

PUT /_security/profile/{uid}/_disable
Api key auth Basic auth Bearer auth

Disable user profiles so that they are not visible in user profile searches.

NOTE: The user profile feature is designed only for use by Kibana and Elastic's Observability, Enterprise Search, and Elastic Security solutions. Individual users and external applications should not call this API directly. Elastic reserves the right to change or remove this feature in future releases without prior notice.

When you activate a user profile, its automatically enabled and visible in user profile searches. You can use the disable user profile API to disable a user profile so it’s not visible in these searches. To re-enable a disabled user profile, use the enable user profile API .

Path parameters

  • uid string Required

    Unique identifier for the user profile.

Query parameters

  • refresh string

    If 'true', Elasticsearch refreshes the affected shards to make this operation visible to search. If 'wait_for', it waits for a refresh to make this operation visible to search. If 'false', it does nothing with refreshes.

    Values are true, false, or wait_for.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
PUT /_security/profile/{uid}/_disable 
curl \
 --request PUT 'http://api.example.com/_security/profile/{uid}/_disable' \
 --header "Authorization: $API_KEY"
Response examples (200) 
{
  "acknowledged": true
}

Get user privileges Added in 6.5.0

GET /_security/user/_privileges
Api key auth Basic auth Bearer auth

Get the security privileges for the logged in user. All users can use this API, but only to determine their own privileges. To check the privileges of other users, you must use the run as feature. To check whether a user has a specific list of privileges, use the has privileges API.

Query parameters

  • application string

    The name of the application. Application privileges are always associated with exactly one application. If you do not specify this parameter, the API returns information about all privileges for all applications.

  • priviledge string

    The name of the privilege. If you do not specify this parameter, the API returns information about all privileges for the requested application.

  • username string | null

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • applications array[object] Required
      Hide applications attributes Show applications attributes object
      • application string Required

        The name of the application to which this entry applies.

      • privileges array[string] Required

        A list of strings, where each element is the name of an application privilege or action.

      • resources array[string] Required

        A list resources to which the privileges are applied.

    • cluster array[string] Required
    • remote_cluster array[object]
      Hide remote_cluster attributes Show remote_cluster attributes object
      • clusters string | array[string] Required
      • privileges array[string] Required

        The cluster level privileges that owners of the role have on the remote cluster.

        Values are monitor_enrich or monitor_stats.

    • global array[object] Required
      Hide global attribute Show global attribute object
      • application object Required
        Hide application attribute Show application attribute object
        • manage object Required
          Hide manage attribute Show manage attribute object
    • indices array[object] Required
      Hide indices attributes Show indices attributes object
      • field_security array[object]

        The document fields that the owners of the role have read access to.

        External documentation
        Hide field_security attributes Show field_security attributes object
      • names array[string] Required

        A list of indices (or index name patterns) to which the permissions in this entry apply.

      • privileges array[string] Required

        The index level privileges that owners of the role have on the specified indices.

      • query array[string | object]

        Search queries that define the documents the user has access to. A document within the specified indices must match these queries for it to be accessible by the owners of the role.

        While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.

        Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.

        While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.

        Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.

        One of:
        IndicesPrivilegesQuery string QueryContainer object RoleTemplateQuery object
      • allow_restricted_indices boolean Required

        Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

    • remote_indices array[object]
      Hide remote_indices attributes Show remote_indices attributes object
      • field_security array[object]

        The document fields that the owners of the role have read access to.

        External documentation
        Hide field_security attributes Show field_security attributes object

      • names string | array[string]

        A list of indices (or index name patterns) to which the permissions in this entry apply.

        One of:
        IndexName string array-2 array[string]
      • privileges array[string] Required

        The index level privileges that owners of the role have on the specified indices.

      • query array[string | object]

        Search queries that define the documents the user has access to. A document within the specified indices must match these queries for it to be accessible by the owners of the role.

        While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.

        Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.

        While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.

        Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.

        One of:
        IndicesPrivilegesQuery string QueryContainer object RoleTemplateQuery object
      • allow_restricted_indices boolean Required

        Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

      • clusters array[string] Required
    • run_as array[string] Required
GET /_security/user/_privileges 
curl \
 --request GET 'http://api.example.com/_security/user/_privileges' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_security/user/_privileges`. 
{
  "cluster" : [
    "all"
  ],
  "global" : [ ],
  "indices" : [
    {
      "names" : [
        "*"
      ],
      "privileges" : [
        "all"
      ],
      "allow_restricted_indices" : true
    }
  ],
  "applications" : [
    {
      "application" : "*",
      "privileges" : [
        "*"
      ],
      "resources" : [
        "*"
      ]
    }
  ],
  "run_as" : [
    "*"
  ]
}

Logout of OpenID Connect

POST /_security/oidc/logout
Api key auth Basic auth Bearer auth

Invalidate an access token and a refresh token that were generated as a response to the /_security/oidc/authenticate API.

If the OpenID Connect authentication realm in Elasticsearch is accordingly configured, the response to this call will contain a URI pointing to the end session endpoint of the OpenID Connect Provider in order to perform single logout.

Elasticsearch exposes all the necessary OpenID Connect related functionality with the OpenID Connect APIs. These APIs are used internally by Kibana in order to provide OpenID Connect based authentication, but can also be used by other, custom web applications or other clients.

application/json

Body Required

  • token string Required

    The access token to be invalidated.

  • refresh_token string

    The refresh token to be invalidated.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • redirect string Required

      A URI that points to the end session endpoint of the OpenID Connect Provider with all the parameters of the logout request as HTTP GET parameters.
POST /_security/oidc/logout 
curl \
 --request POST 'http://api.example.com/_security/oidc/logout' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"token\" : \"dGhpcyBpcyBub3QgYSByZWFsIHRva2VuIGJ1dCBpdCBpcyBvbmx5IHRlc3QgZGF0YS4gZG8gbm90IHRyeSB0byByZWFkIHRva2VuIQ==\",\n  \"refresh_token\": \"vLBPvmAB6KvwvJZr27cS\"\n}"'
Request example
Run `POST /_security/oidc/logout` to perform the logout. 
{
  "token" : "dGhpcyBpcyBub3QgYSByZWFsIHRva2VuIGJ1dCBpdCBpcyBvbmx5IHRlc3QgZGF0YS4gZG8gbm90IHRyeSB0byByZWFkIHRva2VuIQ==",
  "refresh_token": "vLBPvmAB6KvwvJZr27cS"
}
Response examples (200)
A successful response from `POST /_security/oidc/logout`, which contains the URI pointing to the End Session Endpoint of the OpenID Connect Provider with all the parameters of the Logout Request as HTTP GET parameters. 
{
  "redirect" : "https://op-provider.org/logout?id_token_hint=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c&post_logout_redirect_uri=http%3A%2F%2Foidc-kibana.elastic.co%2Floggedout&state=lGYK0EcSLjqH6pkT5EVZjC6eIW5YCGgywj2sxROO"
}

Prepare OpenID connect authentication

POST /_security/oidc/prepare
Api key auth Basic auth Bearer auth

Create an oAuth 2.0 authentication request as a URL string based on the configuration of the OpenID Connect authentication realm in Elasticsearch.

The response of this API is a URL pointing to the Authorization Endpoint of the configured OpenID Connect Provider, which can be used to redirect the browser of the user in order to continue the authentication process.

Elasticsearch exposes all the necessary OpenID Connect related functionality with the OpenID Connect APIs. These APIs are used internally by Kibana in order to provide OpenID Connect based authentication, but can also be used by other, custom web applications or other clients.

application/json

Body Required

  • iss string

    In the case of a third party initiated single sign on, this is the issuer identifier for the OP that the RP is to send the authentication request to. It cannot be specified when realm is specified. One of realm or iss is required.

  • login_hint string

    In the case of a third party initiated single sign on, it is a string value that is included in the authentication request as the login_hint parameter. This parameter is not valid when realm is specified.

  • nonce string

    The value used to associate a client session with an ID token and to mitigate replay attacks. If the caller of the API does not provide a value, Elasticsearch will generate one with sufficient entropy and return it in the response.

  • realm string

    The name of the OpenID Connect realm in Elasticsearch the configuration of which should be used in order to generate the authentication request. It cannot be specified when iss is specified. One of realm or iss is required.

  • state string

    The value used to maintain state between the authentication request and the response, typically used as a Cross-Site Request Forgery mitigation. If the caller of the API does not provide a value, Elasticsearch will generate one with sufficient entropy and return it in the response.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • nonce string Required
    • realm string Required
    • redirect string Required

      A URI that points to the authorization endpoint of the OpenID Connect Provider with all the parameters of the authentication request as HTTP GET parameters.

    • state string Required
POST /_security/oidc/prepare 
curl \
 --request POST 'http://api.example.com/_security/oidc/prepare' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"realm\" : \"oidc1\"\n}"'
Request examples
Run `POST /_security/oidc/prepare` to generate an authentication request for the OpenID Connect Realm `oidc1`. 
{
  "realm" : "oidc1"
}
Run `POST /_security/oidc/prepare` to generate an authentication request for the OpenID Connect Realm `oidc1`, where the values for the `state` and the `nonce` have been generated by the client. 
{
  "realm" : "oidc1",
  "state" : "lGYK0EcSLjqH6pkT5EVZjC6eIW5YCGgywj2sxROO",
  "nonce" : "zOBXLJGUooRrbLbQk5YCcyC8AXw3iloynvluYhZ5"
}
Run `POST /_security/oidc/prepare` to generate an authentication request for a third party initiated single sign on. Specify the issuer that should be used for matching the appropriate OpenID Connect Authentication realm. 
{
  "iss" : "http://127.0.0.1:8080",
  "login_hint": "this_is_an_opaque_string"
}
Response examples (200)
A successful response from `POST /_security/oidc/prepare`. It contains the URI pointing to the Authorization Endpoint of the OpenID Connect Provider with all the parameters of the Authentication Request as HTTP GET parameters. 
{
  "redirect" : "http://127.0.0.1:8080/c2id-login?scope=openid&response_type=id_token&redirect_uri=https%3A%2F%2Fmy.fantastic.rp%2Fcb&state=4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I&nonce=WaBPH0KqPVdG5HHdSxPRjfoZbXMCicm5v1OiAj0DUFM&client_id=elasticsearch-rp",
  "state" : "4dbrihtIAt3wBTwo6DxK-vdk-sSyDBV8Yf0AjdkdT5I",
  "nonce" : "WaBPH0KqPVdG5HHdSxPRjfoZbXMCicm5v1OiAj0DUFM",
  "realm" : "oidc1"
}

Find API keys with a query Added in 7.15.0

POST /_security/_query/api_key
Api key auth Basic auth Bearer auth

Get a paginated list of API keys and their information. You can optionally filter the results with a query.

To use this API, you must have at least the manage_own_api_key or the read_security cluster privileges. If you have only the manage_own_api_key privilege, this API returns only the API keys that you own. If you have the read_security, manage_api_key, or greater privileges (including manage_security), this API returns all API keys regardless of ownership.

Query parameters

  • with_limited_by boolean

    Return the snapshot of the owner user's role descriptors associated with the API key. An API key's actual permission is the intersection of its assigned role descriptors and the owner user's role descriptors (effectively limited by it). An API key cannot retrieve any API key’s limited-by role descriptors (including itself) unless it has manage_api_key or higher privileges.

  • with_profile_uid boolean

    Determines whether to also retrieve the profile UID for the API key owner principal. If it exists, the profile UID is returned under the profile_uid response field for each API key.

  • typed_keys boolean

    Determines whether aggregation names are prefixed by their respective types in the response.

application/json

Body

  • aggregations object

    Any aggregations to run over the corpus of returned API keys. Aggregations and queries work together. Aggregations are computed only on the API keys that match the query. This supports only a subset of aggregation types, namely: terms, range, date_range, missing, cardinality, value_count, composite, filter, and filters. Additionally, aggregations only run over the same subset of fields that query works with.

  • query object
    Hide query attributes Show query attributes object
    • bool
    • exists
    • ids
    • match object

      Returns documents that match a provided text, number, date or boolean value. The provided text is analyzed before matching.

    • match_all
    • prefix object

      Returns documents that contain a specific prefix in a provided field.

    • range object

      Returns documents that contain terms within a provided range.

    • simple_query_string
    • term object

      Returns documents that contain an exact term in a provided field. To return a document, the query term must exactly match the queried field's value, including whitespace and capitalization.

    • terms
    • wildcard object

      Returns documents that contain terms matching a wildcard pattern.

  • from number

    The starting document offset. It must not be negative. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • sort string | object | array[string | object]

    One of:
    Field string SortOptions object Sort array[string | object]

    Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • size number

    The number of hits to return. It must not be negative. The size parameter can be set to 0, in which case no API key matches are returned, only the aggregation results. By default, you cannot page through more than 10,000 hits using the from and size parameters. To page through more hits, use the search_after parameter.

  • search_after array[number | string | boolean | null | object]

    A field value.

    A field value.

    One of:
    FieldValue number FieldValue number FieldValue string FieldValue boolean FieldValue string | null FieldValue object

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • total number Required

      The total number of API keys found.

    • count number Required

      The number of API keys returned in the response.

    • api_keys array[object] Required

      A list of API key information.

      Hide api_keys attributes Show api_keys attributes object
      • id string Required
      • name string Required
      • type string Required

        Values are rest or cross_cluster.

      • creation number

        Time unit for milliseconds

      • expiration number

        Time unit for milliseconds

      • invalidated boolean Required

        Invalidation status for the API key. If the key has been invalidated, it has a value of true. Otherwise, it is false.

      • invalidation number

        Time unit for milliseconds

      • username string Required
      • realm string Required

        Realm name of the principal for which this API key was created.

      • realm_type string

        Realm type of the principal for which this API key was created

      • metadata object Required
        Hide metadata attribute Show metadata attribute object
        • * object Additional properties
      • role_descriptors object

        The role descriptors assigned to this API key when it was created or last updated. An empty role descriptor means the API key inherits the owner user’s permissions.

        Hide role_descriptors attribute Show role_descriptors attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • cluster array[string]

            A list of cluster privileges. These privileges define the cluster level actions that API keys are able to execute.

          • indices array[object]

            A list of indices permissions entries.

          • remote_indices array[object]

            A list of indices permissions for remote clusters.

          • remote_cluster array[object]

            A list of cluster permissions for remote clusters. NOTE: This is limited a subset of the cluster permissions.

          • global array[object] | object

            An object defining global privileges. A global privilege is a form of cluster privilege that is request-aware. Support for global privileges is currently limited to the management of application privileges.

            One of:
            array-1 array[object] GlobalPrivilege object
          • applications array[object]

            A list of application privilege entries

          • metadata object
            Hide metadata attribute Show metadata attribute object
            • * object Additional properties
          • run_as array[string]

            A list of users that the API keys can impersonate. NOTE: In Elastic Cloud Serverless, the run-as feature is disabled. For API compatibility, you can still specify an empty run_as field, but a non-empty list will be rejected.

          • description string

            Optional description of the role descriptor

          • restriction object
            Hide restriction attribute Show restriction attribute object
            • workflows array[string] Required

              A list of workflows to which the API key is restricted. NOTE: In order to use a role restriction, an API key must be created with a single role descriptor.

          • transient_metadata object
            Hide transient_metadata attribute Show transient_metadata attribute object
            • * object Additional properties
      • limited_by array[object]

        The owner user’s permissions associated with the API key. It is a point-in-time snapshot captured at creation and subsequent updates. An API key’s effective permissions are an intersection of its assigned privileges and the owner user’s permissions.

        Hide limited_by attribute Show limited_by attribute object
        • * object Additional properties
          Hide * attributes Show * attributes object
          • cluster array[string]

            A list of cluster privileges. These privileges define the cluster level actions that API keys are able to execute.

          • indices array[object]

            A list of indices permissions entries.

          • remote_indices array[object]

            A list of indices permissions for remote clusters.

          • remote_cluster array[object]

            A list of cluster permissions for remote clusters. NOTE: This is limited a subset of the cluster permissions.

          • global
          • applications array[object]

            A list of application privilege entries

          • metadata object
          • run_as array[string]

            A list of users that the API keys can impersonate. NOTE: In Elastic Cloud Serverless, the run-as feature is disabled. For API compatibility, you can still specify an empty run_as field, but a non-empty list will be rejected.

          • description string

            Optional description of the role descriptor

          • restriction object
          • transient_metadata object
      • access object
        Hide access attributes Show access attributes object
        • replication array[object]

          A list of indices permission entries for cross-cluster replication.

          Hide replication attributes Show replication attributes object
        • search array[object]

          A list of indices permission entries for cross-cluster search.

          Hide search attributes Show search attributes object
          • field_security object
          • names
          • query
          • allow_restricted_indices boolean

            Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.

      • profile_uid string

        The profile uid for the API key owner principal, if requested and if it exists

      • _sort array[number | string | boolean | null | object]

        A field value.

        A field value.

        One of:
        FieldValue number FieldValue number FieldValue string FieldValue boolean FieldValue string | null FieldValue object
    • aggregations object

      The aggregations result, if requested.
POST /_security/_query/api_key 
curl \
 --request POST 'http://api.example.com/_security/_query/api_key' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query\": {\n    \"ids\": {\n      \"values\": [\n        \"VuaCfGcBCdbkQm-e5aOx\"\n      ]\n    }\n  }\n}"'
Request examples
Run `GET /_security/_query/api_key?with_limited_by=true` to retrieve an API key by ID. 
{
  "query": {
    "ids": {
      "values": [
        "VuaCfGcBCdbkQm-e5aOx"
      ]
    }
  }
}
Run `GET /_security/_query/api_key`. Use a `bool` query to issue complex logical conditions and use `from`, `size`, and `sort` to help paginate the result. For example, the API key name must begin with `app1-key-` and must not be `app1-key-01`. It must be owned by a username with the wildcard pattern `org-*-user` and the `environment` metadata field must have a `production` value. The offset to begin the search result is the twentieth (zero-based index) API key. The page size of the response is 10 API keys. The result is first sorted by creation date in descending order, then by name in ascending order. 
{
  "query": {
    "bool": {
      "must": [
        {
          "prefix": {
            "name": "app1-key-" 
          }
        },
        {
          "term": {
            "invalidated": "false" 
          }
        }
      ],
      "must_not": [
        {
          "term": {
            "name": "app1-key-01" 
          }
        }
      ],
      "filter": [
        {
          "wildcard": {
            "username": "org-*-user" 
          }
        },
        {
          "term": {
            "metadata.environment": "production" 
          }
        }
      ]
    }
  },
  "from": 20, 
  "size": 10, 
  "sort": [ 
    { "creation": { "order": "desc", "format": "date_time" } },
    "name"
  ]
}
Run `GET /_security/_query/api_key` to retrieve the API key by name. 
{
  "query": {
    "term": {
      "name": {
        "value": "application-key-1"
      }
    }
  }
}
Response examples (200)
A successful response from `GET /_security/_query/api_key?with_limited_by=true`. The `limited_by` details are the owner user's permissions associated with the API key. It is a point-in-time snapshot captured at creation and subsequent updates. An API key's effective permissions are an intersection of its assigned privileges and the owner user's permissions. 
{
  "api_keys": [
    {
      "id": "VuaCfGcBCdbkQm-e5aOx",
      "name": "application-key-1",
      "creation": 1548550550158,
      "expiration": 1548551550158,
      "invalidated": false,
      "username": "myuser",
      "realm": "native1",
      "realm_type": "native",
      "metadata": {
        "application": "my-application"
      },
      "role_descriptors": { },
      "limited_by": [ 
        {
          "role-power-user": {
            "cluster": [
              "monitor"
            ],
            "indices": [
              {
                "names": [
                  "*"
                ],
                "privileges": [
                  "read"
                ],
                "allow_restricted_indices": false
              }
            ],
            "applications": [ ],
            "run_as": [ ],
            "metadata": { },
            "transient_metadata": {
              "enabled": true
            }
          }
        }
      ]
    }
  ]
}
An abbreviated response from `GET /_security/_query/api_key` that contains a list of matched API keys along with their sort values. The first sort value is creation time, which is displayed in `date_time` format. The second sort value is the API key name. 
{
  "total": 100,
  "count": 10,
  "api_keys": [
    {
      "id": "CLXgVnsBOGkf8IyjcXU7",
      "name": "app1-key-79",
      "creation": 1629250154811,
      "invalidated": false,
      "username": "org-admin-user",
      "realm": "native1",
      "metadata": {
        "environment": "production"
      },
      "role_descriptors": { },
      "_sort": [
        "2021-08-18T01:29:14.811Z",  
        "app1-key-79"  
      ]
    },
    {
      "id": "BrXgVnsBOGkf8IyjbXVB",
      "name": "app1-key-78",
      "creation": 1629250153794,
      "invalidated": false,
      "username": "org-admin-user",
      "realm": "native1",
      "metadata": {
        "environment": "production"
      },
      "role_descriptors": { },
      "_sort": [
        "2021-08-18T01:29:13.794Z",
        "app1-key-78"
      ]
    }
  ]
}
A successful response from `GET /_security/_query/api_key`. It includes the role descriptors that are assigned to each API key when it was created or last updated. Note that an API key's effective permissions are an intersection of its assigned privileges and the point-in-time snapshot of the owner user's permissions. An empty role descriptors object means the API key inherits the owner user's permissions. 
{
  "total": 3,
  "count": 3,
  "api_keys": [ 
    {
      "id": "nkvrGXsB8w290t56q3Rg",
      "name": "my-api-key-1",
      "creation": 1628227480421,
      "expiration": 1629091480421,
      "invalidated": false,
      "username": "elastic",
      "realm": "reserved",
      "realm_type": "reserved",
      "metadata": {
        "letter": "a"
      },
      "role_descriptors": { 
        "role-a": {
          "cluster": [
            "monitor"
          ],
          "indices": [
            {
              "names": [
                "index-a"
              ],
              "privileges": [
                "read"
              ],
              "allow_restricted_indices": false
            }
          ],
          "applications": [ ],
          "run_as": [ ],
          "metadata": { },
          "transient_metadata": {
            "enabled": true
          }
        }
      }
    },
    {
      "id": "oEvrGXsB8w290t5683TI",
      "name": "my-api-key-2",
      "creation": 1628227498953,
      "expiration": 1628313898953,
      "invalidated": false,
      "username": "elastic",
      "realm": "reserved",
      "metadata": {
        "letter": "b"
      },
      "role_descriptors": { } 
    }
  ]
}

Prepare SAML authentication Added in 7.5.0

POST /_security/saml/prepare
Api key auth Basic auth Bearer auth

Create a SAML authentication request (<AuthnRequest>) as a URL string based on the configuration of the respective SAML realm in Elasticsearch.

NOTE: This API is intended for use by custom web applications other than Kibana. If you are using Kibana, refer to the documentation for configuring SAML single-sign-on on the Elastic Stack.

This API returns a URL pointing to the SAML Identity Provider. You can use the URL to redirect the browser of the user in order to continue the authentication process. The URL includes a single parameter named SAMLRequest, which contains a SAML Authentication request that is deflated and Base64 encoded. If the configuration dictates that SAML authentication requests should be signed, the URL has two extra parameters named SigAlg and Signature. These parameters contain the algorithm used for the signature and the signature value itself. It also returns a random string that uniquely identifies this SAML Authentication request. The caller of this API needs to store this identifier as it needs to be used in a following step of the authentication process.

External documentation
application/json

Body Required

  • acs string

    The Assertion Consumer Service URL that matches the one of the SAML realms in Elasticsearch. The realm is used to generate the authentication request. You must specify either this parameter or the realm parameter.

  • realm string

    The name of the SAML realm in Elasticsearch for which the configuration is used to generate the authentication request. You must specify either this parameter or the acs parameter.

  • relay_state string

    A string that will be included in the redirect URL that this API returns as the RelayState query parameter. If the Authentication Request is signed, this value is used as part of the signature computation.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • id string Required
    • realm string Required

      The name of the Elasticsearch realm that was used to construct the authentication request.

    • redirect string Required

      The URL to redirect the user to.
POST /_security/saml/prepare 
curl \
 --request POST 'http://api.example.com/_security/saml/prepare' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"realm\" : \"saml1\"\n}"'
Request examples
Run `POST /_security/saml/prepare` to generate a SAML authentication request for the SAML realm named `saml1`. 
{
  "realm" : "saml1"
}
Run `POST /_security/saml/prepare` to generate a SAML authentication request for the SAML realm with an Assertion Consuming Service (ACS) URL. 
{
  "acs" : "https://kibana.org/api/security/saml/callback"
}
Response examples (200)
A successful response from `POST /_security/saml/prepare`. 
{
  "redirect": "https://my-idp.org/login?SAMLRequest=fVJdc6IwFP0rmbwDgUKLGbFDtc462%2B06FX3Yl50rBJsKCZsbrPbXL6J22hdfk%2FNx7zl3eL%2BvK7ITBqVWCfVdRolQuS6k2iR0mU2dmN6Phgh1FTQ8be2rehH%2FWoGWdESF%2FPST0NYorgElcgW1QG5zvkh%2FPfHAZbwx2upcV5SkiMLYzmqsFba1MAthdjIXy5enhL5a23DPOyo6W7kGBa7cwhZ2gO7G8OiW%2BR400kORt0bag7fzezAlk24eqcD2OxxlsNN5O3MdsW9c6CZnbq7rntF4d3s0D7BaHTZhIWN52P%2BcjiuGRbDU6cdj%2BEjJbJLQv4N4ADdhxBiEZbQuWclY4Q8iABbCXczCdSiKMAC%2FgyO2YqbQgrIJDZg%2FcFjsMD%2Fzb3gUcBa5sR%2F9oWR%2BzuJBqlPG14Jbn0DIf2TZ3Jn%2FXmSUrC5ddQB6bob37uZrJdeF4dIDHV3iuhb70Ptq83kOz53ubDLXlcwPJK0q%2FT42AqxIaAkVCkqm2tRgr49yfJGFU%2FZQ3hy3QyuUpd7obPv97kb%2FAQ%3D%3D"}",
  "realm": "saml1",
  "id": "_989a34500a4f5bf0f00d195aa04a7804b4ed42a1"
}

Update user profile data Added in 8.2.0

POST /_security/profile/{uid}/_data
Api key auth Basic auth Bearer auth

Update specific data for the user profile that is associated with a unique ID.

NOTE: The user profile feature is designed only for use by Kibana and Elastic's Observability, Enterprise Search, and Elastic Security solutions. Individual users and external applications should not call this API directly. Elastic reserves the right to change or remove this feature in future releases without prior notice.

To use this API, you must have one of the following privileges:

  • The manage_user_profile cluster privilege.
  • The update_profile_data global privilege for the namespaces that are referenced in the request.

This API updates the labels and data fields of an existing user profile document with JSON objects. New keys and their values are added to the profile document and conflicting keys are replaced by data that's included in the request.

For both labels and data, content is namespaced by the top-level fields. The update_profile_data global privilege grants privileges for updating only the allowed namespaces.

Path parameters

  • uid string Required

    A unique identifier for the user profile.

Query parameters

  • if_seq_no number

    Only perform the operation if the document has this sequence number.

  • if_primary_term number

    Only perform the operation if the document has this primary term.

  • refresh string

    If 'true', Elasticsearch refreshes the affected shards to make this operation visible to search. If 'wait_for', it waits for a refresh to make this operation visible to search. If 'false', nothing is done with refreshes.

    Values are true, false, or wait_for.

application/json

Body Required

  • labels object

    Searchable data that you want to associate with the user profile. This field supports a nested data structure. Within the labels object, top-level keys cannot begin with an underscore (_) or contain a period (.).

    Hide labels attribute Show labels attribute object
    • * object Additional properties
  • data object

    Non-searchable data that you want to associate with the user profile. This field supports a nested data structure. Within the data object, top-level keys cannot begin with an underscore (_) or contain a period (.). The data object is not searchable, but can be retrieved with the get user profile API.

    Hide data attribute Show data attribute object
    • * object Additional properties

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
POST /_security/profile/{uid}/_data 
curl \
 --request POST 'http://api.example.com/_security/profile/{uid}/_data' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"labels\": {\n    \"direction\": \"east\"\n  },\n  \"data\": {\n    \"app1\": {\n      \"theme\": \"default\"\n    }\n  }\n}"'
Request example
Run `POST /_security/profile/u_P_0BMHgaOK3p7k-PFWUCbw9dQ-UFjt01oWJ_Dp2PmPc_0/_data` to update a profile document for the `u_P_0BMHgaOK3p7k-PFWUCbw9dQ-UFjt01oWJ_Dp2PmPc_0` user profile. 
{
  "labels": {
    "direction": "east"
  },
  "data": {
    "app1": {
      "theme": "default"
    }
  }
}
Response examples (200)
A successful response from `POST /_security/profile/u_P_0BMHgaOK3p7k-PFWUCbw9dQ-UFjt01oWJ_Dp2PmPc_0/_data`, which indicates that the request is acknowledged. 
{
  "acknowledged": true
}

Clean up the snapshot repository Added in 7.4.0

POST /_snapshot/{repository}/_cleanup
Api key auth Basic auth Bearer auth

Trigger the review of the contents of a snapshot repository and delete any stale data not referenced by existing snapshots.

External documentation

Path parameters

  • repository string Required

    Snapshot repository to clean up.

Query parameters

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • results object Required
      Hide results attributes Show results attributes object
      • deleted_blobs number Required

        Number of binary large objects (blobs) removed during cleanup.

      • deleted_bytes number Required

        Number of bytes freed by cleanup operations.
POST /_snapshot/{repository}/_cleanup 
curl \
 --request POST 'http://api.example.com/_snapshot/{repository}/_cleanup' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `POST /_snapshot/my_repository/_cleanup`. 
{
  "results": {
    "deleted_bytes": 20,
    "deleted_blobs": 5
  }
}

Get snapshot repository information Added in 0.0.0

GET /_snapshot/{repository}
Api key auth Basic auth Bearer auth

Path parameters

  • repository string | array[string] Required

    A comma-separated list of repository names

Query parameters

  • local boolean

    Return local information, do not retrieve the state from master node (default: false)

  • master_timeout string

    Explicit operation timeout for connection to master node

Responses

GET /_snapshot/{repository} 
curl \
 --request GET 'http://api.example.com/_snapshot/{repository}' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_snapshot/my_repository`. 
{
  "my_repository" : {
    "type" : "fs",
    "uuid" : "0JLknrXbSUiVPuLakHjBrQ",
    "settings" : {
      "location" : "my_backup_location"
    }
  }
}

Create or update a snapshot repository Added in 0.0.0

PUT /_snapshot/{repository}
Api key auth Basic auth Bearer auth

IMPORTANT: If you are migrating searchable snapshots, the repository name must be identical in the source and destination clusters. To register a snapshot repository, the cluster's global metadata must be writeable. Ensure there are no cluster blocks (for example, cluster.blocks.read_only and clsuter.blocks.read_only_allow_delete settings) that prevent write access.

External documentation

Path parameters

Query parameters

  • master_timeout string

    Explicit operation timeout for connection to master node

  • timeout string

    Explicit operation timeout

  • verify boolean

    Whether to verify the repository after creation

application/json

Body object Required

One of:
AzureRepository object GcsRepository object S3Repository object SharedFileSystemRepository object ReadOnlyUrlRepository object SourceOnlyRepository object

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.
PUT /_snapshot/{repository} 
curl \
 --request PUT 'http://api.example.com/_snapshot/{repository}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"type\": \"fs\",\n  \"settings\": {\n    \"location\": \"my_backup_location\"\n  }\n}"'
Request examples
Run `PUT /_snapshot/my_repository` to create or update a shared file system snapshot repository. 
{
  "type": "fs",
  "settings": {
    "location": "my_backup_location"
  }
}
Run `PUT /_snapshot/my_repository` to create or update an Azure snapshot repository. 
{
  "type": "azure",
  "settings": {
    "client": "secondary"
  }
}
Run `PUT /_snapshot/my_gcs_repository` to create or update a Google Cloud Storage snapshot repository. 
{
  "type": "gcs",
  "settings": {
    "bucket": "my_other_bucket",
    "base_path": "dev"
  }
}
Run `PUT /_snapshot/my_s3_repository` to create or update an AWS S3 snapshot repository. 
{
  "type": "s3",
  "settings": {
    "bucket": "my-bucket"
  }
}
Run `PUT _snapshot/my_src_only_repository` to create or update a source-only snapshot repository. 
{
  "type": "source",
  "settings": {
    "delegate_type": "fs",
    "location": "my_backup_repository"
  }
}
Run `PUT _snapshot/my_read_only_url_repository` to create or update a read-only URL snapshot repository. 
{
  "type": "url",
  "settings": {
    "url": "file:/mount/backups/my_fs_backup_location"
  }
}
Response examples (200) 
{
  "acknowledged": true
}

Get snapshot repository information Added in 0.0.0

GET /_snapshot
Api key auth Basic auth Bearer auth

Query parameters

  • local boolean

    Return local information, do not retrieve the state from master node (default: false)

  • master_timeout string

    Explicit operation timeout for connection to master node

Responses

GET /_snapshot 
curl \
 --request GET 'http://api.example.com/_snapshot' \
 --header "Authorization: $API_KEY"
Response examples (200)
A successful response from `GET /_snapshot/my_repository`. 
{
  "my_repository" : {
    "type" : "fs",
    "uuid" : "0JLknrXbSUiVPuLakHjBrQ",
    "settings" : {
      "location" : "my_backup_location"
    }
  }
}

Synonyms

The synonyms management API provides a convenient way to define and manage synonyms in an internal system index. Related synonyms can be grouped in a "synonyms set". Create as many synonym sets as you need.

Test a Grok pattern Added in 8.13.0

POST /_text_structure/test_grok_pattern
Api key auth Basic auth Bearer auth

Test a Grok pattern on one or more lines of text. The API indicates whether the lines match the pattern together with the offsets and lengths of the matched substrings.

External documentation

Query parameters

  • ecs_compatibility string

    The mode of compatibility with ECS compliant Grok patterns. Use this parameter to specify whether to use ECS Grok patterns instead of legacy ones when the structure finder creates a Grok pattern. Valid values are disabled and v1.

application/json

Body Required

  • grok_pattern string Required
  • text array[string] Required

    The lines of text to run the Grok pattern on.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • matches array[object] Required
      Hide matches attributes Show matches attributes object
      • matched boolean Required
      • fields object
        Hide fields attribute Show fields attribute object
        • * array[object] Additional properties
          Hide * attributes Show * attributes object
POST /_text_structure/test_grok_pattern 
curl \
 --request POST 'http://api.example.com/_text_structure/test_grok_pattern' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"grok_pattern\": \"Hello %{WORD:first_name} %{WORD:last_name}\",\n  \"text\": [\n    \"Hello John Doe\",\n    \"this does not match\"\n  ]\n}"'
Request example
Run `GET _text_structure/test_grok_pattern` to test a Grok pattern. 
{
  "grok_pattern": "Hello %{WORD:first_name} %{WORD:last_name}",
  "text": [
    "Hello John Doe",
    "this does not match"
  ]
}
Response examples (200)
A successful response from `GET _text_structure/test_grok_pattern`. 
{
  "matches": [
    {
      "matched": true,
      "fields": {
        "first_name": [
          {
            "match": "John",
            "offset": 6,
            "length": 4
          }
        ],
        "last_name": [
          {
            "match": "Doe",
            "offset": 11,
            "length": 3
          }
        ]
      }
    },
    {
      "matched": false
    }
  ]
}

Preview a transform Added in 7.2.0

POST /_transform/_preview
Api key auth Basic auth Bearer auth

Generates a preview of the results that you will get when you create a transform with the same configuration.

It returns a maximum of 100 results. The calculations are based on all the current data in the source index. It also generates a list of mappings and settings for the destination index. These values are determined based on the field types of the source index and the transform aggregations.

Query parameters

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

application/json

Body

  • dest object
    Hide dest attributes Show dest attributes object
  • description string

    Free text description of the transform.

  • frequency string

    A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • pivot object
    Hide pivot attributes Show pivot attributes object
    • aggregations object

      Defines how to aggregate the grouped data. The following aggregations are currently supported: average, bucket script, bucket selector, cardinality, filter, geo bounds, geo centroid, geo line, max, median absolute deviation, min, missing, percentiles, rare terms, scripted metric, stats, sum, terms, top metrics, value count, weighted average.

    • group_by object

      Defines how to group the data. More than one grouping can be defined per pivot. The following groupings are currently supported: date histogram, geotile grid, histogram, terms.

      Hide group_by attribute Show group_by attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • date_histogram object
          Hide date_histogram attributes Show date_histogram attributes object
          • calendar_interval string

            Values are second, 1s, minute, 1m, hour, 1h, day, 1d, week, 1w, month, 1M, quarter, 1q, year, or 1y.

          • extended_bounds object
            Hide extended_bounds attributes Show extended_bounds attributes object
          • hard_bounds object
            Hide hard_bounds attributes Show hard_bounds attributes object
          • field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • fixed_interval string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • format string

            The date format used to format key_as_string in the response. If no format is specified, the first date format specified in the field mapping is used.

          • interval string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • min_doc_count number

            Only returns buckets that have min_doc_count number of documents. By default, all buckets between the first bucket that matches documents and the last one are returned.

          • missing string
          • offset string

            A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • order object | array[object]

            One of:
            AggregateOrder object AggregateOrder array[object]
          • params object
            Hide params attribute Show params attribute object
            • * object Additional properties
          • script object
            Hide script attributes Show script attributes object
            • source string

              The script source.

            • id string
            • params object

              Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            • lang
            • options object
          • time_zone string
          • keyed boolean

            Set to true to associate a unique string key with each bucket and return the ranges as a hash rather than an array.

        • geotile_grid object
          Hide geotile_grid attributes Show geotile_grid attributes object
        • histogram object
          Hide histogram attributes Show histogram attributes object
          • extended_bounds object
            Hide extended_bounds attributes Show extended_bounds attributes object
            • max number

              Maximum value for the bound.

            • min number

              Minimum value for the bound.

          • hard_bounds object
            Hide hard_bounds attributes Show hard_bounds attributes object
            • max number

              Maximum value for the bound.

            • min number

              Minimum value for the bound.

          • field string

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • interval number

            The interval for the buckets. Must be a positive decimal.

          • min_doc_count number

            Only returns buckets that have min_doc_count number of documents. By default, the response will fill gaps in the histogram with empty buckets.

          • missing number

            The value to apply to documents that do not have a value. By default, documents without a value are ignored.

          • offset number

            By default, the bucket keys start with 0 and then continue in even spaced steps of interval. The bucket boundaries can be shifted by using the offset option.

          • order object | array[object]

            One of:
            AggregateOrder object AggregateOrder array[object]
          • script object
            Hide script attributes Show script attributes object
            • source string

              The script source.

            • id string
            • params object

              Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            • lang
            • options object
          • format string
          • keyed boolean

            If true, returns buckets as a hash instead of an array, keyed by the bucket keys.

        • terms
  • source object
    Hide source attributes Show source attributes object
    • index string | array[string] Required
    • runtime_mappings object
      Hide runtime_mappings attribute Show runtime_mappings attribute object
      • * object Additional properties
        Hide * attributes Show * attributes object
        • fields object

          For type composite

          Hide fields attribute Show fields attribute object
          • * object Additional properties
            Hide * attribute Show * attribute object
            • type string Required

              Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

        • fetch_fields array[object]

          For type lookup

          Hide fetch_fields attributes Show fetch_fields attributes object
          • field string Required

            Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

          • format string
        • format string

          A custom format for date type runtime fields.

        • input_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_field string

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • target_index string
        • script object
          Hide script attributes Show script attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

            Hide params attribute Show params attribute object
            • * object Additional properties

          • lang string

            Any of:
            ScriptLanguage string ScriptLanguage string

            Values are painless, expression, mustache, or java.

          • options object
            Hide options attribute Show options attribute object
            • * string Additional properties
        • type string Required

          Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

    • query object

      A query clause that retrieves a subset of data from the source index.

      Query DSL
  • settings object
    Hide settings attributes Show settings attributes object
    • align_checkpoints boolean

      Specifies whether the transform checkpoint ranges should be optimized for performance. Such optimization can align checkpoint ranges with the date histogram interval when date histogram is specified as a group source in the transform config. As a result, less document updates in the destination index will be performed thus improving overall performance.

    • dates_as_epoch_millis boolean

      Defines if dates in the ouput should be written as ISO formatted string or as millis since epoch. epoch_millis was the default for transforms created before version 7.11. For compatible output set this value to true.

    • deduce_mappings boolean

      Specifies whether the transform should deduce the destination index mappings from the transform configuration.

    • docs_per_second number

      Specifies a limit on the number of input documents per second. This setting throttles the transform by adding a wait time between search requests. The default value is null, which disables throttling.

    • max_page_search_size number

      Defines the initial page size to use for the composite aggregation for each checkpoint. If circuit breaker exceptions occur, the page size is dynamically adjusted to a lower value. The minimum value is 10 and the maximum is 65,536.

    • unattended boolean

      If true, the transform runs in unattended mode. In unattended mode, the transform retries indefinitely in case of an error which means the transform never fails. Setting the number of retries other than infinite fails in validation.

  • sync object
    Hide sync attribute Show sync attribute object
    • time object
      Hide time attributes Show time attributes object
      • delay string

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

  • retention_policy object
    Hide retention_policy attribute Show retention_policy attribute object
    • time object
      Hide time attributes Show time attributes object
      • field string Required

        Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • max_age string Required

        A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

  • latest object
    Hide latest attributes Show latest attributes object
    • sort string Required

      Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

    • unique_key array[string] Required

      Specifies an array of one or more fields that are used to group the data.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • generated_dest_index object Required
      Hide generated_dest_index attributes Show generated_dest_index attributes object
      • aliases object
        Hide aliases attribute Show aliases attribute object
      • mappings object
        Hide mappings attributes Show mappings attributes object
      • settings object
        Hide settings attributes Show settings attributes object
        • index object
        • mode string

        • routing_path string | array[string]

          One of:
          string-1 string array-2 array[string]
        • soft_deletes object
          Hide soft_deletes attributes Show soft_deletes attributes object
          • enabled boolean

            Indicates whether soft deletes are enabled on the index.

          • retention_lease object
            Hide retention_lease attribute Show retention_lease attribute object
            • period string Required

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • sort object
          Hide sort attributes Show sort attributes object

        • number_of_shards number | string

          One of:
          number-1 number string-2 string

        • number_of_replicas number | string

          One of:
          number-1 number string-2 string
        • number_of_routing_shards number
        • check_on_startup string

          Values are true, false, or checksum.

        • codec string

        • routing_partition_size number | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          Stringifiedinteger number Stringifiedinteger string
        • load_fixed_bitset_filters_eagerly boolean

        • hidden boolean | string

          One of:
          boolean-1 boolean string-2 string

        • auto_expand_replicas string | null

          One of:
          string-1 string NullValue string | null
        • merge object
          Hide merge attribute Show merge attribute object
          • scheduler object
            Hide scheduler attributes Show scheduler attributes object

            • max_thread_count number | string

              Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

              Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

              One of:
              Stringifiedinteger number Stringifiedinteger string

            • max_merge_count number | string

              Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

              Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

              One of:
              Stringifiedinteger number Stringifiedinteger string
        • search object
          Hide search attributes Show search attributes object
          • idle object
            Hide idle attribute Show idle attribute object
            • after string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • slowlog object
            Hide slowlog attributes Show slowlog attributes object
        • refresh_interval string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • max_result_window number
        • max_inner_result_window number
        • max_rescore_window number
        • max_docvalue_fields_search number
        • max_script_fields number
        • max_ngram_diff number
        • max_shingle_diff number
        • blocks object
          Hide blocks attributes Show blocks attributes object

          • read_only boolean | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string

          • read_only_allow_delete boolean | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string

          • read boolean | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string

          • write boolean | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string

          • metadata boolean | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string
        • max_refresh_listeners number
        • analyze object
          Hide analyze attribute Show analyze attribute object

          • max_token_count number | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedinteger number Stringifiedinteger string
        • highlight object
          Hide highlight attribute Show highlight attribute object
        • max_terms_count number
        • max_regex_length number
        • routing object
          Hide routing attributes Show routing attributes object
        • gc_deletes string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • default_pipeline string
        • final_pipeline string
        • lifecycle object
          Hide lifecycle attributes Show lifecycle attributes object
          • name string

          • indexing_complete boolean | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string
          • origination_date number

            If specified, this is the timestamp used to calculate the index age for its phase transitions. Use this setting if you create a new index that contains old data and want to use the original creation date to calculate the index age. Specified as a Unix epoch value in milliseconds.

          • parse_origination_date boolean

            Set to true to parse the origination date from the index name. This origination date is used to calculate the index age for its phase transitions. The index name must match the pattern .*-{date_format}-\d+, where the date_format is yyyy.MM.dd and the trailing digits are optional. An index that was rolled over would normally match the full format, for example logs-2016.10.31-000002). If the index name doesn’t match the pattern, index creation fails.

          • step object
            Hide step attribute Show step attribute object
            • wait_time_threshold string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • rollover_alias string

            The index alias to update when the index rolls over. Specify when using a policy that contains a rollover action. When the index rolls over, the alias is updated to reflect that the index is no longer the write index. For more information about rolling indices, see Rollover.

          • prefer_ilm boolean | string

            Preference for the system that manages a data stream backing index (preferring ILM when both ILM and DLM are applicable for an index).

            One of:
            boolean-1 boolean string-2 string
        • provided_name string

        • creation_date number | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          UnitMillis number StringifiedEpochTimeUnitMillis string

          Time unit for milliseconds

        • creation_date_string string | number

          A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

          One of:
          DateTime string UnitMillis number
        • uuid string
        • version object
          Hide version attributes Show version attributes object

        • verified_before_close boolean | string

          One of:
          boolean-1 boolean string-2 string

        • format string | number

          One of:
          string-1 string number-2 number
        • max_slices_per_scroll number
        • translog object
          Hide translog attributes Show translog attributes object
        • query_string object
          Hide query_string attribute Show query_string attribute object

          • lenient boolean | string Required

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string

        • priority number | string

          One of:
          number-1 number string-2 string
        • top_metrics_max_size number
        • analysis object
          Hide analysis attributes Show analysis attributes object
        • settings object
        • time_series object
          Hide time_series attributes Show time_series attributes object

          • end_time string | number

            A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

            One of:
            DateTime string UnitMillis number

          • start_time string | number

            A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

            One of:
            DateTime string UnitMillis number
        • queries object
          Hide queries attribute Show queries attribute object
          • cache object
            Hide cache attribute Show cache attribute object
        • similarity object

          Configure custom similarity settings to customize how search results are scored.

        • mapping object
          Hide mapping attributes Show mapping attributes object
          • coerce boolean
          • total_fields object
            Hide total_fields attributes Show total_fields attributes object

            • limit number | string

              The maximum number of fields in an index. Field and object mappings, as well as field aliases count towards this limit. The limit is in place to prevent mappings and searches from becoming too large. Higher values can lead to performance degradations and memory issues, especially in clusters with a high load or few resources.

              One of:
              number-1 number string-2 string

            • ignore_dynamic_beyond_limit boolean | string

              This setting determines what happens when a dynamically mapped field would exceed the total fields limit. When set to false (the default), the index request of the document that tries to add a dynamic field to the mapping will fail with the message Limit of total fields [X] has been exceeded. When set to true, the index request will not fail. Instead, fields that would exceed the limit are not added to the mapping, similar to dynamic: false. The fields that were not added to the mapping will be added to the _ignored field.

              One of:
              boolean-1 boolean string-2 string
          • depth object
            Hide depth attribute Show depth attribute object
            • limit number

              The maximum depth for a field, which is measured as the number of inner objects. For instance, if all fields are defined at the root object level, then the depth is 1. If there is one object mapping, then the depth is 2, etc.

          • nested_fields object
            Hide nested_fields attribute Show nested_fields attribute object
            • limit number

              The maximum number of distinct nested mappings in an index. The nested type should only be used in special cases, when arrays of objects need to be queried independently of each other. To safeguard against poorly designed mappings, this setting limits the number of unique nested types per index.

          • nested_objects object
            Hide nested_objects attribute Show nested_objects attribute object
            • limit number

              The maximum number of nested JSON objects that a single document can contain across all nested types. This limit helps to prevent out of memory errors when a document contains too many nested objects.

          • field_name_length object
            Hide field_name_length attribute Show field_name_length attribute object
            • limit number

              Setting for the maximum length of a field name. This setting isn’t really something that addresses mappings explosion but might still be useful if you want to limit the field length. It usually shouldn’t be necessary to set this setting. The default is okay unless a user starts to add a huge number of fields with really long names. Default is Long.MAX_VALUE (no limit).

          • dimension_fields object
            Hide dimension_fields attribute Show dimension_fields attribute object
            • limit number

              [preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

          • source object
            Hide source attribute Show source attribute object
            • mode string Required

              Values are disabled, stored, or synthetic.

          • ignore_malformed boolean | string

            One of:
            boolean-1 boolean string-2 string
        • indexing.slowlog object
          Hide indexing.slowlog attributes Show indexing.slowlog attributes object
          • level string
          • source number
          • reformat boolean
          • threshold object
            Hide threshold attribute Show threshold attribute object
            • index object
              Hide index attributes Show index attributes object
              • warn string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

              • info string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

              • debug string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

              • trace string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • indexing_pressure object
          Hide indexing_pressure attribute Show indexing_pressure attribute object
          • memory object Required
            Hide memory attribute Show memory attribute object
            • limit number

              Number of outstanding bytes that may be consumed by indexing requests. When this limit is reached or exceeded, the node will reject new coordinating and primary operations. When replica operations consume 1.5x this limit, the node will reject new replica operations. Defaults to 10% of the heap.

        • store object
          Hide store attributes Show store attributes object

          • type string Required

            Any of:
            StorageType string StorageType string

            Values are fs, niofs, mmapfs, or hybridfs.

          • allow_mmap boolean

            You can restrict the use of the mmapfs and the related hybridfs store type via the setting node.store.allow_mmap. This is a boolean setting indicating whether or not memory-mapping is allowed. The default is to allow it. This setting is useful, for example, if you are in an environment where you can not control the ability to create a lot of memory maps so you need disable the ability to use memory-mapping.

      • defaults object
        Hide defaults attributes Show defaults attributes object
        • index object
        • mode string

        • routing_path string | array[string]

          One of:
          string-1 string array-2 array[string]
        • soft_deletes object
          Hide soft_deletes attributes Show soft_deletes attributes object
          • enabled boolean

            Indicates whether soft deletes are enabled on the index.

          • retention_lease object
            Hide retention_lease attribute Show retention_lease attribute object
            • period string Required

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • sort object
          Hide sort attributes Show sort attributes object

        • number_of_shards number | string

          One of:
          number-1 number string-2 string

        • number_of_replicas number | string

          One of:
          number-1 number string-2 string
        • number_of_routing_shards number
        • check_on_startup string

          Values are true, false, or checksum.

        • codec string

        • routing_partition_size number | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          Stringifiedinteger number Stringifiedinteger string
        • load_fixed_bitset_filters_eagerly boolean

        • hidden boolean | string

          One of:
          boolean-1 boolean string-2 string

        • auto_expand_replicas string | null

          One of:
          string-1 string NullValue string | null
        • merge object
          Hide merge attribute Show merge attribute object
          • scheduler object
            Hide scheduler attributes Show scheduler attributes object

            • max_thread_count number | string

              Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

              Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

              One of:
              Stringifiedinteger number Stringifiedinteger string

            • max_merge_count number | string

              Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

              Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

              One of:
              Stringifiedinteger number Stringifiedinteger string
        • search object
          Hide search attributes Show search attributes object
          • idle object
            Hide idle attribute Show idle attribute object
            • after string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • slowlog object
            Hide slowlog attributes Show slowlog attributes object
        • refresh_interval string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • max_result_window number
        • max_inner_result_window number
        • max_rescore_window number
        • max_docvalue_fields_search number
        • max_script_fields number
        • max_ngram_diff number
        • max_shingle_diff number
        • blocks object
          Hide blocks attributes Show blocks attributes object

          • read_only boolean | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string

          • read_only_allow_delete boolean | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string

          • read boolean | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string

          • write boolean | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string

          • metadata boolean | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string
        • max_refresh_listeners number
        • analyze object
          Hide analyze attribute Show analyze attribute object

          • max_token_count number | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedinteger number Stringifiedinteger string
        • highlight object
          Hide highlight attribute Show highlight attribute object
        • max_terms_count number
        • max_regex_length number
        • routing object
          Hide routing attributes Show routing attributes object
        • gc_deletes string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • default_pipeline string
        • final_pipeline string
        • lifecycle object
          Hide lifecycle attributes Show lifecycle attributes object
          • name string

          • indexing_complete boolean | string

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string
          • origination_date number

            If specified, this is the timestamp used to calculate the index age for its phase transitions. Use this setting if you create a new index that contains old data and want to use the original creation date to calculate the index age. Specified as a Unix epoch value in milliseconds.

          • parse_origination_date boolean

            Set to true to parse the origination date from the index name. This origination date is used to calculate the index age for its phase transitions. The index name must match the pattern .*-{date_format}-\d+, where the date_format is yyyy.MM.dd and the trailing digits are optional. An index that was rolled over would normally match the full format, for example logs-2016.10.31-000002). If the index name doesn’t match the pattern, index creation fails.

          • step object
            Hide step attribute Show step attribute object
            • wait_time_threshold string

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

          • rollover_alias string

            The index alias to update when the index rolls over. Specify when using a policy that contains a rollover action. When the index rolls over, the alias is updated to reflect that the index is no longer the write index. For more information about rolling indices, see Rollover.

          • prefer_ilm boolean | string

            Preference for the system that manages a data stream backing index (preferring ILM when both ILM and DLM are applicable for an index).

            One of:
            boolean-1 boolean string-2 string
        • provided_name string

        • creation_date number | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

          One of:
          UnitMillis number StringifiedEpochTimeUnitMillis string

          Time unit for milliseconds

        • creation_date_string string | number

          A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

          One of:
          DateTime string UnitMillis number
        • uuid string
        • version object
          Hide version attributes Show version attributes object

        • verified_before_close boolean | string

          One of:
          boolean-1 boolean string-2 string

        • format string | number

          One of:
          string-1 string number-2 number
        • max_slices_per_scroll number
        • translog object
          Hide translog attributes Show translog attributes object
        • query_string object
          Hide query_string attribute Show query_string attribute object

          • lenient boolean | string Required

            Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

            Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

            One of:
            Stringifiedboolean boolean Stringifiedboolean string

        • priority number | string

          One of:
          number-1 number string-2 string
        • top_metrics_max_size number
        • analysis object
          Hide analysis attributes Show analysis attributes object
        • settings object
        • time_series object
          Hide time_series attributes Show time_series attributes object

          • end_time string | number

            A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

            One of:
            DateTime string UnitMillis number

          • start_time string | number

            A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

            One of:
            DateTime string UnitMillis number
        • queries object
          Hide queries attribute Show queries attribute object
          • cache object
            Hide cache attribute Show cache attribute object
        • similarity object

          Configure custom similarity settings to customize how search results are scored.

        • mapping object
          Hide mapping attributes Show mapping attributes object
          • coerce boolean
          • total_fields object
            Hide total_fields attributes Show total_fields attributes object

            • limit number | string

              The maximum number of fields in an index. Field and object mappings, as well as field aliases count towards this limit. The limit is in place to prevent mappings and searches from becoming too large. Higher values can lead to performance degradations and memory issues, especially in clusters with a high load or few resources.

              One of:
              number-1 number string-2 string

            • ignore_dynamic_beyond_limit boolean | string

              This setting determines what happens when a dynamically mapped field would exceed the total fields limit. When set to false (the default), the index request of the document that tries to add a dynamic field to the mapping will fail with the message Limit of total fields [X] has been exceeded. When set to true, the index request will not fail. Instead, fields that would exceed the limit are not added to the mapping, similar to dynamic: false. The fields that were not added to the mapping will be added to the _ignored field.

              One of:
              boolean-1 boolean string-2 string
          • depth object
            Hide depth attribute Show depth attribute object
            • limit number

              The maximum depth for a field, which is measured as the number of inner objects. For instance, if all fields are defined at the root object level, then the depth is 1. If there is one object mapping, then the depth is 2, etc.

          • nested_fields object
            Hide nested_fields attribute Show nested_fields attribute object
            • limit number

              The maximum number of distinct nested mappings in an index. The nested type should only be used in special cases, when arrays of objects need to be queried independently of each other. To safeguard against poorly designed mappings, this setting limits the number of unique nested types per index.

          • nested_objects object
            Hide nested_objects attribute Show nested_objects attribute object
            • limit number

              The maximum number of nested JSON objects that a single document can contain across all nested types. This limit helps to prevent out of memory errors when a document contains too many nested objects.

          • field_name_length object
            Hide field_name_length attribute Show field_name_length attribute object
            • limit number

              Setting for the maximum length of a field name. This setting isn’t really something that addresses mappings explosion but might still be useful if you want to limit the field length. It usually shouldn’t be necessary to set this setting. The default is okay unless a user starts to add a huge number of fields with really long names. Default is Long.MAX_VALUE (no limit).

          • dimension_fields object
            Hide dimension_fields attribute Show dimension_fields attribute object
            • limit number

              [preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

          • source object
            Hide source attribute Show source attribute object
            • mode string Required

              Values are disabled, stored, or synthetic.

          • ignore_malformed boolean | string

            One of:
            boolean-1 boolean string-2 string
        • indexing.slowlog object
          Hide indexing.slowlog attributes Show indexing.slowlog attributes object
          • level string
          • source number
          • reformat boolean
          • threshold object
            Hide threshold attribute Show threshold attribute object
            • index object
              Hide index attributes Show index attributes object
              • warn string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

              • info string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

              • debug string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

              • trace string

                A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • indexing_pressure object
          Hide indexing_pressure attribute Show indexing_pressure attribute object
          • memory object Required
            Hide memory attribute Show memory attribute object
            • limit number

              Number of outstanding bytes that may be consumed by indexing requests. When this limit is reached or exceeded, the node will reject new coordinating and primary operations. When replica operations consume 1.5x this limit, the node will reject new replica operations. Defaults to 10% of the heap.

        • store object
          Hide store attributes Show store attributes object

          • type string Required

            Any of:
            StorageType string StorageType string

            Values are fs, niofs, mmapfs, or hybridfs.

          • allow_mmap boolean

            You can restrict the use of the mmapfs and the related hybridfs store type via the setting node.store.allow_mmap. This is a boolean setting indicating whether or not memory-mapping is allowed. The default is to allow it. This setting is useful, for example, if you are in an environment where you can not control the ability to create a lot of memory maps so you need disable the ability to use memory-mapping.

      • data_stream string
      • lifecycle object
        Hide lifecycle attributes Show lifecycle attributes object
        • data_retention string

          A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • downsampling object
          Hide downsampling attribute Show downsampling attribute object
          • rounds array[object] Required

            The list of downsampling rounds to execute as part of this downsampling configuration

            Hide rounds attributes Show rounds attributes object
            • after string Required

              A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

            • config object Required
        • enabled boolean

          If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

    • preview array[object] Required
POST /_transform/_preview 
curl \
 --request POST 'http://api.example.com/_transform/_preview' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"source\": {\n    \"index\": \"kibana_sample_data_ecommerce\"\n  },\n  \"pivot\": {\n    \"group_by\": {\n      \"customer_id\": {\n        \"terms\": {\n          \"field\": \"customer_id\",\n          \"missing_bucket\": true\n        }\n      }\n    },\n    \"aggregations\": {\n      \"max_price\": {\n        \"max\": {\n          \"field\": \"taxful_total_price\"\n        }\n      }\n    }\n  }\n}"'
Request example
Run `POST _transform/_preview` to preview a transform that uses the pivot method. 
{
  "source": {
    "index": "kibana_sample_data_ecommerce"
  },
  "pivot": {
    "group_by": {
      "customer_id": {
        "terms": {
          "field": "customer_id",
          "missing_bucket": true
        }
      }
    },
    "aggregations": {
      "max_price": {
        "max": {
          "field": "taxful_total_price"
        }
      }
    }
  }
}
Response examples (200)
An abbreviated response from `POST _transform/_preview` that contains a preview a transform that uses the pivot method. 
{
  "preview": [
    {
      "max_price": 171,
      "customer_id": "10"
    },
    {
      "max_price": 233,
      "customer_id": "11"
    },
    {
      "max_price": 200,
      "customer_id": "12"
    },
    {
      "max_price": 301,
      "customer_id": "13"
    },
    {
      "max_price": 176,
      "customer_id": "14"
    },
    {
      "max_price": 2250,
      "customer_id": "15"
    },
    {
      "max_price": 170,
      "customer_id": "16"
    },
    {
      "max_price": 243,
      "customer_id": "17"
    },
    {
      "max_price": 154,
      "customer_id": "18"
    },
    {
      "max_price": 393,
      "customer_id": "19"
    },
    {
      "max_price": 165,
      "customer_id": "20"
    },
    {
      "max_price": 115,
      "customer_id": "21"
    },
    {
      "max_price": 192,
      "customer_id": "22"
    },
    {
      "max_price": 169,
      "customer_id": "23"
    },
    {
      "max_price": 230,
      "customer_id": "24"
    },
    {
      "max_price": 278,
      "customer_id": "25"
    },
    {
      "max_price": 200,
      "customer_id": "26"
    },
    {
      "max_price": 344,
      "customer_id": "27"
    },
    {
      "max_price": 175,
      "customer_id": "28"
    },
    {
      "max_price": 177,
      "customer_id": "29"
    },
    {
      "max_price": 190,
      "customer_id": "30"
    },
    {
      "max_price": 190,
      "customer_id": "31"
    },
    {
      "max_price": 205,
      "customer_id": "32"
    },
    {
      "max_price": 215,
      "customer_id": "33"
    },
    {
      "max_price": 270,
      "customer_id": "34"
    },
    {
      "max_price": 184,
      "customer_id": "36"
    },
    {
      "max_price": 222,
      "customer_id": "37"
    },
    {
      "max_price": 370,
      "customer_id": "38"
    },
    {
      "max_price": 240,
      "customer_id": "39"
    },
    {
      "max_price": 230,
      "customer_id": "4"
    },
    {
      "max_price": 229,
      "customer_id": "41"
    },
    {
      "max_price": 190,
      "customer_id": "42"
    },
    {
      "max_price": 150,
      "customer_id": "43"
    },
    {
      "max_price": 175,
      "customer_id": "44"
    },
    {
      "max_price": 190,
      "customer_id": "45"
    },
    {
      "max_price": 150,
      "customer_id": "46"
    },
    {
      "max_price": 310,
      "customer_id": "48"
    },
    {
      "max_price": 223,
      "customer_id": "49"
    },
    {
      "max_price": 283,
      "customer_id": "5"
    },
    {
      "max_price": 185,
      "customer_id": "50"
    },
    {
      "max_price": 190,
      "customer_id": "51"
    },
    {
      "max_price": 333,
      "customer_id": "52"
    },
    {
      "max_price": 165,
      "customer_id": "6"
    },
    {
      "max_price": 144,
      "customer_id": "7"
    },
    {
      "max_price": 198,
      "customer_id": "8"
    },
    {
      "max_price": 210,
      "customer_id": "9"
    }
  ],
  "generated_dest_index": {
    "mappings": {
      "_meta": {
        "_transform": {
          "transform": "transform-preview",
          "version": {
            "created": "10.0.0"
          },
          "creation_date_in_millis": 1712948905889
        },
        "created_by": "transform"
      },
      "properties": {
        "max_price": {
          "type": "half_float"
        },
        "customer_id": {
          "type": "keyword"
        }
      }
    },
    "settings": {
      "index": {
        "number_of_shards": "1",
        "auto_expand_replicas": "0-1"
      }
    },
    "aliases": {}
  }
}