Update user profile data | Elasticsearch API documentation (v8)

Update the connector features Technical preview

PUT /_connector/{connector_id}/_features

Update the connector features in the connector document. This API can be used to control the following aspects of a connector:

document-level security
incremental syncs
advanced sync rules
basic sync rules

Normally, the running connector service automatically manages these features. However, you can use this API to override the default behavior.

To sync data using self-managed connectors, you need to deploy the Elastic connector service on your own infrastructure. This service runs automatically on Elastic Cloud for Elastic managed connectors.

Path parameters

connector_id string Required

The unique identifier of the connector to be updated.

application/json

Body Required

features object Required
Hide features attributes Show features attributes object
- document_level_security object
  Hide document_level_security attribute Show document_level_security attribute object
  
  enabled boolean Required
- incremental_sync object
  Hide incremental_sync attribute Show incremental_sync attribute object
  
  enabled boolean Required
- native_connector_api_keys object
  Hide native_connector_api_keys attribute Show native_connector_api_keys attribute object
  
  enabled boolean Required
- sync_rules object
  Hide sync_rules attributes Show sync_rules attributes object
  
  advanced object
  
  Hide advanced attribute Show advanced attribute object
  
  enabled boolean Required
  
  basic object
  
  Hide basic attribute Show basic attribute object
  
  enabled boolean Required

Responses

200 application/json
Hide response attribute Show response attribute object
- result string Required
  
  Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_features

curl \
 --request PUT 'http://api.example.com/_connector/{connector_id}/_features' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"features\": {\n    \"document_level_security\": {\n      \"enabled\": true\n    },\n    \"incremental_sync\": {\n      \"enabled\": true\n    },\n    \"sync_rules\": {\n      \"advanced\": {\n        \"enabled\": false\n      },\n      \"basic\": {\n        \"enabled\": true\n      }\n    }\n  }\n}"'

Request examples

{
  "features": {
    "document_level_security": {
      "enabled": true
    },
    "incremental_sync": {
      "enabled": true
    },
    "sync_rules": {
      "advanced": {
        "enabled": false
      },
      "basic": {
        "enabled": true
      }
    }
  }
}

{
  "features": {
    "document_level_security": {
      "enabled": true
    }
  }
}

Response examples (200)

{
  "result": "updated"
}

Create a new document in the index Added in 5.0.0

PUT /{index}/_create/{id}

Api key auth Basic auth Bearer auth

You can index a new JSON document with the /<target>/_doc/ or /<target>/_create/<_id> APIs Using _create guarantees that the document is indexed only if it does not already exist. It returns a 409 response when a document with a same ID already exists in the index. To update an existing document, you must use the /<target>/_doc/ API.

If the Elasticsearch security features are enabled, you must have the following index privileges for the target data stream, index, or index alias:

To add a document using the PUT /<target>/_create/<_id> or POST /<target>/_create/<_id> request formats, you must have the create_doc, create, index, or write index privilege.
To automatically create a data stream or index with this API request, you must have the auto_configure, create_index, or manage index privilege.

Automatic data stream creation requires a matching index template with data stream enabled.

Automatically create data streams and indices

If the request's target doesn't exist and matches an index template with a data_stream definition, the index operation automatically creates the data stream.

If the target doesn't exist and doesn't match a data stream template, the operation automatically creates the index and applies any matching index templates.

NOTE: Elasticsearch includes several built-in index templates. To avoid naming collisions with these templates, refer to index pattern documentation.

If no mapping exists, the index operation creates a dynamic mapping. By default, new fields and objects are automatically added to the mapping if needed.

Automatic index creation is controlled by the action.auto_create_index setting. If it is true, any index can be created automatically. You can modify this setting to explicitly allow or block automatic creation of indices that match specified patterns or set it to false to turn off automatic index creation entirely. Specify a comma-separated list of patterns you want to allow or prefix each pattern with + or - to indicate whether it should be allowed or blocked. When a list is specified, the default behaviour is to disallow.

NOTE: The action.auto_create_index setting affects the automatic creation of indices only. It does not affect the creation of data streams.

Routing

By default, shard placement — or routing — is controlled by using a hash of the document's ID value. For more explicit control, the value fed into the hash function used by the router can be directly specified on a per-operation basis using the routing parameter.

When setting up explicit mapping, you can also use the _routing field to direct the index operation to extract the routing value from the document itself. This does come at the (very minimal) cost of an additional document parsing pass. If the _routing mapping is defined and set to be required, the index operation will fail if no routing value is provided or extracted.

NOTE: Data streams do not support custom routing unless they were created with the allow_custom_routing setting enabled in the template.

Distributed

The index operation is directed to the primary shard based on its route and performed on the actual node containing this shard. After the primary shard completes the operation, if needed, the update is distributed to applicable replicas.

Active shards

To improve the resiliency of writes to the system, indexing operations can be configured to wait for a certain number of active shard copies before proceeding with the operation. If the requisite number of active shard copies are not available, then the write operation must wait and retry, until either the requisite shard copies have started or a timeout occurs. By default, write operations only wait for the primary shards to be active before proceeding (that is to say wait_for_active_shards is 1). This default can be overridden in the index settings dynamically by setting index.write.wait_for_active_shards. To alter this behavior per operation, use the wait_for_active_shards request parameter.

Valid values are all or any positive integer up to the total number of configured copies per shard in the index (which is number_of_replicas+1). Specifying a negative value or a number greater than the number of shard copies will throw an error.

For example, suppose you have a cluster of three nodes, A, B, and C and you create an index index with the number of replicas set to 3 (resulting in 4 shard copies, one more copy than there are nodes). If you attempt an indexing operation, by default the operation will only ensure the primary copy of each shard is available before proceeding. This means that even if B and C went down and A hosted the primary shard copies, the indexing operation would still proceed with only one copy of the data. If wait_for_active_shards is set on the request to 3 (and all three nodes are up), the indexing operation will require 3 active shard copies before proceeding. This requirement should be met because there are 3 active nodes in the cluster, each one holding a copy of the shard. However, if you set wait_for_active_shards to all (or to 4, which is the same in this situation), the indexing operation will not proceed as you do not have all 4 copies of each shard active in the index. The operation will timeout unless a new node is brought up in the cluster to host the fourth copy of the shard.

It is important to note that this setting greatly reduces the chances of the write operation not writing to the requisite number of shard copies, but it does not completely eliminate the possibility, because this check occurs before the write operation starts. After the write operation is underway, it is still possible for replication to fail on any number of shard copies but still succeed on the primary. The _shards section of the API response reveals the number of shard copies on which replication succeeded and failed.

External documentation

Path parameters

index string Required

The name of the data stream or index to target. If the target doesn't exist and matches the name or wildcard (*) pattern of an index template with a data_stream definition, this request creates the data stream. If the target doesn't exist and doesn’t match a data stream template, this request creates the index.
id string Required

A unique identifier for the document. To automatically generate a document ID, use the POST /<target>/_doc/ request format.

Query parameters

if_primary_term number

Only perform the operation if the document has this primary term.
if_seq_no number

Only perform the operation if the document has this sequence number.
include_source_on_error boolean

True or false if to include the document source in the error message in case of parsing errors.
op_type string

Set to create to only index the document if it does not already exist (put if absent). If a document with the specified _id already exists, the indexing operation will fail. The behavior is the same as using the <index>/_create endpoint. If a document ID is specified, this paramater defaults to index. Otherwise, it defaults to create. If the request targets a data stream, an op_type of create is required.

Values are index or create.
pipeline string

The ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, setting the value to _none turns off the default ingest pipeline for this request. If a final pipeline is configured, it will always run regardless of the value of this parameter.
refresh string

If true, Elasticsearch refreshes the affected shards to make this operation visible to search. If wait_for, it waits for a refresh to make this operation visible to search. If false, it does nothing with refreshes.

Values are true, false, or wait_for.
require_alias boolean

If true, the destination must be an index alias.
require_data_stream boolean

If true, the request's actions must target a data stream (existing or to be created).
routing string

A custom value that is used to route operations to a specific shard.
timeout string

The period the request waits for the following operations: automatic index creation, dynamic mapping updates, waiting for active shards. Elasticsearch waits for at least the specified timeout period before failing. The actual wait time could be longer, particularly when multiple waits occur.

This parameter is useful for situations where the primary shard assigned to perform the operation might not be available when the operation runs. Some reasons for this might be that the primary shard is currently recovering from a gateway or undergoing relocation. By default, the operation will wait on the primary shard to become available for at least 1 minute before failing and responding with an error. The actual wait time could be longer, particularly when multiple waits occur.
version number

The explicit version number for concurrency control. It must be a non-negative long number.
version_type string

The version type.

Values are internal, external, external_gte, or force.
wait_for_active_shards number | string

The number of shard copies that must be active before proceeding with the operation. You can set it to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). The default value of 1 means it waits for each primary shard to be active.

application/json

Body Required

object

Responses

200 application/json
Hide response attributes Show response attributes object
- _id string Required
- _index string Required
- _primary_term number
  
  The primary term assigned to the document for the indexing operation.
- result string Required
  
  Values are created, updated, deleted, not_found, or noop.
- _seq_no number
- _shards object Required
  
  Hide _shards attributes Show _shards attributes object
  
  failed number Required
  
  successful number Required
  
  total number Required
  
  failures array[object]
  
  Hide failures attributes Show failures attributes object
  
  index string
  
  node string
  
  reason object Required
  
  Hide reason attributes Show reason attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  shard number Required
  
  status string
  
  skipped number
- _version number Required
- forced_refresh boolean

PUT /{index}/_create/{id}

curl \
 --request PUT 'http://api.example.com/{index}/_create/{id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"@timestamp\": \"2099-11-15T13:12:00\",\n  \"message\": \"GET /search HTTP/1.1 200 1070000\",\n  \"user\": {\n    \"id\": \"kimchy\"\n  }\n}"'

Request example

Run `PUT my-index-000001/_create/1` to index a document into the `my-index-000001` index if no document with that ID exists.

{
  "@timestamp": "2099-11-15T13:12:00",
  "message": "GET /search HTTP/1.1 200 1070000",
  "user": {
    "id": "kimchy"
  }
}

Response examples (200)

{
  "_id": "string",
  "_index": "string",
  "_primary_term": 42.0,
  "result": "created",
  "_seq_no": 42.0,
  "_shards": {
    "failed": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "failures": [
      {
        "index": "string",
        "node": "string",
        "reason": {
          "type": "string",
          "reason": "string",
          "stack_trace": "string",
          "caused_by": {},
          "root_cause": [
            {}
          ],
          "suppressed": [
            {}
          ]
        },
        "shard": 42.0,
        "status": "string"
      }
    ],
    "skipped": 42.0
  },
  "_version": 42.0,
  "forced_refresh": true
}

Get term vector information

GET /{index}/_termvectors

Api key auth Basic auth Bearer auth

Get information and statistics about terms in the fields of a particular document.

You can retrieve term vectors for documents stored in the index or for artificial documents passed in the body of the request. You can specify the fields you are interested in through the fields parameter or by adding the fields to the request body. For example:

GET /my-index-000001/_termvectors/1?fields=message

Fields can be specified using wildcards, similar to the multi match query.

Term vectors are real-time by default, not near real-time. This can be changed by setting realtime parameter to false.

You can request three types of values: term information, term statistics, and field statistics. By default, all term information and field statistics are returned for all fields but term statistics are excluded.

Term information

term frequency in the field (always returned)
term positions (positions: true)
start and end offsets (offsets: true)
term payloads (payloads: true), as base64 encoded bytes

If the requested information wasn't stored in the index, it will be computed on the fly if possible. Additionally, term vectors could be computed for documents not even existing in the index, but instead provided by the user.

Start and end offsets assume UTF-16 encoding is being used. If you want to use these offsets in order to get the original text that produced this token, you should make sure that the string you are taking a sub-string of is also encoded using UTF-16.

Behaviour

The term and field statistics are not accurate. Deleted documents are not taken into account. The information is only retrieved for the shard the requested document resides in. The term and field statistics are therefore only useful as relative measures whereas the absolute numbers have no meaning in this context. By default, when requesting term vectors of artificial documents, a shard to get the statistics from is randomly selected. Use routing only to hit a particular shard.

Path parameters

index string Required

The name of the index that contains the document.

Query parameters

fields string | array[string]

A comma-separated list or wildcard expressions of fields to include in the statistics. It is used as the default list unless a specific field list is provided in the completion_fields or fielddata_fields parameters.
field_statistics boolean
If true, the response includes:
- The document count (how many documents contain this field).
- The sum of document frequencies (the sum of document frequencies for all terms in this field).
- The sum of total term frequencies (the sum of total term frequencies of each term in this field).
offsets boolean

If true, the response includes term offsets.
payloads boolean

If true, the response includes term payloads.
positions boolean

If true, the response includes term positions.
preference string

The node or shard the operation should be performed on. It is random by default.
realtime boolean

If true, the request is real-time as opposed to near-real-time.
routing string

A custom value that is used to route operations to a specific shard.
term_statistics boolean
If true, the response includes:
- The total term frequency (how often a term occurs in all documents).
- The document frequency (the number of documents containing the current term).
By default these values are not returned since term statistics can have a serious performance impact.
version number

If true, returns the document version as part of a hit.
version_type string

The version type.

Values are internal, external, external_gte, or force.

application/json

Body

doc object

An artificial document (a document not present in the index) for which you want to retrieve term vectors.
filter object
Hide filter attributes Show filter attributes object
- max_doc_freq number
  
  Ignore words which occur in more than this many docs. Defaults to unbounded.
- max_num_terms number
  
  The maximum number of terms that must be returned per field.
- max_term_freq number
  
  Ignore words with more than this frequency in the source doc. It defaults to unbounded.
- max_word_length number
  
  The maximum word length above which words will be ignored. Defaults to unbounded.
- min_doc_freq number
  
  Ignore terms which do not occur in at least this many docs.
- min_term_freq number
  
  Ignore words with less than this frequency in the source doc.
- min_word_length number
  
  The minimum word length below which words will be ignored.
per_field_analyzer object

Override the default per-field analyzer. This is useful in order to generate term vectors in any fashion, especially when using artificial documents. When providing an analyzer for a field that already stores term vectors, the term vectors will be regenerated.
Hide per_field_analyzer attribute Show per_field_analyzer attribute object
- * string Additional properties
fields string | array[string]
field_statistics boolean
If true, the response includes:
- The document count (how many documents contain this field).
- The sum of document frequencies (the sum of document frequencies for all terms in this field).
- The sum of total term frequencies (the sum of total term frequencies of each term in this field).
offsets boolean

If true, the response includes term offsets.
payloads boolean

If true, the response includes term payloads.
positions boolean

If true, the response includes term positions.
term_statistics boolean
If true, the response includes:
- The total term frequency (how often a term occurs in all documents).
- The document frequency (the number of documents containing the current term).
By default these values are not returned since term statistics can have a serious performance impact.
routing string
version number
version_type string

Values are internal, external, external_gte, or force.

Responses

200 application/json
Hide response attributes Show response attributes object
- found boolean Required
- _id string
- _index string Required
- term_vectors object
  
  Hide term_vectors attribute Show term_vectors attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  field_statistics object
  
  Hide field_statistics attributes Show field_statistics attributes object
  
  doc_count number Required
  
  sum_doc_freq number Required
  
  sum_ttf number Required
  
  terms object Required
  
  Hide terms attribute Show terms attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  doc_freq number
  
  score number
  
  term_freq number Required
  
  tokens array[object]
  
  ttf number
- took number Required
- _version number Required

GET /{index}/_termvectors

curl \
 --request GET 'http://api.example.com/{index}/_termvectors' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"fields\" : [\"text\"],\n  \"offsets\" : true,\n  \"payloads\" : true,\n  \"positions\" : true,\n  \"term_statistics\" : true,\n  \"field_statistics\" : true\n}"'

Request examples

Run `GET /my-index-000001/_termvectors/1` to return all information and statistics for field `text` in document 1.

{
  "fields" : ["text"],
  "offsets" : true,
  "payloads" : true,
  "positions" : true,
  "term_statistics" : true,
  "field_statistics" : true
}

Run `GET /my-index-000001/_termvectors/1` to set per-field analyzers. A different analyzer than the one at the field may be provided by using the `per_field_analyzer` parameter.

{
  "doc" : {
    "fullname" : "John Doe",
    "text" : "test test test"
  },
  "fields": ["fullname"],
  "per_field_analyzer" : {
    "fullname": "keyword"
  }
}

Run `GET /imdb/_termvectors` to filter the terms returned based on their tf-idf scores. It returns the three most "interesting" keywords from the artificial document having the given "plot" field value. Notice that the keyword "Tony" or any stop words are not part of the response, as their tf-idf must be too low.

{
  "doc": {
    "plot": "When wealthy industrialist Tony Stark is forced to build an armored suit after a life-threatening incident, he ultimately decides to use its technology to fight against evil."
  },
  "term_statistics": true,
  "field_statistics": true,
  "positions": false,
  "offsets": false,
  "filter": {
    "max_num_terms": 3,
    "min_term_freq": 1,
    "min_doc_freq": 1
  }
}

Run `GET /my-index-000001/_termvectors/1`. Term vectors which are not explicitly stored in the index are automatically computed on the fly. This request returns all information and statistics for the fields in document 1, even though the terms haven't been explicitly stored in the index. Note that for the field text, the terms are not regenerated.

{
  "fields" : ["text", "some_field_without_term_vectors"],
  "offsets" : true,
  "positions" : true,
  "term_statistics" : true,
  "field_statistics" : true
}

Run `GET /my-index-000001/_termvectors`. Term vectors can be generated for artificial documents, that is for documents not present in the index. If dynamic mapping is turned on (default), the document fields not in the original mapping will be dynamically created.

{
  "doc" : {
    "fullname" : "John Doe",
    "text" : "test test test"
  }
}

Response examples (200)

A successful response from `GET /my-index-000001/_termvectors/1`.

{
  "_index": "my-index-000001",
  "_id": "1",
  "_version": 1,
  "found": true,
  "took": 6,
  "term_vectors": {
    "text": {
      "field_statistics": {
        "sum_doc_freq": 4,
        "doc_count": 2,
        "sum_ttf": 6
      },
      "terms": {
        "test": {
          "doc_freq": 2,
          "ttf": 4,
          "term_freq": 3,
          "tokens": [
            {
              "position": 0,
              "start_offset": 0,
              "end_offset": 4,
              "payload": "d29yZA=="
            },
            {
              "position": 1,
              "start_offset": 5,
              "end_offset": 9,
              "payload": "d29yZA=="
            },
            {
              "position": 2,
              "start_offset": 10,
              "end_offset": 14,
              "payload": "d29yZA=="
            }
          ]
        }
      }
    }
  }
}

A successful response from `GET /my-index-000001/_termvectors` with `per_field_analyzer` in the request body.

{
  "_index": "my-index-000001",
  "_version": 0,
  "found": true,
  "took": 6,
  "term_vectors": {
    "fullname": {
      "field_statistics": {
          "sum_doc_freq": 2,
          "doc_count": 4,
          "sum_ttf": 4
      },
      "terms": {
          "John Doe": {
            "term_freq": 1,
            "tokens": [
                {
                  "position": 0,
                  "start_offset": 0,
                  "end_offset": 8
                }
            ]
          }
      }
    }
  }
}

A successful response from `GET /my-index-000001/_termvectors` with a `filter` in the request body.

{
  "_index": "imdb",
  "_version": 0,
  "found": true,
  "term_vectors": {
      "plot": {
        "field_statistics": {
            "sum_doc_freq": 3384269,
            "doc_count": 176214,
            "sum_ttf": 3753460
        },
        "terms": {
            "armored": {
              "doc_freq": 27,
              "ttf": 27,
              "term_freq": 1,
              "score": 9.74725
            },
            "industrialist": {
              "doc_freq": 88,
              "ttf": 88,
              "term_freq": 1,
              "score": 8.590818
            },
            "stark": {
              "doc_freq": 44,
              "ttf": 47,
              "term_freq": 1,
              "score": 9.272792
            }
        }
      }
  }
}

Get an enrich policy Added in 7.5.0

GET /_enrich/policy/{name}

Api key auth Basic auth Bearer auth

Returns information about an enrich policy.

Path parameters

name string | array[string] Required

Comma-separated list of enrich policy names used to limit the request. To return information for all enrich policies, omit this parameter.

Query parameters

master_timeout string

Period to wait for a connection to the master node.

Responses

200 application/json
Hide response attribute Show response attribute object
- policies array[object] Required
  
  Hide policies attribute Show policies attribute object
  
  config object Required
  
  Hide config attribute Show config attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  enrich_fields string | array[string] Required
  
  indices string | array[string] Required
  
  match_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  query object
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  External documentation
  
  name string
  
  elasticsearch_version string

GET /_enrich/policy/{name}

curl \
 --request GET 'http://api.example.com/_enrich/policy/{name}' \
 --header "Authorization: $API_KEY"

Response examples (200)

{
  "policies": [
    {
      "config": {
        "additionalProperty1": {
          "enrich_fields": "string",
          "indices": "string",
          "match_field": "string",
          "query": {},
          "name": "string",
          "elasticsearch_version": "string"
        },
        "additionalProperty2": {
          "enrich_fields": "string",
          "indices": "string",
          "match_field": "string",
          "query": {},
          "name": "string",
          "elasticsearch_version": "string"
        }
      }
    }
  ]
}

Stop async ES|QL query Added in 8.18.0

POST /_query/async/{id}/stop

Api key auth Basic auth Bearer auth

This API interrupts the query execution and returns the results so far. If the Elasticsearch security features are enabled, only the user who first submitted the ES|QL query can stop it.

External documentation

Path parameters

id string Required

The unique identifier of the query. A query ID is provided in the ES|QL async query API response for a query that does not complete in the designated time. A query ID is also provided when the request was submitted with the keep_on_completion parameter set to true.

Query parameters

drop_null_columns boolean

Indicates whether columns that are entirely null will be removed from the columns and values portion of the results. If true, the response will include an extra section under the name all_columns which has the name of all the columns.

Responses

200 application/json

POST /_query/async/{id}/stop

curl \
 --request POST 'http://api.example.com/_query/async/{id}/stop' \
 --header "Authorization: $API_KEY"

Response examples (200)

{}

Simulate a pipeline Added in 5.0.0

POST /_ingest/pipeline/_simulate

Api key auth Basic auth Bearer auth

Run an ingest pipeline against a set of provided documents. You can either specify an existing pipeline to use with the provided documents or supply a pipeline definition in the body of the request.

Query parameters

verbose boolean

If true, the response includes output data for each processor in the executed pipeline.

application/json

Body Required

docs array[object] Required

Sample documents to test in the pipeline.
Hide docs attributes Show docs attributes object
- _id string
- _index string
- _source object Required
  
  JSON body for the document.
pipeline object Additional properties
Hide pipeline attributes Show pipeline attributes object
- description string
  
  Description of the ingest pipeline.
- on_failure array[object]
  
  Processors to run immediately after a processor failure.
  Hide on_failure attributes Show on_failure attributes object
  
  append object
  
  Hide append attributes Show append attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  value object | array[object] Required
  
  The value to be appended. Supports template snippets.
  
  One of:
  object-1 object array-2 array[object]
  
  allow_duplicates boolean
  
  If false, the processor does not append values already present in the field.
  
  attachment object
  
  Hide attachment attributes Show attachment attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  indexed_chars number
  
  The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.
  
  indexed_chars_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  remove_binary boolean
  
  If true, the binary field will be removed from the document
  
  resource_name string
  
  Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.
  
  bytes object
  
  Hide bytes attributes Show bytes attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  circle object
  
  Hide circle attributes Show circle attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  error_distance number Required
  
  The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  shape_type string Required
  
  Values are geo_shape or shape.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  community_id object
  
  Hide community_id attributes Show community_id attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  source_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  iana_number string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_type string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_code string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  transport string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  seed number
  
  Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  convert object
  
  Hide convert attributes Show convert attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  type string Required
  
  Values are integer, long, double, float, boolean, ip, string, or auto.
  
  csv object
  
  Hide csv attributes Show csv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  empty_value object
  
  Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  quote string
  
  Quote used in CSV, has to be single character string.
  
  separator string
  
  Separator used in CSV, has to be single character string.
  
  target_fields string | array[string] Required
  
  trim boolean
  
  Trim whitespaces in unquoted fields.
  
  date object
  
  Hide date attributes Show date attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  formats array[string] Required
  
  An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  locale string
  
  The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  timezone string
  
  The timezone to use when parsing the date. Supports template snippets.
  
  output_format string
  
  The format to use when writing the date to target_field. Must be a valid java time pattern.
  
  date_index_name object
  
  Hide date_index_name attributes Show date_index_name attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  date_formats array[string] Required
  
  An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  date_rounding string Required
  
  How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  index_name_format string
  
  The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.
  
  index_name_prefix string
  
  A prefix of the index name to be prepended before the printed date. Supports template snippets.
  
  locale string
  
  The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.
  
  timezone string
  
  The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.
  
  dissect object
  
  Hide dissect attributes Show dissect attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  append_separator string
  
  The character(s) that separate the appended fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to apply to the field.
  
  dot_expander object
  
  Hide dot_expander attributes Show dot_expander attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  override boolean
  
  Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.
  
  path string
  
  The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.
  
  drop object
  
  Hide drop attributes Show drop attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  enrich object
  
  Hide enrich attributes Show enrich attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  max_matches number
  
  The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.
  
  override boolean
  
  If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  policy_name string Required
  
  The name of the enrich policy to use.
  
  shape_relation string
  
  Values are intersects, disjoint, within, or contains.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  fail object
  
  Hide fail attributes Show fail attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  message string Required
  
  The error message thrown by the processor. Supports template snippets.
  
  fingerprint object
  
  Hide fingerprint attributes Show fingerprint attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  fields string | array[string] Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  salt string
  
  Salt value for the hash function.
  
  method string
  
  Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.
  
  ignore_missing boolean
  
  If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.
  
  foreach object
  
  Hide foreach attributes Show foreach attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true, the processor silently exits without changing the document if the field is null or missing.
  
  processor object Required
  
  ip_location object
  
  Hide ip_location attributes Show ip_location attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found IP location data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the IP location lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  geo_grid object
  
  Hide geo_grid attributes Show geo_grid attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to interpret as a geo-tile.= The field format is determined by the tile_type.
  
  tile_type string Required
  
  Values are geotile, geohex, or geohash.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  parent_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  non_children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  precision_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_format string
  
  Values are geojson or wkt.
  
  geoip object
  
  Hide geoip attributes Show geoip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found geoip data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the geoip lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  grok object
  
  Hide grok attributes Show grok attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  ecs_compatibility string
  
  Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern_definitions object
  
  A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.
  
  Hide pattern_definitions attribute Show pattern_definitions attribute object
  
  * string Additional properties
  
  patterns array[string] Required
  
  An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.
  
  trace_match boolean
  
  When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.
  
  gsub object
  
  Hide gsub attributes Show gsub attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to be replaced.
  
  replacement string Required
  
  The string to replace the matching patterns with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  html_strip object
  
  Hide html_strip attributes Show html_strip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document,
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  inference object
  
  Hide inference attributes Show inference attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  model_id string Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_map object
  
  Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.
  
  Hide field_map attribute Show field_map attribute object
  
  * object Additional properties
  
  inference_config object
  
  Hide inference_config attributes Show inference_config attributes object
  
  regression object
  
  classification object
  
  input_output object | array[object]
  
  Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.
  
  One of:
  InputConfig object array-2 array[object]
  
  ignore_missing boolean
  
  If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.
  
  join object
  
  Hide join attributes Show join attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  separator string Required
  
  The separator character.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  json object
  
  Hide json attributes Show json attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  add_to_root boolean
  
  Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.
  
  add_to_root_conflict_strategy string
  
  Values are replace or merge.
  
  allow_duplicate_keys boolean
  
  When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  kv object
  
  Hide kv attributes Show kv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  exclude_keys array[string]
  
  List of keys to exclude from document.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_split string Required
  
  Regex pattern to use for splitting key-value pairs.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  include_keys array[string]
  
  List of keys to filter and insert into document. Defaults to including all keys.
  
  prefix string
  
  Prefix to be added to extracted keys.
  
  strip_brackets boolean
  
  If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  trim_key string
  
  String of characters to trim from extracted keys.
  
  trim_value string
  
  String of characters to trim from extracted values.
  
  value_split string Required
  
  Regex pattern to use for splitting the key from the value within a key-value pair.
  
  lowercase object
  
  Hide lowercase attributes Show lowercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  network_direction object
  
  Hide network_direction attributes Show network_direction attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  internal_networks array[string]
  
  List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.
  
  internal_networks_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  pipeline object
  
  Hide pipeline attributes Show pipeline attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  name string Required
  
  ignore_missing_pipeline boolean
  
  Whether to ignore missing pipelines instead of failing.
  
  redact object
  
  Hide redact attributes Show redact attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  patterns array[string] Required
  
  A list of grok expressions to match and redact named captures with
  
  pattern_definitions object
  
  Hide pattern_definitions attribute Show pattern_definitions attribute object
  
  * string Additional properties
  
  prefix string
  
  Start a redacted section with this token
  
  suffix string
  
  End a redacted section with this token
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  skip_if_unlicensed boolean
  
  If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document
  
  trace_redact boolean
  
  If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted
  
  registered_domain object
  
  Hide registered_domain attributes Show registered_domain attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  remove object
  
  Hide remove attributes Show remove attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string | array[string] Required
  
  keep string | array[string]
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  rename object
  
  Hide rename attributes Show rename attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  reroute object
  
  Hide reroute attributes Show reroute attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  destination string
  
  A static value for the target. Can’t be set when the dataset or namespace option is set.
  
  dataset string | array[string]
  
  Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.
  
  Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.
  
  default {{data_stream.dataset}}
  
  One of:
  string-1 string array-2 array[string]
  
  namespace string | array[string]
  
  Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.
  
  Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.
  
  default {{data_stream.namespace}}
  
  One of:
  string-1 string array-2 array[string]
  
  script object
  
  Hide script attributes Show script attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  id string
  
  lang string
  
  Script language.
  
  params object
  
  Object containing parameters for the script.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  source string
  
  Inline script. If no id is specified, this parameter is required.
  
  set object
  
  Hide set attributes Show set attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  copy_from string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_empty_value boolean
  
  If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.
  
  media_type string
  
  The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.
  
  override boolean
  
  If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  value object
  
  The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.
  
  set_security_user object
  
  Hide set_security_user attributes Show set_security_user attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what user related properties are added to the field.
  
  sort object
  
  Hide sort attributes Show sort attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  order string
  
  Values are asc or desc.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  split object
  
  Hide split attributes Show split attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  preserve_trailing boolean
  
  Preserves empty trailing fields, if any.
  
  separator string Required
  
  A regex which matches the separator, for example, , or \s+.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  terminate object
  
  Hide terminate attributes Show terminate attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  trim object
  
  Hide trim attributes Show trim attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uppercase object
  
  Hide uppercase attributes Show uppercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  urldecode object
  
  Hide urldecode attributes Show urldecode attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uri_parts object
  
  Hide uri_parts attributes Show uri_parts attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  keep_original boolean
  
  If true, the processor copies the unparsed URI to <target_field>.original.
  
  remove_if_successful boolean
  
  If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  user_agent object
  
  Hide user_agent attributes Show user_agent attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  regex_file string
  
  The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what properties are added to target_field.
  
  Values are name, os, device, original, or version.
  
  extract_device_type boolean Beta
  
  Extracts device type from the user agent string on a best-effort basis.
- processors array[object]
  
  Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.
  Hide processors attributes Show processors attributes object
  
  append object
  
  Hide append attributes Show append attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  value object | array[object] Required
  
  The value to be appended. Supports template snippets.
  
  One of:
  object-1 object array-2 array[object]
  
  allow_duplicates boolean
  
  If false, the processor does not append values already present in the field.
  
  attachment object
  
  Hide attachment attributes Show attachment attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  indexed_chars number
  
  The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.
  
  indexed_chars_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  remove_binary boolean
  
  If true, the binary field will be removed from the document
  
  resource_name string
  
  Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.
  
  bytes object
  
  Hide bytes attributes Show bytes attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  circle object
  
  Hide circle attributes Show circle attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  error_distance number Required
  
  The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  shape_type string Required
  
  Values are geo_shape or shape.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  community_id object
  
  Hide community_id attributes Show community_id attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  source_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  iana_number string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_type string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_code string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  transport string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  seed number
  
  Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  convert object
  
  Hide convert attributes Show convert attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  type string Required
  
  Values are integer, long, double, float, boolean, ip, string, or auto.
  
  csv object
  
  Hide csv attributes Show csv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  empty_value object
  
  Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  quote string
  
  Quote used in CSV, has to be single character string.
  
  separator string
  
  Separator used in CSV, has to be single character string.
  
  target_fields string | array[string] Required
  
  trim boolean
  
  Trim whitespaces in unquoted fields.
  
  date object
  
  Hide date attributes Show date attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  formats array[string] Required
  
  An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  locale string
  
  The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  timezone string
  
  The timezone to use when parsing the date. Supports template snippets.
  
  output_format string
  
  The format to use when writing the date to target_field. Must be a valid java time pattern.
  
  date_index_name object
  
  Hide date_index_name attributes Show date_index_name attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  date_formats array[string] Required
  
  An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  date_rounding string Required
  
  How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  index_name_format string
  
  The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.
  
  index_name_prefix string
  
  A prefix of the index name to be prepended before the printed date. Supports template snippets.
  
  locale string
  
  The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.
  
  timezone string
  
  The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.
  
  dissect object
  
  Hide dissect attributes Show dissect attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  append_separator string
  
  The character(s) that separate the appended fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to apply to the field.
  
  dot_expander object
  
  Hide dot_expander attributes Show dot_expander attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  override boolean
  
  Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.
  
  path string
  
  The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.
  
  drop object
  
  Hide drop attributes Show drop attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  enrich object
  
  Hide enrich attributes Show enrich attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  max_matches number
  
  The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.
  
  override boolean
  
  If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  policy_name string Required
  
  The name of the enrich policy to use.
  
  shape_relation string
  
  Values are intersects, disjoint, within, or contains.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  fail object
  
  Hide fail attributes Show fail attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  message string Required
  
  The error message thrown by the processor. Supports template snippets.
  
  fingerprint object
  
  Hide fingerprint attributes Show fingerprint attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  fields string | array[string] Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  salt string
  
  Salt value for the hash function.
  
  method string
  
  Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.
  
  ignore_missing boolean
  
  If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.
  
  foreach object
  
  Hide foreach attributes Show foreach attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true, the processor silently exits without changing the document if the field is null or missing.
  
  processor object Required
  
  ip_location object
  
  Hide ip_location attributes Show ip_location attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found IP location data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the IP location lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  geo_grid object
  
  Hide geo_grid attributes Show geo_grid attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to interpret as a geo-tile.= The field format is determined by the tile_type.
  
  tile_type string Required
  
  Values are geotile, geohex, or geohash.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  parent_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  non_children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  precision_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_format string
  
  Values are geojson or wkt.
  
  geoip object
  
  Hide geoip attributes Show geoip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found geoip data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the geoip lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  grok object
  
  Hide grok attributes Show grok attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  ecs_compatibility string
  
  Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern_definitions object
  
  A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.
  
  Hide pattern_definitions attribute Show pattern_definitions attribute object
  
  * string Additional properties
  
  patterns array[string] Required
  
  An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.
  
  trace_match boolean
  
  When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.
  
  gsub object
  
  Hide gsub attributes Show gsub attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to be replaced.
  
  replacement string Required
  
  The string to replace the matching patterns with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  html_strip object
  
  Hide html_strip attributes Show html_strip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document,
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  inference object
  
  Hide inference attributes Show inference attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  model_id string Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_map object
  
  Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.
  
  Hide field_map attribute Show field_map attribute object
  
  * object Additional properties
  
  inference_config object
  
  Hide inference_config attributes Show inference_config attributes object
  
  regression object
  
  classification object
  
  input_output object | array[object]
  
  Input fields for inference and output (destination) fields for the inference results. This option is incompatible with the target_field and field_map options.
  
  One of:
  InputConfig object array-2 array[object]
  
  ignore_missing boolean
  
  If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.
  
  join object
  
  Hide join attributes Show join attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  separator string Required
  
  The separator character.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  json object
  
  Hide json attributes Show json attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  add_to_root boolean
  
  Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.
  
  add_to_root_conflict_strategy string
  
  Values are replace or merge.
  
  allow_duplicate_keys boolean
  
  When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  kv object
  
  Hide kv attributes Show kv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  exclude_keys array[string]
  
  List of keys to exclude from document.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_split string Required
  
  Regex pattern to use for splitting key-value pairs.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  include_keys array[string]
  
  List of keys to filter and insert into document. Defaults to including all keys.
  
  prefix string
  
  Prefix to be added to extracted keys.
  
  strip_brackets boolean
  
  If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  trim_key string
  
  String of characters to trim from extracted keys.
  
  trim_value string
  
  String of characters to trim from extracted values.
  
  value_split string Required
  
  Regex pattern to use for splitting the key from the value within a key-value pair.
  
  lowercase object
  
  Hide lowercase attributes Show lowercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  network_direction object
  
  Hide network_direction attributes Show network_direction attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  internal_networks array[string]
  
  List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.
  
  internal_networks_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  pipeline object
  
  Hide pipeline attributes Show pipeline attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  name string Required
  
  ignore_missing_pipeline boolean
  
  Whether to ignore missing pipelines instead of failing.
  
  redact object
  
  Hide redact attributes Show redact attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  patterns array[string] Required
  
  A list of grok expressions to match and redact named captures with
  
  pattern_definitions object
  
  Hide pattern_definitions attribute Show pattern_definitions attribute object
  
  * string Additional properties
  
  prefix string
  
  Start a redacted section with this token
  
  suffix string
  
  End a redacted section with this token
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  skip_if_unlicensed boolean
  
  If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document
  
  trace_redact boolean
  
  If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted
  
  registered_domain object
  
  Hide registered_domain attributes Show registered_domain attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  remove object
  
  Hide remove attributes Show remove attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string | array[string] Required
  
  keep string | array[string]
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  rename object
  
  Hide rename attributes Show rename attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  reroute object
  
  Hide reroute attributes Show reroute attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  destination string
  
  A static value for the target. Can’t be set when the dataset or namespace option is set.
  
  dataset string | array[string]
  
  Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.
  
  Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.
  
  default {{data_stream.dataset}}
  
  One of:
  string-1 string array-2 array[string]
  
  namespace string | array[string]
  
  Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.
  
  Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.
  
  default {{data_stream.namespace}}
  
  One of:
  string-1 string array-2 array[string]
  
  script object
  
  Hide script attributes Show script attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  id string
  
  lang string
  
  Script language.
  
  params object
  
  Object containing parameters for the script.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  source string
  
  Inline script. If no id is specified, this parameter is required.
  
  set object
  
  Hide set attributes Show set attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  copy_from string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_empty_value boolean
  
  If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.
  
  media_type string
  
  The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.
  
  override boolean
  
  If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  value object
  
  The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.
  
  set_security_user object
  
  Hide set_security_user attributes Show set_security_user attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what user related properties are added to the field.
  
  sort object
  
  Hide sort attributes Show sort attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  order string
  
  Values are asc or desc.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  split object
  
  Hide split attributes Show split attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  preserve_trailing boolean
  
  Preserves empty trailing fields, if any.
  
  separator string Required
  
  A regex which matches the separator, for example, , or \s+.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  terminate object
  
  Hide terminate attributes Show terminate attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  trim object
  
  Hide trim attributes Show trim attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uppercase object
  
  Hide uppercase attributes Show uppercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  urldecode object
  
  Hide urldecode attributes Show urldecode attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uri_parts object
  
  Hide uri_parts attributes Show uri_parts attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  keep_original boolean
  
  If true, the processor copies the unparsed URI to <target_field>.original.
  
  remove_if_successful boolean
  
  If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  user_agent object
  
  Hide user_agent attributes Show user_agent attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if string
  
  Conditionally execute the processor.
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  regex_file string
  
  The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what properties are added to target_field.
  
  Values are name, os, device, original, or version.
  
  extract_device_type boolean Beta
  
  Extracts device type from the user agent string on a best-effort basis.
- version number
- deprecated boolean
  
  Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.
- _meta object
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties

Responses

200 application/json
Hide response attribute Show response attribute object
- docs array[object] Required
  
  Hide docs attributes Show docs attributes object
  
  doc object
  
  Hide doc attributes Show doc attributes object
  
  _id string Required
  
  _index string Required
  
  _ingest object Required
  
  Hide _ingest attributes Show _ingest attributes object
  
  _redact object
  
  Hide _redact attribute Show _redact attribute object
  
  _is_redacted boolean Required
  
  indicates if document has been redacted
  
  timestamp string
  
  pipeline string
  
  _routing string
  
  Value used to send the document to a specific primary shard.
  
  _source object Required
  
  JSON body for the document.
  
  Hide _source attribute Show _source attribute object
  
  * object Additional properties
  
  _version number | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  VersionNumber number StringifiedVersionNumber string
  
  _version_type string
  
  Values are internal, external, external_gte, or force.
  
  error object
  
  Hide error attributes Show error attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  processor_results array[object]
  
  Hide processor_results attributes Show processor_results attributes object
  
  doc object
  
  Hide doc attributes Show doc attributes object
  
  _id string Required
  
  _index string Required
  
  _ingest object Required
  
  _routing string
  
  Value used to send the document to a specific primary shard.
  
  _source object Required
  
  JSON body for the document.
  
  _version
  
  _version_type string
  
  Values are internal, external, external_gte, or force.
  
  tag string
  
  processor_type string
  
  status string
  
  Values are success, failure, simulated, or throttled.
  
  description string
  
  ignored_error object
  
  Hide ignored_error attributes Show ignored_error attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  error object
  
  Hide error attributes Show error attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]

POST /_ingest/pipeline/_simulate

curl \
 --request POST 'http://api.example.com/_ingest/pipeline/_simulate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"pipeline\" :\n  {\n    \"description\": \"_description\",\n    \"processors\": [\n      {\n        \"set\" : {\n          \"field\" : \"field2\",\n          \"value\" : \"_value\"\n        }\n      }\n    ]\n  },\n  \"docs\": [\n    {\n      \"_index\": \"index\",\n      \"_id\": \"id\",\n      \"_source\": {\n        \"foo\": \"bar\"\n      }\n    },\n    {\n      \"_index\": \"index\",\n      \"_id\": \"id\",\n      \"_source\": {\n        \"foo\": \"rab\"\n      }\n    }\n  ]\n}"'

Request example

You can specify the used pipeline either in the request body or as a path parameter.

{
  "pipeline" :
  {
    "description": "_description",
    "processors": [
      {
        "set" : {
          "field" : "field2",
          "value" : "_value"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}

Response examples (200)

A successful response for running an ingest pipeline against a set of provided documents.

{
   "docs": [
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "bar"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.187Z"
            }
         }
      },
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "rab"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.188Z"
            }
         }
      }
   ]
}

Get model snapshots info Added in 5.4.0

POST /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}

Api key auth Basic auth Bearer auth

Path parameters

job_id string Required

Identifier for the anomaly detection job.
snapshot_id string Required

A numerical character string that uniquely identifies the model snapshot. You can get information for multiple snapshots by using a comma-separated list or a wildcard expression. You can get all snapshots by using _all, by specifying * as the snapshot ID, or by omitting the snapshot ID.

Query parameters

desc boolean

If true, the results are sorted in descending order.
end string | number

Returns snapshots with timestamps earlier than this time.
from number

Skips the specified number of snapshots.
size number

Specifies the maximum number of snapshots to obtain.
sort string

Specifies the sort field for the requested snapshots. By default, the snapshots are sorted by their timestamp.
start string | number

Returns snapshots with timestamps after this time.

application/json

Body

desc boolean

Refer to the description for the desc query parameter.
end string | number

A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

One of:
DateTime string UnitMillis number
page object
Hide page attributes Show page attributes object
- from number
  
  Skips the specified number of items.
- size number
  
  Specifies the maximum number of items to obtain.
sort string

Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
start string | number

A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

One of:
DateTime string UnitMillis number

Responses

200 application/json
Hide response attributes Show response attributes object
- count number Required
- model_snapshots array[object] Required
  
  Hide model_snapshots attributes Show model_snapshots attributes object
  
  description string
  
  An optional description of the job.
  
  job_id string Required
  
  latest_record_time_stamp number
  
  The timestamp of the latest processed record.
  
  latest_result_time_stamp number
  
  The timestamp of the latest bucket result.
  
  min_version string Required
  
  model_size_stats object
  
  Hide model_size_stats attributes Show model_size_stats attributes object
  
  bucket_allocation_failures_count number Required
  
  job_id string Required
  
  log_time string | number Required
  
  A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
  
  One of:
  DateTime string UnitMillis number
  
  memory_status string Required
  
  Values are ok, soft_limit, or hard_limit.
  
  model_bytes number | string Required
  
  One of:
  ByteSize number ByteSize string
  
  model_bytes_exceeded number | string
  
  One of:
  ByteSize number ByteSize string
  
  model_bytes_memory_limit number | string
  
  One of:
  ByteSize number ByteSize string
  
  output_memory_allocator_bytes number | string
  
  One of:
  ByteSize number ByteSize string
  
  peak_model_bytes number | string
  
  One of:
  ByteSize number ByteSize string
  
  assignment_memory_basis string
  
  result_type string Required
  
  total_by_field_count number Required
  
  total_over_field_count number Required
  
  total_partition_field_count number Required
  
  categorization_status string Required
  
  Values are ok or warn.
  
  categorized_doc_count number Required
  
  dead_category_count number Required
  
  failed_category_count number Required
  
  frequent_category_count number Required
  
  rare_category_count number Required
  
  total_category_count number Required
  
  timestamp number
  
  retain boolean Required
  
  If true, this snapshot will not be deleted during automatic cleanup of snapshots older than model_snapshot_retention_days. However, this snapshot will be deleted when the job is deleted. The default value is false.
  
  snapshot_doc_count number Required
  
  For internal use only.
  
  snapshot_id string Required
  
  timestamp number Required
  
  The creation timestamp for the snapshot.

POST /_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}

curl \
 --request POST 'http://api.example.com/_ml/anomaly_detectors/{job_id}/model_snapshots/{snapshot_id}' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"desc":true,"":"string","page":{"from":42.0,"size":42.0},"sort":"string"}'

Request examples

{
  "desc": true,
  "": "string",
  "page": {
    "from": 42.0,
    "size": 42.0
  },
  "sort": "string"
}

Response examples (200)

{
  "count": 42.0,
  "model_snapshots": [
    {
      "description": "string",
      "job_id": "string",
      "latest_record_time_stamp": 42.0,
      "latest_result_time_stamp": 42.0,
      "min_version": "string",
      "model_size_stats": {
        "bucket_allocation_failures_count": 42.0,
        "job_id": "string",
        "": 42.0,
        "memory_status": "ok",
        "assignment_memory_basis": "string",
        "result_type": "string",
        "total_by_field_count": 42.0,
        "total_over_field_count": 42.0,
        "total_partition_field_count": 42.0,
        "categorization_status": "ok",
        "categorized_doc_count": 42.0,
        "dead_category_count": 42.0,
        "failed_category_count": 42.0,
        "frequent_category_count": 42.0,
        "rare_category_count": 42.0,
        "total_category_count": 42.0,
        "timestamp": 42.0
      },
      "retain": true,
      "snapshot_doc_count": 42.0,
      "snapshot_id": "string",
      "timestamp": 42.0
    }
  ]
}

Get model snapshots info Added in 5.4.0

GET /_ml/anomaly_detectors/{job_id}/model_snapshots

Api key auth Basic auth Bearer auth

Path parameters

job_id string Required

Identifier for the anomaly detection job.

Query parameters

desc boolean

If true, the results are sorted in descending order.
end string | number

Returns snapshots with timestamps earlier than this time.
from number

Skips the specified number of snapshots.
size number

Specifies the maximum number of snapshots to obtain.
sort string

Specifies the sort field for the requested snapshots. By default, the snapshots are sorted by their timestamp.
start string | number

Returns snapshots with timestamps after this time.

application/json

Body

desc boolean

Refer to the description for the desc query parameter.
end string | number

A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

One of:
DateTime string UnitMillis number
page object
Hide page attributes Show page attributes object
- from number
  
  Skips the specified number of items.
- size number
  
  Specifies the maximum number of items to obtain.
sort string

Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
start string | number

A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.

One of:
DateTime string UnitMillis number

Responses

200 application/json
Hide response attributes Show response attributes object
- count number Required
- model_snapshots array[object] Required
  
  Hide model_snapshots attributes Show model_snapshots attributes object
  
  description string
  
  An optional description of the job.
  
  job_id string Required
  
  latest_record_time_stamp number
  
  The timestamp of the latest processed record.
  
  latest_result_time_stamp number
  
  The timestamp of the latest bucket result.
  
  min_version string Required
  
  model_size_stats object
  
  Hide model_size_stats attributes Show model_size_stats attributes object
  
  bucket_allocation_failures_count number Required
  
  job_id string Required
  
  log_time string | number Required
  
  A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
  
  One of:
  DateTime string UnitMillis number
  
  memory_status string Required
  
  Values are ok, soft_limit, or hard_limit.
  
  model_bytes number | string Required
  
  One of:
  ByteSize number ByteSize string
  
  model_bytes_exceeded number | string
  
  One of:
  ByteSize number ByteSize string
  
  model_bytes_memory_limit number | string
  
  One of:
  ByteSize number ByteSize string
  
  output_memory_allocator_bytes number | string
  
  One of:
  ByteSize number ByteSize string
  
  peak_model_bytes number | string
  
  One of:
  ByteSize number ByteSize string
  
  assignment_memory_basis string
  
  result_type string Required
  
  total_by_field_count number Required
  
  total_over_field_count number Required
  
  total_partition_field_count number Required
  
  categorization_status string Required
  
  Values are ok or warn.
  
  categorized_doc_count number Required
  
  dead_category_count number Required
  
  failed_category_count number Required
  
  frequent_category_count number Required
  
  rare_category_count number Required
  
  total_category_count number Required
  
  timestamp number
  
  retain boolean Required
  
  If true, this snapshot will not be deleted during automatic cleanup of snapshots older than model_snapshot_retention_days. However, this snapshot will be deleted when the job is deleted. The default value is false.
  
  snapshot_doc_count number Required
  
  For internal use only.
  
  snapshot_id string Required
  
  timestamp number Required
  
  The creation timestamp for the snapshot.

GET /_ml/anomaly_detectors/{job_id}/model_snapshots

curl \
 --request GET 'http://api.example.com/_ml/anomaly_detectors/{job_id}/model_snapshots' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"desc":true,"":"string","page":{"from":42.0,"size":42.0},"sort":"string"}'

Request examples

{
  "desc": true,
  "": "string",
  "page": {
    "from": 42.0,
    "size": 42.0
  },
  "sort": "string"
}

Response examples (200)

{
  "count": 42.0,
  "model_snapshots": [
    {
      "description": "string",
      "job_id": "string",
      "latest_record_time_stamp": 42.0,
      "latest_result_time_stamp": 42.0,
      "min_version": "string",
      "model_size_stats": {
        "bucket_allocation_failures_count": 42.0,
        "job_id": "string",
        "": 42.0,
        "memory_status": "ok",
        "assignment_memory_basis": "string",
        "result_type": "string",
        "total_by_field_count": 42.0,
        "total_over_field_count": 42.0,
        "total_partition_field_count": 42.0,
        "categorization_status": "ok",
        "categorized_doc_count": 42.0,
        "dead_category_count": 42.0,
        "failed_category_count": 42.0,
        "frequent_category_count": 42.0,
        "rare_category_count": 42.0,
        "total_category_count": 42.0,
        "timestamp": 42.0
      },
      "retain": true,
      "snapshot_doc_count": 42.0,
      "snapshot_id": "string",
      "timestamp": 42.0
    }
  ]
}

Delete a data frame analytics job Added in 7.3.0

DELETE /_ml/data_frame/analytics/{id}

Api key auth Basic auth Bearer auth

Path parameters

id string Required

Identifier for the data frame analytics job.

Query parameters

force boolean

If true, it deletes a job that is not stopped; this method is quicker than stopping and deleting the job.
timeout string

The time to wait for the job to be deleted.

Responses

200 application/json
Hide response attribute Show response attribute object
- acknowledged boolean Required
  
  For a successful response, this value is always true. On failure, an exception is returned instead.

DELETE /_ml/data_frame/analytics/{id}

curl \
 --request DELETE 'http://api.example.com/_ml/data_frame/analytics/{id}' \
 --header "Authorization: $API_KEY"

Response examples (200)

A successful response when deleting a data frame analytics job.

{
  "acknowledged": true
}

Get rollup job information Deprecated Technical preview

GET /_rollup/job

Api key auth Basic auth Bearer auth

Get the configuration, stats, and status of rollup jobs.

NOTE: This API returns only active (both STARTED and STOPPED) jobs. If a job was created, ran for a while, then was deleted, the API does not return any details about it. For details about a historical rollup job, the rollup capabilities API may be more useful.

Responses

200 application/json
Hide response attribute Show response attribute object
- jobs array[object] Required
  
  Hide jobs attributes Show jobs attributes object
  
  config object Required
  
  Hide config attributes Show config attributes object
  
  cron string Required
  
  groups object Required
  
  Hide groups attributes Show groups attributes object
  
  date_histogram object
  
  Hide date_histogram attributes Show date_histogram attributes object
  
  delay string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  format string
  
  interval string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  calendar_interval string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  fixed_interval string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  time_zone string
  
  histogram object
  
  Hide histogram attributes Show histogram attributes object
  
  fields string | array[string] Required
  
  interval number Required
  
  The interval of histogram buckets to be generated when rolling up. For example, a value of 5 creates buckets that are five units wide (0-5, 5-10, etc). Note that only one interval can be specified in the histogram group, meaning that all fields being grouped via the histogram must share the same interval.
  
  terms object
  
  Hide terms attribute Show terms attribute object
  
  fields string | array[string] Required
  
  id string Required
  
  index_pattern string Required
  
  metrics array[object] Required
  
  Hide metrics attributes Show metrics attributes object
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  metrics array[string] Required
  
  An array of metrics to collect for the field. At least one metric must be configured.
  
  Values are min, max, sum, avg, or value_count.
  
  page_size number Required
  
  rollup_index string Required
  
  timeout string Required
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  stats object Required
  
  Hide stats attributes Show stats attributes object
  
  documents_processed number Required
  
  index_failures number Required
  
  index_time_in_ms number
  
  Time unit for milliseconds
  
  index_total number Required
  
  pages_processed number Required
  
  rollups_indexed number Required
  
  search_failures number Required
  
  search_time_in_ms number
  
  Time unit for milliseconds
  
  search_total number Required
  
  trigger_count number Required
  
  processing_time_in_ms number
  
  Time unit for milliseconds
  
  processing_total number Required
  
  status object Required
  
  Hide status attributes Show status attributes object
  
  current_position object
  
  Hide current_position attribute Show current_position attribute object
  
  * object Additional properties
  
  job_state string Required
  
  Values are started, indexing, stopping, stopped, or aborting.
  
  upgraded_doc_id boolean

GET /_rollup/job

curl \
 --request GET 'http://api.example.com/_rollup/job' \
 --header "Authorization: $API_KEY"

Response examples (200)

A successful response from `GET _rollup/job/sensor`.

{
  "jobs": [
    {
      "config": {
        "id": "sensor",
        "index_pattern": "sensor-*",
        "rollup_index": "sensor_rollup",
        "cron": "*/30 * * * * ?",
        "groups": {
          "date_histogram": {
            "fixed_interval": "1h",
            "delay": "7d",
            "field": "timestamp",
            "time_zone": "UTC"
          },
          "terms": {
            "fields": [
              "node"
            ]
          }
        },
        "metrics": [
          {
            "field": "temperature",
            "metrics": [
              "min",
              "max",
              "sum"
            ]
          },
          {
            "field": "voltage",
            "metrics": [
              "avg"
            ]
          }
        ],
        "timeout": "20s",
        "page_size": 1000
      },
      "status": {
        "job_state": "stopped"
      },
      "stats": {
        "pages_processed": 0,
        "documents_processed": 0,
        "rollups_indexed": 0,
        "trigger_count": 0,
        "index_failures": 0,
        "index_time_in_ms": 0,
        "index_total": 0,
        "search_failures": 0,
        "search_time_in_ms": 0,
        "search_total": 0,
        "processing_time_in_ms": 0,
        "processing_total": 0
      }
    }
  ]
}

Get the search shards

POST /{index}/_search_shards

Api key auth Basic auth Bearer auth

Get the indices and shards that a search request would be run against. This information can be useful for working out issues or planning optimizations with routing and shard preferences. When filtered aliases are used, the filter is returned as part of the indices section.

If the Elasticsearch security features are enabled, you must have the view_index_metadata or manage index privilege for the target data stream, index, or alias.

Path parameters

index string | array[string] Required

A comma-separated list of data streams, indices, and aliases to search. It supports wildcards (*). To search all data streams and indices, omit this parameter or use * or _all.

Query parameters

allow_no_indices boolean

If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
expand_wildcards string | array[string]

Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.
ignore_unavailable boolean

If false, the request returns an error if it targets a missing or closed index.
local boolean

If true, the request retrieves information from the local node only.
master_timeout string

The period to wait for a connection to the master node. If the master node is not available before the timeout expires, the request fails and returns an error. IT can also be set to -1 to indicate that the request should never timeout.
preference string

The node or shard the operation should be performed on. It is random by default.
routing string

A custom value used to route operations to a specific shard.

Responses

200 application/json
Hide response attributes Show response attributes object
- nodes object Required
  
  Hide nodes attribute Show nodes attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  name string Required
  
  ephemeral_id string Required
  
  transport_address string Required
  
  external_id string Required
  
  attributes object Required
  
  Lists node attributes.
  
  Hide attributes attribute Show attributes attribute object
  
  * string Additional properties
  
  roles array[string] Required
  
  @doc_id node-roles
  
  Values are master, data, data_cold, data_content, data_frozen, data_hot, data_warm, client, ingest, ml, voting_only, transform, remote_cluster_client, or coordinating_only.
  
  version string Required
  
  min_index_version number Required
  
  max_index_version number Required
- shards array[array] Required
  
  Hide shards attributes Show shards attributes object
  
  state string Required
  
  Values are UNASSIGNED, INITIALIZING, STARTED, or RELOCATING.
  
  primary boolean Required
  
  node string
  
  shard number Required
  
  index string Required
  
  allocation_id object
  
  Hide allocation_id attribute Show allocation_id attribute object
  
  * string
  
  recovery_source object
  
  Hide recovery_source attribute Show recovery_source attribute object
  
  * string
  
  unassigned_info object
  
  Hide unassigned_info attributes Show unassigned_info attributes object
  
  at string
  
  last_allocation_status string
  
  reason string Required
  
  Values are INDEX_CREATED, CLUSTER_RECOVERED, INDEX_REOPENED, DANGLING_INDEX_IMPORTED, NEW_INDEX_RESTORED, EXISTING_INDEX_RESTORED, REPLICA_ADDED, ALLOCATION_FAILED, NODE_LEFT, REROUTE_CANCELLED, REINITIALIZED, REALLOCATED_REPLICA, PRIMARY_FAILED, FORCED_EMPTY_PRIMARY, or MANUAL_ALLOCATION.
  
  details string
  
  failed_allocation_attempts number
  
  delayed boolean
  
  allocation_status string
  
  relocating_node string | null
  
  One of:
  NodeId string string-2 string | null
  
  relocation_failure_info object
  
  Hide relocation_failure_info attribute Show relocation_failure_info attribute object
  
  failed_attempts number Required
- indices object Required
  
  Hide indices attribute Show indices attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  aliases array[string]
  
  filter object
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  External documentation

POST /{index}/_search_shards

curl \
 --request POST 'http://api.example.com/{index}/_search_shards' \
 --header "Authorization: $API_KEY"

Response examples (200)

An abbreviated response from `GET /my-index-000001/_search_shards`.

{
  "nodes": {},
  "indices": {
      "my-index-000001": { }
  },
  "shards": [
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 0,
          "state": "STARTED",
          "allocation_id": {"id":"0TvkCyF7TAmM1wHP4a42-A"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 1,
          "state": "STARTED",
          "allocation_id": {"id":"fMju3hd1QHWmWrIgFnI4Ww"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 2,
          "state": "STARTED",
          "allocation_id": {"id":"Nwl0wbMBTHCWjEEbGYGapg"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 3,
          "state": "STARTED",
          "allocation_id": {"id":"bU_KLGJISbW0RejwnwDPKw"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ],
      [
      {
          "index": "my-index-000001",
          "node": "JklnKbD7Tyqi9TP3_Q_tBg",
          "relocating_node": null,
          "primary": true,
          "shard": 4,
          "state": "STARTED",
          "allocation_id": {"id":"DMs7_giNSwmdqVukF7UydA"},
          "relocation_failure_info" : {
          "failed_attempts" : 0
          }
      }
      ]
    ]
  }

Run a search application search Beta

POST /_application/search_application/{name}/_search

Api key auth Basic auth Bearer auth

Generate and run an Elasticsearch query that uses the specified query parameteter and the search template associated with the search application or default template. Unspecified template parameters are assigned their default values if applicable.

Path parameters

name string Required

The name of the search application to be searched.

Query parameters

typed_keys boolean

Determines whether aggregation names are prefixed by their respective types in the response.

application/json

Body

params object

Query parameters specific to this request, which will override any defaults specified in the template.
Hide params attribute Show params attribute object
- * object Additional properties

Responses

200 application/json
Hide response attributes Show response attributes object
- took number Required
  
  The number of milliseconds it took Elasticsearch to run the request. This value is calculated by measuring the time elapsed between receipt of a request on the coordinating node and the time at which the coordinating node is ready to send the response. It includes:
  
  Communication time between the coordinating node and data nodes
  
  Time the request spends in the search thread pool, queued for execution
  
  Actual run time
  
  It does not include:
  
  Time needed to send the request to Elasticsearch
  
  Time needed to serialize the JSON response
  
  Time needed to send the response to a client
- timed_out boolean Required
  
  If true, the request timed out before completion; returned results may be partial or empty.
- _shards object Required
  
  Hide _shards attributes Show _shards attributes object
  
  failed number Required
  
  successful number Required
  
  total number Required
  
  failures array[object]
  
  Hide failures attributes Show failures attributes object
  
  index string
  
  node string
  
  reason object Required
  
  Hide reason attributes Show reason attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  shard number Required
  
  status string
  
  skipped number
- hits object Required
  
  Hide hits attributes Show hits attributes object
  
  total object | number
  
  Total hit count information, present only if track_total_hits wasn't false in the search request.
  
  One of:
  TotalHits object number-2 number
  
  Hide attributes Show attributes
  
  relation string Required
  
  Values are eq or gte.
  
  value number Required
  
  hits array[object] Required
  
  Hide hits attributes Show hits attributes object
  
  _index string Required
  
  _id string
  
  _score number | string | null
  
  One of:
  number-1 number string-2 string | null
  
  _explanation object
  
  Hide _explanation attributes Show _explanation attributes object
  
  description string Required
  
  details array[object] Required
  
  value number Required
  
  fields object
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  highlight object
  
  Hide highlight attribute Show highlight attribute object
  
  * array[string] Additional properties
  
  inner_hits object
  
  Hide inner_hits attribute Show inner_hits attribute object
  
  * object Additional properties
  
  Hide * attribute Show * attribute object
  
  hits object Required
  
  matched_queries array[string] | object
  
  One of:
  array-1 array[string] object-2 object
  
  _nested object
  
  Hide _nested attributes Show _nested attributes object
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  offset number Required
  
  _nested object
  
  _ignored array[string]
  
  ignored_field_values object
  
  Hide ignored_field_values attribute Show ignored_field_values attribute object
  
  * array[number | string | boolean | null | object] Additional properties
  
  A field value.
  
  _shard string
  
  _node string
  
  _routing string
  
  _source object
  
  _rank number
  
  _seq_no number
  
  _primary_term number
  
  _version number
  
  sort array[number | string | boolean | null | object]
  
  A field value.
  
  A field value.
  
  One of:
  FieldValue number FieldValue number FieldValue string FieldValue boolean FieldValue string | null FieldValue object
  
  max_score number | string | null
  
  One of:
  number-1 number string-2 string | null
- aggregations object
- _clusters object
  
  Hide _clusters attributes Show _clusters attributes object
  
  skipped number Required
  
  successful number Required
  
  total number Required
  
  running number Required
  
  partial number Required
  
  failed number Required
  
  details object
  
  Hide details attribute Show details attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  status string Required
  
  Values are running, successful, partial, skipped, or failed.
  
  indices string Required
  
  took number
  
  Time unit for milliseconds
  
  timed_out boolean Required
  
  _shards object
  
  Hide _shards attributes Show _shards attributes object
  
  failed number Required
  
  successful number Required
  
  total number Required
  
  failures array[object]
  
  skipped number
  
  failures array[object]
  
  Hide failures attributes Show failures attributes object
  
  index string
  
  node string
  
  reason object Required
  
  shard number Required
  
  status string
- fields object
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
- max_score number
- num_reduce_phases number
- profile object
  
  Hide profile attribute Show profile attribute object
  
  shards array[object] Required
  
  Hide shards attributes Show shards attributes object
  
  aggregations array[object] Required
  
  Hide aggregations attributes Show aggregations attributes object
  
  breakdown object Required
  
  description string Required
  
  time_in_nanos
  
  type string Required
  
  debug object
  
  children array[object]
  
  cluster string Required
  
  dfs object
  
  Hide dfs attributes Show dfs attributes object
  
  statistics object
  
  Hide statistics attributes Show statistics attributes object
  
  type string Required
  
  description string Required
  
  time string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  time_in_nanos
  
  breakdown object Required
  
  debug object
  
  children array[object]
  
  knn array[object]
  
  fetch object
  
  Hide fetch attributes Show fetch attributes object
  
  type string Required
  
  description string Required
  
  time_in_nanos number
  
  Time unit for nanoseconds
  
  breakdown object Required
  
  Hide breakdown attributes Show breakdown attributes object
  
  load_source number
  
  load_source_count number
  
  load_stored_fields number
  
  load_stored_fields_count number
  
  next_reader number
  
  next_reader_count number
  
  process_count number
  
  process number
  
  debug object
  
  Hide debug attributes Show debug attributes object
  
  stored_fields array[string]
  
  fast_path number
  
  children array[object]
  
  id string Required
  
  index string Required
  
  node_id string Required
  
  searches array[object] Required
  
  Hide searches attributes Show searches attributes object
  
  collector array[object] Required
  
  query array[object] Required
  
  rewrite_time number Required
  
  shard_id number Required
- pit_id string
- _scroll_id string
- suggest object
  
  Hide suggest attribute Show suggest attribute object
  
  * array[object] Additional properties
  
  One of:
  CompletionSuggest object PhraseSuggest object TermSuggest object
  
  Hide attributes Show attributes
  
  length number Required
  
  offset number Required
  
  text string Required
  
  options
- terminated_early boolean

POST /_application/search_application/{name}/_search

curl \
 --request POST 'http://api.example.com/_application/search_application/{name}/_search' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"params\": {\n    \"query_string\": \"my first query\",\n    \"text_fields\": [\n      {\"name\": \"title\", \"boost\": 5},\n      {\"name\": \"description\", \"boost\": 1}\n    ]\n  }\n}"'

Request example

Use `POST _application/search_application/my-app/_search` to run a search against a search application called `my-app` that uses a search template.

{
  "params": {
    "query_string": "my first query",
    "text_fields": [
      {"name": "title", "boost": 5},
      {"name": "description", "boost": 1}
    ]
  }
}

Response examples (200)

{
  "took": 42.0,
  "timed_out": true,
  "_shards": {
    "failed": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "failures": [
      {
        "index": "string",
        "node": "string",
        "reason": {
          "type": "string",
          "reason": "string",
          "stack_trace": "string",
          "caused_by": {},
          "root_cause": [
            {}
          ],
          "suppressed": [
            {}
          ]
        },
        "shard": 42.0,
        "status": "string"
      }
    ],
    "skipped": 42.0
  },
  "hits": {
    "total": {
      "relation": "eq",
      "value": 42.0
    },
    "hits": [
      {
        "_index": "string",
        "_id": "string",
        "_score": 42.0,
        "_explanation": {
          "description": "string",
          "details": [
            {}
          ],
          "value": 42.0
        },
        "fields": {
          "additionalProperty1": {},
          "additionalProperty2": {}
        },
        "highlight": {
          "additionalProperty1": [
            "string"
          ],
          "additionalProperty2": [
            "string"
          ]
        },
        "inner_hits": {
          "additionalProperty1": {
            "hits": {}
          },
          "additionalProperty2": {
            "hits": {}
          }
        },
        "matched_queries": [
          "string"
        ],
        "_nested": {
          "field": "string",
          "offset": 42.0,
          "_nested": {}
        },
        "_ignored": [
          "string"
        ],
        "ignored_field_values": {
          "additionalProperty1": [
            {}
          ],
          "additionalProperty2": [
            {}
          ]
        },
        "_shard": "string",
        "_node": "string",
        "_routing": "string",
        "_source": {},
        "_rank": 42.0,
        "_seq_no": 42.0,
        "_primary_term": 42.0,
        "_version": 42.0,
        "sort": [
          42.0
        ]
      }
    ],
    "max_score": 42.0
  },
  "aggregations": {},
  "_clusters": {
    "skipped": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "running": 42.0,
    "partial": 42.0,
    "failed": 42.0,
    "details": {
      "additionalProperty1": {
        "status": "running",
        "indices": "string",
        "": 42.0,
        "timed_out": true,
        "_shards": {
          "failed": 42.0,
          "successful": 42.0,
          "total": 42.0,
          "failures": [
            {}
          ],
          "skipped": 42.0
        },
        "failures": [
          {
            "index": "string",
            "node": "string",
            "reason": {},
            "shard": 42.0,
            "status": "string"
          }
        ]
      },
      "additionalProperty2": {
        "status": "running",
        "indices": "string",
        "": 42.0,
        "timed_out": true,
        "_shards": {
          "failed": 42.0,
          "successful": 42.0,
          "total": 42.0,
          "failures": [
            {}
          ],
          "skipped": 42.0
        },
        "failures": [
          {
            "index": "string",
            "node": "string",
            "reason": {},
            "shard": 42.0,
            "status": "string"
          }
        ]
      }
    }
  },
  "fields": {
    "additionalProperty1": {},
    "additionalProperty2": {}
  },
  "max_score": 42.0,
  "num_reduce_phases": 42.0,
  "profile": {
    "shards": [
      {
        "aggregations": [
          {
            "breakdown": {},
            "description": "string",
            "type": "string",
            "debug": {},
            "children": [
              {}
            ]
          }
        ],
        "cluster": "string",
        "dfs": {
          "statistics": {
            "type": "string",
            "description": "string",
            "time": "string",
            "breakdown": {},
            "debug": {},
            "children": [
              {}
            ]
          },
          "knn": [
            {}
          ]
        },
        "fetch": {
          "type": "string",
          "description": "string",
          "": 42.0,
          "breakdown": {
            "load_source": 42.0,
            "load_source_count": 42.0,
            "load_stored_fields": 42.0,
            "load_stored_fields_count": 42.0,
            "next_reader": 42.0,
            "next_reader_count": 42.0,
            "process_count": 42.0,
            "process": 42.0
          },
          "debug": {
            "stored_fields": [
              "string"
            ],
            "fast_path": 42.0
          },
          "children": [
            {}
          ]
        },
        "id": "string",
        "index": "string",
        "node_id": "string",
        "searches": [
          {
            "collector": [
              {}
            ],
            "query": [
              {}
            ],
            "rewrite_time": 42.0
          }
        ],
        "shard_id": 42.0
      }
    ]
  },
  "pit_id": "string",
  "_scroll_id": "string",
  "suggest": {
    "additionalProperty1": [
      {
        "length": 42.0,
        "offset": 42.0,
        "text": "string"
      }
    ],
    "additionalProperty2": [
      {
        "length": 42.0,
        "offset": 42.0,
        "text": "string"
      }
    ]
  },
  "terminated_early": true
}

Get service accounts Added in 7.13.0

GET /_security/service

Api key auth Basic auth Bearer auth

Get a list of service accounts that match the provided path parameters.

NOTE: Currently, only the elastic/fleet-server service account is available.

External documentation

Responses

200 application/json
Hide response attribute Show response attribute object
- * object Additional properties
  
  Hide * attribute Show * attribute object
  
  role_descriptor object Required
  
  Hide role_descriptor attributes Show role_descriptor attributes object
  
  cluster array[string] Required
  
  A list of cluster privileges. These privileges define the cluster level actions that API keys are able to execute.
  
  indices array[object] Required
  
  A list of indices permissions entries.
  
  Hide indices attributes Show indices attributes object
  
  field_security object
  
  Hide field_security attributes Show field_security attributes object
  
  except string | array[string]
  
  grant string | array[string]
  
  names array[string] Required
  
  A list of indices (or index name patterns) to which the permissions in this entry apply.
  
  privileges array[string] Required
  
  The index level privileges that owners of the role have on the specified indices.
  
  query string | object
  
  While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.
  
  Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.
  
  One of:
  IndicesPrivilegesQuery string QueryContainer object RoleTemplateQuery object
  
  allow_restricted_indices boolean
  
  Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.
  
  remote_indices array[object]
  
  A list of indices permissions for remote clusters.
  
  Hide remote_indices attributes Show remote_indices attributes object
  
  clusters string | array[string] Required
  
  field_security object
  
  Hide field_security attributes Show field_security attributes object
  
  except string | array[string]
  
  grant string | array[string]
  
  names array[string] Required
  
  A list of indices (or index name patterns) to which the permissions in this entry apply.
  
  privileges array[string] Required
  
  The index level privileges that owners of the role have on the specified indices.
  
  query string | object
  
  While creating or updating a role you can provide either a JSON structure or a string to the API. However, the response provided by Elasticsearch will only be string with a json-as-text content.
  
  Since this is embedded in IndicesPrivileges, the same structure is used for clarity in both contexts.
  
  One of:
  IndicesPrivilegesQuery string QueryContainer object RoleTemplateQuery object
  
  allow_restricted_indices boolean
  
  Set to true if using wildcard or regular expressions for patterns that cover restricted indices. Implicitly, restricted indices have limited privileges that can cause pattern tests to fail. If restricted indices are explicitly included in the names list, Elasticsearch checks privileges against these indices regardless of the value set for allow_restricted_indices.
  
  remote_cluster array[object]
  
  A list of cluster permissions for remote clusters. NOTE: This is limited a subset of the cluster permissions.
  
  Hide remote_cluster attributes Show remote_cluster attributes object
  
  clusters string | array[string] Required
  
  privileges array[string] Required
  
  The cluster level privileges that owners of the role have on the remote cluster.
  
  Values are monitor_enrich or monitor_stats.
  
  global array[object] | object
  
  An object defining global privileges. A global privilege is a form of cluster privilege that is request-aware. Support for global privileges is currently limited to the management of application privileges.
  
  One of:
  array-1 array[object] GlobalPrivilege object
  
  Hide attribute Show attribute object
  
  application object Required
  
  applications array[object]
  
  A list of application privilege entries
  
  Hide applications attributes Show applications attributes object
  
  application string Required
  
  The name of the application to which this entry applies.
  
  privileges array[string] Required
  
  A list of strings, where each element is the name of an application privilege or action.
  
  resources array[string] Required
  
  A list resources to which the privileges are applied.
  
  metadata object
  
  Hide metadata attribute Show metadata attribute object
  
  * object Additional properties
  
  run_as array[string]
  
  A list of users that the API keys can impersonate.
  
  description string
  
  An optional description of the role descriptor.
  
  restriction object
  
  Hide restriction attribute Show restriction attribute object
  
  workflows array[string] Required
  
  A list of workflows to which the API key is restricted. NOTE: In order to use a role restriction, an API key must be created with a single role descriptor.
  
  transient_metadata object
  
  Hide transient_metadata attribute Show transient_metadata attribute object
  
  * object Additional properties

GET /_security/service

curl \
 --request GET 'http://api.example.com/_security/service' \
 --header "Authorization: $API_KEY"

Response examples (200)

A successful response from `GET /_security/service/elastic/fleet-server`. The response contains information about the `elastic/fleet-server` service account.

{
  "elastic/fleet-server": {
    "role_descriptor": {
      "cluster": [
        "monitor",
        "manage_own_api_key",
        "read_fleet_secrets"
      ],
      "indices": [
        {
          "names": [
            "logs-*",
            "metrics-*",
            "traces-*",
            ".logs-endpoint.diagnostic.collection-*",
            ".logs-endpoint.action.responses-*",
            ".logs-endpoint.heartbeat-*"
          ],
          "privileges": [
            "write",
            "create_index",
            "auto_configure"
          ],
          "allow_restricted_indices": false
        },
        {
          "names": [
            "profiling-*"
          ],
          "privileges": [
            "read",
            "write"
          ],
          "allow_restricted_indices": false
        },
        {
          "names": [
            "traces-apm.sampled-*"
          ],
          "privileges": [
            "read",
            "monitor",
            "maintenance"
          ],
          "allow_restricted_indices": false
        },
        {
          "names": [
            ".fleet-secrets*"
          ],
          "privileges": [
            "read"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-actions*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-agents*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-artifacts*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-enrollment-api-keys*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-policies*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-policies-leader*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-servers*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            ".fleet-fileds*"
          ],
          "privileges": [
            "read",
            "write",
            "monitor",
            "create_index",
            "auto_configure",
            "maintenance"
          ],
          "allow_restricted_indices": true
        },
        {
          "names": [
            "synthetics-*"
          ],
          "privileges": [
            "read",
            "write",
            "create_index",
            "auto_configure"
          ],
          "allow_restricted_indices": false
        }
      ],
      "applications": [
        {
          "application": "kibana-*",
          "privileges": [
            "reserved_fleet-setup"
          ],
          "resources": [
            "*"
          ]
        }
      ],
      "run_as": [],
      "metadata": {},
      "transient_metadata": {
        "enabled": true
      }
    }
  }
}

Invalidate SAML Added in 7.5.0

POST /_security/saml/invalidate

Api key auth Basic auth Bearer auth

Submit a SAML LogoutRequest message to Elasticsearch for consumption.

NOTE: This API is intended for use by custom web applications other than Kibana. If you are using Kibana, refer to the documentation for configuring SAML single-sign-on on the Elastic Stack.

The logout request comes from the SAML IdP during an IdP initiated Single Logout. The custom web application can use this API to have Elasticsearch process the LogoutRequest. After successful validation of the request, Elasticsearch invalidates the access token and refresh token that corresponds to that specific SAML principal and provides a URL that contains a SAML LogoutResponse message. Thus the user can be redirected back to their IdP.

External documentation

application/json

Body Required

acs string

The Assertion Consumer Service URL that matches the one of the SAML realm in Elasticsearch that should be used. You must specify either this parameter or the realm parameter.
query_string string Required

The query part of the URL that the user was redirected to by the SAML IdP to initiate the Single Logout. This query should include a single parameter named SAMLRequest that contains a SAML logout request that is deflated and Base64 encoded. If the SAML IdP has signed the logout request, the URL should include two extra parameters named SigAlg and Signature that contain the algorithm used for the signature and the signature value itself. In order for Elasticsearch to be able to verify the IdP's signature, the value of the query_string field must be an exact match to the string provided by the browser. The client application must not attempt to parse or process the string in any way.
realm string

The name of the SAML realm in Elasticsearch the configuration. You must specify either this parameter or the acs parameter.

Responses

200 application/json
Hide response attributes Show response attributes object
- invalidated number Required
  
  The number of tokens that were invalidated as part of this logout.
- realm string Required
  
  The realm name of the SAML realm in Elasticsearch that authenticated the user.
- redirect string Required
  
  A SAML logout response as a parameter so that the user can be redirected back to the SAML IdP.

POST /_security/saml/invalidate

curl \
 --request POST 'http://api.example.com/_security/saml/invalidate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"query_string\" : \"SAMLRequest=nZFda4MwFIb%2FiuS%2BmviRpqFaClKQdbvo2g12M2KMraCJ9cRR9utnW4Wyi13sMie873MeznJ1aWrnS3VQGR0j4mLkKC1NUeljjA77zYyhVbIE0dR%2By7fmaHq7U%2BdegXWGpAZ%2B%2F4pR32luBFTAtWgUcCv56%2Fp5y30X87Yz1khTIycdgpUW9kY7WdsC9zxoXTvMvWuVV98YyMnSGH2SYE5pwALBIr9QKiwDGpW0oGVUznGeMyJZKFkQ4jBf5HnhUymjIhzCAL3KNFihbYx8TBYzzGaY7EnIyZwHzCWMfiDnbRIftkSjJr%2BFu0e9v%2B0EgOquRiiZjKpiVFp6j50T4WXoyNJ%2FEWC9fdqc1t%2F1%2B2F3aUpjzhPiXpqMz1%2FHSn4A\u0026SigAlg=http%3A%2F%2Fwww.w3.org%2F2001%2F04%2Fxmldsig-more%23rsa-sha256\u0026Signature=MsAYz2NFdovMG2mXf6TSpu5vlQQyEJAg%2B4KCwBqJTmrb3yGXKUtIgvjqf88eCAK32v3eN8vupjPC8LglYmke1ZnjK0%2FKxzkvSjTVA7mMQe2AQdKbkyC038zzRq%2FYHcjFDE%2Bz0qISwSHZY2NyLePmwU7SexEXnIz37jKC6NMEhus%3D\",\n  \"realm\" : \"saml1\"\n}"'

Request example

Run `POST /_security/saml/invalidate` to invalidate all the tokens for realm `saml1` pertaining to the user that is identified in the SAML Logout Request.

{
  "query_string" : "SAMLRequest=nZFda4MwFIb%2FiuS%2BmviRpqFaClKQdbvo2g12M2KMraCJ9cRR9utnW4Wyi13sMie873MeznJ1aWrnS3VQGR0j4mLkKC1NUeljjA77zYyhVbIE0dR%2By7fmaHq7U%2BdegXWGpAZ%2B%2F4pR32luBFTAtWgUcCv56%2Fp5y30X87Yz1khTIycdgpUW9kY7WdsC9zxoXTvMvWuVV98YyMnSGH2SYE5pwALBIr9QKiwDGpW0oGVUznGeMyJZKFkQ4jBf5HnhUymjIhzCAL3KNFihbYx8TBYzzGaY7EnIyZwHzCWMfiDnbRIftkSjJr%2BFu0e9v%2B0EgOquRiiZjKpiVFp6j50T4WXoyNJ%2FEWC9fdqc1t%2F1%2B2F3aUpjzhPiXpqMz1%2FHSn4A&SigAlg=http%3A%2F%2Fwww.w3.org%2F2001%2F04%2Fxmldsig-more%23rsa-sha256&Signature=MsAYz2NFdovMG2mXf6TSpu5vlQQyEJAg%2B4KCwBqJTmrb3yGXKUtIgvjqf88eCAK32v3eN8vupjPC8LglYmke1ZnjK0%2FKxzkvSjTVA7mMQe2AQdKbkyC038zzRq%2FYHcjFDE%2Bz0qISwSHZY2NyLePmwU7SexEXnIz37jKC6NMEhus%3D",
  "realm" : "saml1"
}

Response examples (200)

A successful response from `POST /_security/saml/invalidate`.

{
  "redirect" : "https://my-idp.org/logout/SAMLResponse=....",
  "invalidated" : 2,
  "realm" : "saml1"
}

Update user profile data Added in 8.2.0

POST /_security/profile/{uid}/_data

Api key auth Basic auth Bearer auth

Update specific data for the user profile that is associated with a unique ID.

NOTE: The user profile feature is designed only for use by Kibana and Elastic's Observability, Enterprise Search, and Elastic Security solutions. Individual users and external applications should not call this API directly. Elastic reserves the right to change or remove this feature in future releases without prior notice.

To use this API, you must have one of the following privileges:

The manage_user_profile cluster privilege.
The update_profile_data global privilege for the namespaces that are referenced in the request.

This API updates the labels and data fields of an existing user profile document with JSON objects. New keys and their values are added to the profile document and conflicting keys are replaced by data that's included in the request.

For both labels and data, content is namespaced by the top-level fields. The update_profile_data global privilege grants privileges for updating only the allowed namespaces.

Path parameters

uid string Required

A unique identifier for the user profile.

Query parameters

if_seq_no number

Only perform the operation if the document has this sequence number.
if_primary_term number

Only perform the operation if the document has this primary term.
refresh string

If 'true', Elasticsearch refreshes the affected shards to make this operation visible to search. If 'wait_for', it waits for a refresh to make this operation visible to search. If 'false', nothing is done with refreshes.

Values are true, false, or wait_for.

application/json

Body Required

labels object

Searchable data that you want to associate with the user profile. This field supports a nested data structure. Within the labels object, top-level keys cannot begin with an underscore (_) or contain a period (.).
Hide labels attribute Show labels attribute object
- * object Additional properties
data object

Non-searchable data that you want to associate with the user profile. This field supports a nested data structure. Within the data object, top-level keys cannot begin with an underscore (_) or contain a period (.). The data object is not searchable, but can be retrieved with the get user profile API.
Hide data attribute Show data attribute object
- * object Additional properties

Responses

200 application/json
Hide response attribute Show response attribute object
- acknowledged boolean Required
  
  For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_security/profile/{uid}/_data

curl \
 --request POST 'http://api.example.com/_security/profile/{uid}/_data' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"labels\": {\n    \"direction\": \"east\"\n  },\n  \"data\": {\n    \"app1\": {\n      \"theme\": \"default\"\n    }\n  }\n}"'

Request example

Run `POST /_security/profile/u_P_0BMHgaOK3p7k-PFWUCbw9dQ-UFjt01oWJ_Dp2PmPc_0/_data` to update a profile document for the `u_P_0BMHgaOK3p7k-PFWUCbw9dQ-UFjt01oWJ_Dp2PmPc_0` user profile.

{
  "labels": {
    "direction": "east"
  },
  "data": {
    "app1": {
      "theme": "default"
    }
  }
}

Response examples (200)

A successful response from `POST /_security/profile/u_P_0BMHgaOK3p7k-PFWUCbw9dQ-UFjt01oWJ_Dp2PmPc_0/_data`, which indicates that the request is acknowledged.

{
  "acknowledged": true
}

Update the connector features Technical preview

Body Required

Create a new document in the index Added in 5.0.0

Body Required

Get an enrich policy Added in 7.5.0

Stop async ES|QL query Added in 8.18.0

Simulate a pipeline Added in 5.0.0

Body Required

value object | array[object] Required

input_output object | array[object]

dataset string | array[string]

namespace string | array[string]

value object | array[object] Required

input_output object | array[object]

dataset string | array[string]

namespace string | array[string]

_version number | string

Get model snapshots info Added in 5.4.0

end string | number

start string | number

log_time string | number Required

model_bytes number | string Required

model_bytes_exceeded number | string

model_bytes_memory_limit number | string

output_memory_allocator_bytes number | string

peak_model_bytes number | string

Get model snapshots info Added in 5.4.0

end string | number

start string | number

log_time string | number Required

model_bytes number | string Required

model_bytes_exceeded number | string

model_bytes_memory_limit number | string

output_memory_allocator_bytes number | string

peak_model_bytes number | string

Delete a data frame analytics job Added in 7.3.0

Get rollup job information Deprecated Technical preview

relocating_node string | null

Run a search application search Beta

total object | number

_score number | string | null

matched_queries array[string] | object

max_score number | string | null

Get service accounts Added in 7.13.0

query string | object

query string | object

global array[object] | object

Invalidate SAML Added in 7.5.0

Body Required

Update user profile data Added in 8.2.0

Body Required