Run a retention policy | Elasticsearch API documentation

Get shard recovery information

GET /_cat/recovery

Get information about ongoing and completed shard recoveries. Shard recovery is the process of initializing a shard copy, such as restoring a primary shard from a snapshot or syncing a replica shard from a primary shard. When a shard recovery completes, the recovered shard is available for search and indexing. For data streams, the API returns information about the stream’s backing indices. IMPORTANT: cat APIs are only intended for human consumption using the command line or Kibana console. They are not intended for use by applications. For application consumption, use the index recovery API.

Query parameters

active_only boolean

If true, the response only includes ongoing shard recoveries.
bytes string

The unit used to display byte values.

Values are b, kb, mb, gb, tb, or pb.
detailed boolean

If true, the response includes detailed information about shard recoveries.
index string | array[string]

Comma-separated list or wildcard expression of index names to limit the returned information
h string | array[string]

List of columns to appear in the response. Supports simple wildcards.
s string | array[string]

List of columns that determine how the table should be sorted. Sorting defaults to ascending and can be changed by setting :asc or :desc as a suffix to the column name.
time string

Unit used to display time values.

Values are nanos, micros, ms, s, m, h, or d.

Responses

200 application/json
Hide response attributes Show response attributes object
- index string
- shard string
  
  The shard name.
- start_time string | number
  
  A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
  
  One of:
  DateTime string UnitMillis number
- start_time_millis number
  
  Time unit for milliseconds
- stop_time string | number
  
  A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
  
  One of:
  DateTime string UnitMillis number
- stop_time_millis number
  
  Time unit for milliseconds
- time string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
- type string
  
  The recovery type.
- stage string
  
  The recovery stage.
- source_host string
  
  The source host.
- source_node string
  
  The source node name.
- target_host string
  
  The target host.
- target_node string
  
  The target node name.
- repository string
  
  The repository name.
- snapshot string
  
  The snapshot name.
- files string
  
  The number of files to recover.
- files_recovered string
  
  The files recovered.
- files_percent string | number
  
  One of:
  Percentage string Percentage number
- files_total string
  
  The total number of files.
- bytes string
  
  The number of bytes to recover.
- bytes_recovered string
  
  The bytes recovered.
- bytes_percent string | number
  
  One of:
  Percentage string Percentage number
- bytes_total string
  
  The total number of bytes.
- translog_ops string
  
  The number of translog operations to recover.
- translog_ops_recovered string
  
  The translog operations recovered.
- translog_ops_percent string | number
  
  One of:
  Percentage string Percentage number

GET /_cat/recovery

curl \
 --request GET 'http://api.example.com/_cat/recovery' \
 --header "Authorization: $API_KEY"

Response examples (200)

A successful response from `GET _cat/recovery?v=true&format=json`. In this example, the source and target nodes are the same because the recovery type is `store`, meaning they were read from local storage on node start.

[
  {
    "index": "my-index-000001 ",
    "shard": "0",
    "time": "13ms",
    "type": "store",
    "stage": "done",
    "source_host": "n/a",
    "source_node": "n/a",
    "target_host": "127.0.0.1",
    "target_node": "node-0",
    "repository": "n/a",
    "snapshot": "n/a",
    "files": "0",
    "files_recovered": "0",
    "files_percent": "100.0%",
    "files_total": "13",
    "bytes": "0b",
    "bytes_recovered": "0b",
    "bytes_percent": "100.0%",
    "bytes_total": "9928b",
    "translog_ops": "0",
    "translog_ops_recovered": "0",
    "translog_ops_percent": "100.0%"
  }
]

A successful response from `GET _cat/recovery?v=true&h=i,s,t,ty,st,shost,thost,f,fp,b,bp&format=json`. You can retrieve information about an ongoing recovery for example when you increase the replica count of an index and bring another node online to host the replicas. In this example, the recovery type is `peer`, meaning the shard recovered from another node. The `files` and `bytes` are real-time measurements.

[
  {
    "i": "my-index-000001",
    "s": "0",
    "t": "1252ms",
    "ty": "peer",
    "st": "done",
    "shost": "192.168.1.1",
    "thost": "192.168.1.1",
    "f": "0",
    "fp": "100.0%",
    "b": "0b",
    "bp": "100.0%",
  }
]

A successful response from `GET _cat/recovery?v=true&h=i,s,t,ty,st,rep,snap,f,fp,b,bp&format=json`. You can restore backups of an index using the snapshot and restore API. You can use the cat recovery API to get information about a snapshot recovery.

[
  {
    "i": "my-index-000001",
    "s": "0",
    "t": "1978ms",
    "ty": "snapshot",
    "st": "done",
    "rep": "my-repo",
    "snap": "snap-1",
    "f": "79",
    "fp": "8.0%",
    "b": "12086",
    "bp": "9.0%"
  }
]

Update field mappings

POST /{index}/_mapping

Api key auth Basic auth Bearer auth

Add new fields to an existing data stream or index. You can also use this API to change the search settings of existing fields and add new properties to existing object fields. For data streams, these changes are applied to all backing indices by default.

Add multi-fields to an existing field

Multi-fields let you index the same field in different ways. You can use this API to update the fields mapping parameter and enable multi-fields for an existing field. WARNING: If an index (or data stream) contains documents when you add a multi-field, those documents will not have values for the new multi-field. You can populate the new multi-field with the update by query API.

Change supported mapping parameters for an existing field

The documentation for each mapping parameter indicates whether you can update it for an existing field using this API. For example, you can use the update mapping API to update the ignore_above parameter.

Change the mapping of an existing field

Except for supported mapping parameters, you can't change the mapping or field type of an existing field. Changing an existing field could invalidate data that's already indexed.

If you need to change the mapping of a field in a data stream's backing indices, refer to documentation about modifying data streams. If you need to change the mapping of a field in other indices, create a new index with the correct mapping and reindex your data into that index.

Rename a field

Renaming a field would invalidate data already indexed under the old field name. Instead, add an alias field to create an alternate field name.

External documentation

Path parameters

index string | array[string] Required

A comma-separated list of index names the mapping should be added to (supports wildcards); use _all or omit to add the mapping on all indices.

Query parameters

allow_no_indices boolean

If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.
expand_wildcards string | array[string]

Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.
ignore_unavailable boolean

If false, the request returns an error if it targets a missing or closed index.
master_timeout string

Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
timeout string

Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
write_index_only boolean

If true, the mappings are applied only to the current write index for the target.

application/json

Body Required

date_detection boolean

Controls whether dynamic date detection is enabled.
dynamic string

Values are strict, runtime, true, or false.
dynamic_date_formats array[string]

If date detection is enabled then new string fields are checked against 'dynamic_date_formats' and if the value matches then a new date field is added instead of string.
dynamic_templates array[object]

Specify dynamic templates for the mapping.
_field_names object
Hide _field_names attribute Show _field_names attribute object
- enabled boolean Required
_meta object
Hide _meta attribute Show _meta attribute object
- * object Additional properties
numeric_detection boolean

Automatically map strings into numeric data types for all fields.
properties object
Mapping for a field. For new fields, this mapping can include:
- Field name
- Field data type
- Mapping parameters
_routing object
Hide _routing attribute Show _routing attribute object
- required boolean Required
_source object
Hide _source attributes Show _source attributes object
- compress boolean
- compress_threshold string
- enabled boolean
- excludes array[string]
- includes array[string]
- mode string
  
  Values are disabled, stored, or synthetic.
runtime object
Hide runtime attribute Show runtime attribute object
- * object Additional properties
  Hide * attributes Show * attributes object
  
  fields object
  
  For type composite
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  Hide * attribute Show * attribute object
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.
  
  fetch_fields array[object]
  
  For type lookup
  
  Hide fetch_fields attributes Show fetch_fields attributes object
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  format string
  
  format string
  
  A custom format for date type runtime fields.
  
  input_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_index string
  
  script object
  
  Hide script attributes Show script attributes object
  
  source string | object
  
  One of:
  ScriptSource string ScriptSource object
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  lang string
  
  Any of:
  ScriptLanguage string ScriptLanguage string
  
  Values are painless, expression, mustache, or java.
  
  options object
  
  Hide options attribute Show options attribute object
  
  * string Additional properties
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.

Responses

200 application/json
Hide response attributes Show response attributes object
- acknowledged boolean Required
  
  For a successful response, this value is always true. On failure, an exception is returned instead.
- _shards object
  
  Hide _shards attributes Show _shards attributes object
  
  failed number Required
  
  successful number Required
  
  total number Required
  
  failures array[object]
  
  Hide failures attributes Show failures attributes object
  
  index string
  
  node string
  
  reason object Required
  
  Hide reason attributes Show reason attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  shard number Required
  
  status string
  
  skipped number

POST /{index}/_mapping

curl \
 --request POST 'http://api.example.com/{index}/_mapping' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"properties\": {\n    \"user\": {\n      \"properties\": {\n        \"name\": {\n          \"type\": \"keyword\"\n        }\n      }\n    }\n  }\n}"'

Request example

The update mapping API can be applied to multiple data streams or indices with a single request. For example, run `PUT /my-index-000001,my-index-000002/_mapping` to update mappings for the `my-index-000001` and `my-index-000002` indices at the same time.

{
  "properties": {
    "user": {
      "properties": {
        "name": {
          "type": "keyword"
        }
      }
    }
  }
}

Response examples (200)

{
  "acknowledged": true,
  "_shards": {
    "failed": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "failures": [
      {
        "index": "string",
        "node": "string",
        "reason": {
          "type": "string",
          "reason": "string",
          "stack_trace": "string",
          "caused_by": {},
          "root_cause": [
            {}
          ],
          "suppressed": [
            {}
          ]
        },
        "shard": 42.0,
        "status": "string"
      }
    ],
    "skipped": 42.0
  }
}

Validate a query Added in 1.3.0

GET /{index}/_validate/query

Api key auth Basic auth Bearer auth

Validates a query without running it.

Path parameters

index string | array[string] Required

Comma-separated list of data streams, indices, and aliases to search. Supports wildcards (*). To search all data streams or indices, omit this parameter or use * or _all.

Query parameters

allow_no_indices boolean

If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.
all_shards boolean

If true, the validation is executed on all shards instead of one random shard per index.
analyzer string

Analyzer to use for the query string. This parameter can only be used when the q query string parameter is specified.
analyze_wildcard boolean

If true, wildcard and prefix queries are analyzed.
default_operator string

The default operator for query string query: AND or OR.

Values are and, AND, or, or OR.
df string

Field to use as default where no field prefix is given in the query string. This parameter can only be used when the q query string parameter is specified.
expand_wildcards string | array[string]

Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.
explain boolean

If true, the response returns detailed information if an error has occurred.
ignore_unavailable boolean

If false, the request returns an error if it targets a missing or closed index.
lenient boolean

If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.
rewrite boolean

If true, returns a more detailed explanation showing the actual Lucene query that will be executed.
q string

Query in the Lucene query string syntax.

application/json

Body

query object

An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

External documentation

Responses

200 application/json
Hide response attributes Show response attributes object
- explanations array[object]
  
  Hide explanations attributes Show explanations attributes object
  
  error string
  
  explanation string
  
  index string Required
  
  valid boolean Required
- _shards object
  
  Hide _shards attributes Show _shards attributes object
  
  failed number Required
  
  successful number Required
  
  total number Required
  
  failures array[object]
  
  Hide failures attributes Show failures attributes object
  
  index string
  
  node string
  
  reason object Required
  
  Hide reason attributes Show reason attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  shard number Required
  
  status string
  
  skipped number
- valid boolean Required
- error string

GET /{index}/_validate/query

curl \
 --request GET 'http://api.example.com/{index}/_validate/query' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '{"query":{}}'

Request examples

{
  "query": {}
}

Response examples (200)

{
  "explanations": [
    {
      "error": "string",
      "explanation": "string",
      "index": "string",
      "valid": true
    }
  ],
  "_shards": {
    "failed": 42.0,
    "successful": 42.0,
    "total": 42.0,
    "failures": [
      {
        "index": "string",
        "node": "string",
        "reason": {
          "type": "string",
          "reason": "string",
          "stack_trace": "string",
          "caused_by": {},
          "root_cause": [
            {}
          ],
          "suppressed": [
            {}
          ]
        },
        "shard": 42.0,
        "status": "string"
      }
    ],
    "skipped": 42.0
  },
  "valid": true,
  "error": "string"
}

Get lifecycle policies Added in 6.6.0

GET /_ilm/policy/{policy}

Api key auth Basic auth Bearer auth

Path parameters

policy string Required

Identifier for the policy.

Query parameters

master_timeout string

Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
timeout string

Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Responses

200 application/json
Hide response attribute Show response attribute object
- * object Additional properties
  
  Hide * attributes Show * attributes object
  
  modified_date string | number Required
  
  A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
  
  One of:
  DateTime string UnitMillis number
  
  policy object Required
  
  Hide policy attributes Show policy attributes object
  
  phases object Required
  
  Hide phases attributes Show phases attributes object
  
  cold object
  
  Hide cold attributes Show cold attributes object
  
  actions object
  
  Hide actions attributes Show actions attributes object
  
  allocate object
  
  delete object
  
  downsample object
  
  freeze object
  
  forcemerge object
  
  migrate object
  
  readonly object
  
  rollover object
  
  set_priority object
  
  searchable_snapshot object
  
  shrink object
  
  unfollow object
  
  wait_for_snapshot object
  
  min_age string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  delete object
  
  Hide delete attributes Show delete attributes object
  
  actions object
  
  Hide actions attributes Show actions attributes object
  
  allocate object
  
  delete object
  
  downsample object
  
  freeze object
  
  forcemerge object
  
  migrate object
  
  readonly object
  
  rollover object
  
  set_priority object
  
  searchable_snapshot object
  
  shrink object
  
  unfollow object
  
  wait_for_snapshot object
  
  min_age string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  frozen object
  
  Hide frozen attributes Show frozen attributes object
  
  actions object
  
  Hide actions attributes Show actions attributes object
  
  allocate object
  
  delete object
  
  downsample object
  
  freeze object
  
  forcemerge object
  
  migrate object
  
  readonly object
  
  rollover object
  
  set_priority object
  
  searchable_snapshot object
  
  shrink object
  
  unfollow object
  
  wait_for_snapshot object
  
  min_age string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  hot object
  
  Hide hot attributes Show hot attributes object
  
  actions object
  
  Hide actions attributes Show actions attributes object
  
  allocate object
  
  delete object
  
  downsample object
  
  freeze object
  
  forcemerge object
  
  migrate object
  
  readonly object
  
  rollover object
  
  set_priority object
  
  searchable_snapshot object
  
  shrink object
  
  unfollow object
  
  wait_for_snapshot object
  
  min_age string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  warm object
  
  Hide warm attributes Show warm attributes object
  
  actions object
  
  Hide actions attributes Show actions attributes object
  
  allocate object
  
  delete object
  
  downsample object
  
  freeze object
  
  forcemerge object
  
  migrate object
  
  readonly object
  
  rollover object
  
  set_priority object
  
  searchable_snapshot object
  
  shrink object
  
  unfollow object
  
  wait_for_snapshot object
  
  min_age string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  _meta object
  
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties
  
  version number Required

GET /_ilm/policy/{policy}

curl \
 --request GET 'http://api.example.com/_ilm/policy/{policy}' \
 --header "Authorization: $API_KEY"

Response examples (200)

A successful response when retrieving a lifecycle policy.

{
  "my_policy": {
    "version": 1,
    "modified_date": 82392349,
    "policy": {
      "phases": {
        "warm": {
          "min_age": "10d",
          "actions": {
            "forcemerge": {
              "max_num_segments": 1
            }
          }
        },
        "delete": {
          "min_age": "30d",
          "actions": {
            "delete": {
              "delete_searchable_snapshot": true
            }
          }
        }
      }
    },
    "in_use_by" : {
      "indices" : [],
      "data_streams" : [],
      "composable_templates" : []
    }
  }
}

Simulate data ingestion Technical preview

GET /_ingest/{index}/_simulate

Api key auth Basic auth Bearer auth

Run ingest pipelines against a set of provided documents, optionally with substitute pipeline definitions, to simulate ingesting data into an index.

This API is meant to be used for troubleshooting or pipeline development, as it does not actually index any data into Elasticsearch.

The API runs the default and final pipeline for that index against a set of documents provided in the body of the request. If a pipeline contains a reroute processor, it follows that reroute processor to the new index, running that index's pipelines as well the same way that a non-simulated ingest would. No data is indexed into Elasticsearch. Instead, the transformed document is returned, along with the list of pipelines that have been run and the name of the index where the document would have been indexed if this were not a simulation. The transformed document is validated against the mappings that would apply to this index, and any validation error is reported in the result.

This API differs from the simulate pipeline API in that you specify a single pipeline for that API, and it runs only that one pipeline. The simulate pipeline API is more useful for developing a single pipeline, while the simulate ingest API is more useful for troubleshooting the interaction of the various pipelines that get applied when ingesting into an index.

By default, the pipeline definitions that are currently in the system are used. However, you can supply substitute pipeline definitions in the body of the request. These will be used in place of the pipeline definitions that are already in the system. This can be used to replace existing pipeline definitions or to create new ones. The pipeline substitutions are used only within this request.

Path parameters

index string Required

The index to simulate ingesting into. This value can be overridden by specifying an index on each document. If you specify this parameter in the request path, it is used for any documents that do not explicitly specify an index argument.

Query parameters

pipeline string

The pipeline to use as the default pipeline. This value can be used to override the default pipeline of the index.

application/json

Body Required

docs array[object] Required

Sample documents to test in the pipeline.
Hide docs attributes Show docs attributes object
- _id string
- _index string
- _source object Required
  
  JSON body for the document.
component_template_substitutions object

A map of component template names to substitute component template definition objects.
Hide component_template_substitutions attribute Show component_template_substitutions attribute object
- * object
  Hide * attributes Show * attributes object
  
  template object Required
  
  Hide template attributes Show template attributes object
  
  _meta object
  
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties
  
  version number
  
  settings object
  
  Hide settings attribute Show settings attribute object
  
  * object
  
  Hide * attributes Show * attributes object
  
  index object
  
  mode string
  
  routing_path string | array[string]
  
  One of:
  string-1 string array-2 array[string]
  
  soft_deletes object
  
  Hide soft_deletes attributes Show soft_deletes attributes object
  
  enabled boolean
  
  Indicates whether soft deletes are enabled on the index.
  
  retention_lease object
  
  sort object
  
  Hide sort attributes Show sort attributes object
  
  field string | array[string]
  
  order
  
  mode
  
  missing
  
  number_of_shards number | string
  
  One of:
  number-1 number string-2 string
  
  number_of_replicas number | string
  
  One of:
  number-1 number string-2 string
  
  number_of_routing_shards number
  
  check_on_startup string
  
  Values are true, false, or checksum.
  
  codec string
  
  routing_partition_size number | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedinteger number Stringifiedinteger string
  
  load_fixed_bitset_filters_eagerly boolean
  
  hidden boolean | string
  
  One of:
  boolean-1 boolean string-2 string
  
  auto_expand_replicas string | null
  
  One of:
  string-1 string NullValue string | null
  
  merge object
  
  Hide merge attribute Show merge attribute object
  
  scheduler object
  
  search object
  
  Hide search attributes Show search attributes object
  
  idle object
  
  slowlog object
  
  refresh_interval string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  max_result_window number
  
  max_inner_result_window number
  
  max_rescore_window number
  
  max_docvalue_fields_search number
  
  max_script_fields number
  
  max_ngram_diff number
  
  max_shingle_diff number
  
  blocks object
  
  Hide blocks attributes Show blocks attributes object
  
  read_only
  
  read_only_allow_delete
  
  read
  
  write
  
  metadata
  
  max_refresh_listeners number
  
  analyze object
  
  Hide analyze attribute Show analyze attribute object
  
  max_token_count
  
  highlight object
  
  Hide highlight attribute Show highlight attribute object
  
  max_analyzed_offset number
  
  max_terms_count number
  
  max_regex_length number
  
  routing object
  
  Hide routing attributes Show routing attributes object
  
  allocation object
  
  rebalance object
  
  gc_deletes string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  default_pipeline string
  
  final_pipeline string
  
  lifecycle object
  
  Hide lifecycle attributes Show lifecycle attributes object
  
  name string
  
  indexing_complete
  
  origination_date number
  
  If specified, this is the timestamp used to calculate the index age for its phase transitions. Use this setting if you create a new index that contains old data and want to use the original creation date to calculate the index age. Specified as a Unix epoch value in milliseconds.
  
  parse_origination_date boolean
  
  Set to true to parse the origination date from the index name. This origination date is used to calculate the index age for its phase transitions. The index name must match the pattern ^{.*-{date_format}-\d+,} where the date_format is yyyy.MM.dd and the trailing digits are optional. An index that was rolled over would normally match the full format, for example logs-2016.10.31-000002). If the index name doesn’t match the pattern, index creation fails.
  
  step object
  
  rollover_alias string
  
  The index alias to update when the index rolls over. Specify when using a policy that contains a rollover action. When the index rolls over, the alias is updated to reflect that the index is no longer the write index. For more information about rolling indices, see Rollover.
  
  prefer_ilm
  
  provided_name string
  
  creation_date string
  
  creation_date_string string
  
  uuid string
  
  version object
  
  Hide version attributes Show version attributes object
  
  created string
  
  created_string string
  
  verified_before_close boolean | string
  
  One of:
  boolean-1 boolean string-2 string
  
  format string | number
  
  One of:
  string-1 string number-2 number
  
  max_slices_per_scroll number
  
  translog object
  
  Hide translog attributes Show translog attributes object
  
  sync_interval string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  durability string
  
  Values are request, REQUEST, async, or ASYNC.
  
  flush_threshold_size
  
  retention object
  
  query_string object
  
  Hide query_string attribute Show query_string attribute object
  
  lenient
  
  priority number | string
  
  One of:
  number-1 number string-2 string
  
  top_metrics_max_size number
  
  analysis object
  
  Hide analysis attributes Show analysis attributes object
  
  analyzer object
  
  char_filter object
  
  filter object
  
  normalizer object
  
  tokenizer object
  
  settings object
  
  time_series object
  
  Hide time_series attributes Show time_series attributes object
  
  end_time
  
  start_time
  
  queries object
  
  Hide queries attribute Show queries attribute object
  
  cache object
  
  similarity object
  
  Configure custom similarity settings to customize how search results are scored.
  
  mapping object
  
  Hide mapping attributes Show mapping attributes object
  
  coerce boolean
  
  total_fields object
  
  depth object
  
  nested_fields object
  
  nested_objects object
  
  field_name_length object
  
  dimension_fields object
  
  source object
  
  ignore_malformed
  
  indexing.slowlog object
  
  Hide indexing.slowlog attributes Show indexing.slowlog attributes object
  
  level string
  
  source number
  
  reformat boolean
  
  threshold object
  
  indexing_pressure object
  
  Hide indexing_pressure attribute Show indexing_pressure attribute object
  
  memory object Required
  
  store object
  
  Hide store attributes Show store attributes object
  
  type
  
  allow_mmap boolean
  
  You can restrict the use of the mmapfs and the related hybridfs store type via the setting node.store.allow_mmap. This is a boolean setting indicating whether or not memory-mapping is allowed. The default is to allow it. This setting is useful, for example, if you are in an environment where you can not control the ability to create a lot of memory maps so you need disable the ability to use memory-mapping.
  
  mappings object
  
  Hide mappings attributes Show mappings attributes object
  
  all_field object
  
  Hide all_field attributes Show all_field attributes object
  
  analyzer string Required
  
  enabled boolean Required
  
  omit_norms boolean Required
  
  search_analyzer string Required
  
  similarity string Required
  
  store boolean Required
  
  store_term_vector_offsets boolean Required
  
  store_term_vector_payloads boolean Required
  
  store_term_vector_positions boolean Required
  
  store_term_vectors boolean Required
  
  date_detection boolean
  
  dynamic string
  
  Values are strict, runtime, true, or false.
  
  dynamic_date_formats array[string]
  
  dynamic_templates array[object]
  
  _field_names object
  
  Hide _field_names attribute Show _field_names attribute object
  
  enabled boolean Required
  
  index_field object
  
  Hide index_field attribute Show index_field attribute object
  
  enabled boolean Required
  
  _meta object
  
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties
  
  numeric_detection boolean
  
  properties object
  
  _routing object
  
  Hide _routing attribute Show _routing attribute object
  
  required boolean Required
  
  _size object
  
  Hide _size attribute Show _size attribute object
  
  enabled boolean Required
  
  _source object
  
  Hide _source attributes Show _source attributes object
  
  compress boolean
  
  compress_threshold string
  
  enabled boolean
  
  excludes array[string]
  
  includes array[string]
  
  mode string
  
  Values are disabled, stored, or synthetic.
  
  runtime object
  
  Hide runtime attribute Show runtime attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  fields object
  
  For type composite
  
  fetch_fields array[object]
  
  For type lookup
  
  format string
  
  A custom format for date type runtime fields.
  
  input_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_index string
  
  script object
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.
  
  enabled boolean
  
  subobjects string
  
  Values are true, false, or auto.
  
  _data_stream_timestamp object
  
  Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
  
  enabled boolean Required
  
  aliases object
  
  Hide aliases attribute Show aliases attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  filter object
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  External documentation
  
  index_routing string
  
  Value used to route indexing operations to a specific shard. If specified, this overwrites the routing value for indexing operations.
  
  is_write_index boolean
  
  If true, the index is the write index for the alias.
  
  routing string
  
  Value used to route indexing and search operations to a specific shard.
  
  search_routing string
  
  Value used to route search operations to a specific shard. If specified, this overwrites the routing value for search operations.
  
  is_hidden boolean
  
  If true, the alias is hidden. All indices for the alias must have the same is_hidden value.
  
  lifecycle object
  
  Hide lifecycle attributes Show lifecycle attributes object
  
  data_retention string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  downsampling object
  
  Hide downsampling attribute Show downsampling attribute object
  
  rounds array[object] Required
  
  The list of downsampling rounds to execute as part of this downsampling configuration
  
  enabled boolean
  
  If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.
  
  rollover object
  
  Hide rollover attributes Show rollover attributes object
  
  min_age string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  max_age string
  
  min_docs number
  
  max_docs number
  
  min_size
  
  max_size
  
  min_primary_shard_size
  
  max_primary_shard_size
  
  min_primary_shard_docs number
  
  max_primary_shard_docs number
  
  version number
  
  _meta object
  
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties
  
  deprecated boolean
index_template_substitutions object

A map of index template names to substitute index template definition objects.
Hide index_template_substitutions attribute Show index_template_substitutions attribute object
- * object
  Hide * attributes Show * attributes object
  
  index_patterns string | array[string] Required
  
  composed_of array[string] Required
  
  An ordered list of component template names. Component templates are merged in the order specified, meaning that the last component template specified has the highest precedence.
  
  template object
  
  Hide template attributes Show template attributes object
  
  aliases object
  
  Aliases to add. If the index template includes a data_stream object, these are data stream aliases. Otherwise, these are index aliases. Data stream aliases ignore the index_routing, routing, and search_routing options.
  
  Hide aliases attribute Show aliases attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  filter object
  
  An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.
  
  External documentation
  
  index_routing string
  
  is_hidden boolean
  
  If true, the alias is hidden. All indices for the alias must have the same is_hidden value.
  
  is_write_index boolean
  
  If true, the index is the write index for the alias.
  
  routing string
  
  search_routing string
  
  mappings object
  
  Hide mappings attributes Show mappings attributes object
  
  all_field object
  
  Hide all_field attributes Show all_field attributes object
  
  analyzer string Required
  
  enabled boolean Required
  
  omit_norms boolean Required
  
  search_analyzer string Required
  
  similarity string Required
  
  store boolean Required
  
  store_term_vector_offsets boolean Required
  
  store_term_vector_payloads boolean Required
  
  store_term_vector_positions boolean Required
  
  store_term_vectors boolean Required
  
  date_detection boolean
  
  dynamic string
  
  Values are strict, runtime, true, or false.
  
  dynamic_date_formats array[string]
  
  dynamic_templates array[object]
  
  _field_names object
  
  Hide _field_names attribute Show _field_names attribute object
  
  enabled boolean Required
  
  index_field object
  
  Hide index_field attribute Show index_field attribute object
  
  enabled boolean Required
  
  _meta object
  
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties
  
  numeric_detection boolean
  
  properties object
  
  _routing object
  
  Hide _routing attribute Show _routing attribute object
  
  required boolean Required
  
  _size object
  
  Hide _size attribute Show _size attribute object
  
  enabled boolean Required
  
  _source object
  
  Hide _source attributes Show _source attributes object
  
  compress boolean
  
  compress_threshold string
  
  enabled boolean
  
  excludes array[string]
  
  includes array[string]
  
  mode string
  
  Values are disabled, stored, or synthetic.
  
  runtime object
  
  Hide runtime attribute Show runtime attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  fields object
  
  For type composite
  
  fetch_fields array[object]
  
  For type lookup
  
  format string
  
  A custom format for date type runtime fields.
  
  input_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_index string
  
  script object
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.
  
  enabled boolean
  
  subobjects string
  
  Values are true, false, or auto.
  
  _data_stream_timestamp object
  
  Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
  
  enabled boolean Required
  
  settings object
  
  Hide settings attributes Show settings attributes object
  
  index object
  
  mode string
  
  routing_path string | array[string]
  
  One of:
  string-1 string array-2 array[string]
  
  soft_deletes object
  
  Hide soft_deletes attributes Show soft_deletes attributes object
  
  enabled boolean
  
  Indicates whether soft deletes are enabled on the index.
  
  retention_lease object
  
  Hide retention_lease attribute Show retention_lease attribute object
  
  period string Required
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  sort object
  
  Hide sort attributes Show sort attributes object
  
  field string | array[string]
  
  order string | array[string]
  
  One of:
  SegmentSortOrder string array-2 array[string]
  
  Values are asc, ASC, desc, or DESC.
  
  mode string | array[string]
  
  One of:
  SegmentSortMode string array-2 array[string]
  
  Values are min, MIN, max, or MAX.
  
  missing string | array[string]
  
  One of:
  SegmentSortMissing string array-2 array[string]
  
  Values are _last or _first.
  
  number_of_shards number | string
  
  One of:
  number-1 number string-2 string
  
  number_of_replicas number | string
  
  One of:
  number-1 number string-2 string
  
  number_of_routing_shards number
  
  check_on_startup string
  
  Values are true, false, or checksum.
  
  codec string
  
  routing_partition_size number | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedinteger number Stringifiedinteger string
  
  load_fixed_bitset_filters_eagerly boolean
  
  hidden boolean | string
  
  One of:
  boolean-1 boolean string-2 string
  
  auto_expand_replicas string | null
  
  One of:
  string-1 string NullValue string | null
  
  merge object
  
  Hide merge attribute Show merge attribute object
  
  scheduler object
  
  Hide scheduler attributes Show scheduler attributes object
  
  max_thread_count
  
  max_merge_count
  
  search object
  
  Hide search attributes Show search attributes object
  
  idle object
  
  Hide idle attribute Show idle attribute object
  
  after string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  slowlog object
  
  Hide slowlog attributes Show slowlog attributes object
  
  level string
  
  source number
  
  reformat boolean
  
  threshold object
  
  refresh_interval string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  max_result_window number
  
  max_inner_result_window number
  
  max_rescore_window number
  
  max_docvalue_fields_search number
  
  max_script_fields number
  
  max_ngram_diff number
  
  max_shingle_diff number
  
  blocks object
  
  Hide blocks attributes Show blocks attributes object
  
  read_only boolean | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  read_only_allow_delete boolean | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  read boolean | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  write boolean | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  metadata boolean | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  max_refresh_listeners number
  
  analyze object
  
  Hide analyze attribute Show analyze attribute object
  
  max_token_count number | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedinteger number Stringifiedinteger string
  
  highlight object
  
  Hide highlight attribute Show highlight attribute object
  
  max_analyzed_offset number
  
  max_terms_count number
  
  max_regex_length number
  
  routing object
  
  Hide routing attributes Show routing attributes object
  
  allocation object
  
  Hide allocation attributes Show allocation attributes object
  
  enable string
  
  Values are all, primaries, new_primaries, or none.
  
  include object
  
  initial_recovery object
  
  disk object
  
  rebalance object
  
  Hide rebalance attribute Show rebalance attribute object
  
  enable string Required
  
  Values are all, primaries, replicas, or none.
  
  gc_deletes string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  default_pipeline string
  
  final_pipeline string
  
  lifecycle object
  
  Hide lifecycle attributes Show lifecycle attributes object
  
  name string
  
  indexing_complete boolean | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  origination_date number
  
  If specified, this is the timestamp used to calculate the index age for its phase transitions. Use this setting if you create a new index that contains old data and want to use the original creation date to calculate the index age. Specified as a Unix epoch value in milliseconds.
  
  parse_origination_date boolean
  
  Set to true to parse the origination date from the index name. This origination date is used to calculate the index age for its phase transitions. The index name must match the pattern ^{.*-{date_format}-\d+,} where the date_format is yyyy.MM.dd and the trailing digits are optional. An index that was rolled over would normally match the full format, for example logs-2016.10.31-000002). If the index name doesn’t match the pattern, index creation fails.
  
  step object
  
  Hide step attribute Show step attribute object
  
  wait_time_threshold string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  rollover_alias string
  
  The index alias to update when the index rolls over. Specify when using a policy that contains a rollover action. When the index rolls over, the alias is updated to reflect that the index is no longer the write index. For more information about rolling indices, see Rollover.
  
  prefer_ilm boolean | string
  
  Preference for the system that manages a data stream backing index (preferring ILM when both ILM and DLM are applicable for an index).
  
  One of:
  boolean-1 boolean string-2 string
  
  provided_name string
  
  creation_date number | string
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  UnitMillis number StringifiedEpochTimeUnitMillis string
  
  Time unit for milliseconds
  
  creation_date_string string | number
  
  A date and time, either as a string whose format can depend on the context (defaulting to ISO 8601), or a number of milliseconds since the Epoch. Elasticsearch accepts both as input, but will generally output a string representation.
  
  One of:
  DateTime string UnitMillis number
  
  uuid string
  
  version object
  
  Hide version attributes Show version attributes object
  
  created string
  
  created_string string
  
  verified_before_close boolean | string
  
  One of:
  boolean-1 boolean string-2 string
  
  format string | number
  
  One of:
  string-1 string number-2 number
  
  max_slices_per_scroll number
  
  translog object
  
  Hide translog attributes Show translog attributes object
  
  sync_interval string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  durability string
  
  Values are request, REQUEST, async, or ASYNC.
  
  flush_threshold_size number | string
  
  One of:
  ByteSize number ByteSize string
  
  retention object
  
  Hide retention attributes Show retention attributes object
  
  size
  
  age string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  query_string object
  
  Hide query_string attribute Show query_string attribute object
  
  lenient boolean | string Required
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  Stringifiedboolean boolean Stringifiedboolean string
  
  priority number | string
  
  One of:
  number-1 number string-2 string
  
  top_metrics_max_size number
  
  analysis object
  
  Hide analysis attributes Show analysis attributes object
  
  analyzer object
  
  char_filter object
  
  filter object
  
  normalizer object
  
  tokenizer object
  
  settings object
  
  time_series object
  
  Hide time_series attributes Show time_series attributes object
  
  end_time string
  
  start_time string
  
  queries object
  
  Hide queries attribute Show queries attribute object
  
  cache object
  
  Hide cache attribute Show cache attribute object
  
  enabled boolean Required
  
  similarity object
  
  Configure custom similarity settings to customize how search results are scored.
  
  mapping object
  
  Hide mapping attributes Show mapping attributes object
  
  coerce boolean
  
  total_fields object
  
  Hide total_fields attributes Show total_fields attributes object
  
  limit
  
  ignore_dynamic_beyond_limit
  
  depth object
  
  Hide depth attribute Show depth attribute object
  
  limit number
  
  The maximum depth for a field, which is measured as the number of inner objects. For instance, if all fields are defined at the root object level, then the depth is 1. If there is one object mapping, then the depth is 2, etc.
  
  nested_fields object
  
  Hide nested_fields attribute Show nested_fields attribute object
  
  limit number
  
  The maximum number of distinct nested mappings in an index. The nested type should only be used in special cases, when arrays of objects need to be queried independently of each other. To safeguard against poorly designed mappings, this setting limits the number of unique nested types per index.
  
  nested_objects object
  
  Hide nested_objects attribute Show nested_objects attribute object
  
  limit number
  
  The maximum number of nested JSON objects that a single document can contain across all nested types. This limit helps to prevent out of memory errors when a document contains too many nested objects.
  
  field_name_length object
  
  Hide field_name_length attribute Show field_name_length attribute object
  
  limit number
  
  Setting for the maximum length of a field name. This setting isn’t really something that addresses mappings explosion but might still be useful if you want to limit the field length. It usually shouldn’t be necessary to set this setting. The default is okay unless a user starts to add a huge number of fields with really long names. Default is Long.MAX_VALUE (no limit).
  
  dimension_fields object
  
  Hide dimension_fields attribute Show dimension_fields attribute object
  
  limit number
  
  [preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
  
  source object
  
  Hide source attribute Show source attribute object
  
  mode string Required
  
  Values are disabled, stored, or synthetic.
  
  ignore_malformed boolean | string
  
  One of:
  boolean-1 boolean string-2 string
  
  indexing.slowlog object
  
  Hide indexing.slowlog attributes Show indexing.slowlog attributes object
  
  level string
  
  source number
  
  reformat boolean
  
  threshold object
  
  Hide threshold attribute Show threshold attribute object
  
  index object
  
  indexing_pressure object
  
  Hide indexing_pressure attribute Show indexing_pressure attribute object
  
  memory object Required
  
  Hide memory attribute Show memory attribute object
  
  limit number
  
  Number of outstanding bytes that may be consumed by indexing requests. When this limit is reached or exceeded, the node will reject new coordinating and primary operations. When replica operations consume 1.5x this limit, the node will reject new replica operations. Defaults to 10% of the heap.
  
  store object
  
  Hide store attributes Show store attributes object
  
  type string Required
  
  Any of:
  StorageType string StorageType string
  
  Values are fs, niofs, mmapfs, or hybridfs.
  
  allow_mmap boolean
  
  You can restrict the use of the mmapfs and the related hybridfs store type via the setting node.store.allow_mmap. This is a boolean setting indicating whether or not memory-mapping is allowed. The default is to allow it. This setting is useful, for example, if you are in an environment where you can not control the ability to create a lot of memory maps so you need disable the ability to use memory-mapping.
  
  lifecycle object
  
  Hide lifecycle attributes Show lifecycle attributes object
  
  data_retention string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  downsampling object
  
  Hide downsampling attribute Show downsampling attribute object
  
  rounds array[object] Required
  
  The list of downsampling rounds to execute as part of this downsampling configuration
  
  enabled boolean
  
  If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.
  
  rollover object
  
  Hide rollover attributes Show rollover attributes object
  
  min_age string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  max_age string
  
  min_docs number
  
  max_docs number
  
  min_size
  
  max_size
  
  min_primary_shard_size
  
  max_primary_shard_size
  
  min_primary_shard_docs number
  
  max_primary_shard_docs number
  
  version number
  
  priority number
  
  Priority to determine index template precedence when a new data stream or index is created. The index template with the highest priority is chosen. If no priority is specified the template is treated as though it is of priority 0 (lowest priority). This number is not automatically generated by Elasticsearch.
  
  _meta object
  
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties
  
  allow_auto_create boolean
  
  data_stream object
  
  Hide data_stream attributes Show data_stream attributes object
  
  hidden boolean
  
  If true, the data stream is hidden.
  
  allow_custom_routing boolean
  
  If true, the data stream supports custom routing.
  
  deprecated boolean
  
  Marks this index template as deprecated. When creating or updating a non-deprecated index template that uses deprecated components, Elasticsearch will emit a deprecation warning.
  
  ignore_missing_component_templates string | array[string]
mapping_addition object
Hide mapping_addition attributes Show mapping_addition attributes object
- all_field object
  Hide all_field attributes Show all_field attributes object
  
  analyzer string Required
  
  enabled boolean Required
  
  omit_norms boolean Required
  
  search_analyzer string Required
  
  similarity string Required
  
  store boolean Required
  
  store_term_vector_offsets boolean Required
  
  store_term_vector_payloads boolean Required
  
  store_term_vector_positions boolean Required
  
  store_term_vectors boolean Required
- date_detection boolean
- dynamic string
  
  Values are strict, runtime, true, or false.
- dynamic_date_formats array[string]
- dynamic_templates array[object]
- _field_names object
  Hide _field_names attribute Show _field_names attribute object
  
  enabled boolean Required
- index_field object
  Hide index_field attribute Show index_field attribute object
  
  enabled boolean Required
- _meta object
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties
- numeric_detection boolean
- properties object
- _routing object
  Hide _routing attribute Show _routing attribute object
  
  required boolean Required
- _size object
  Hide _size attribute Show _size attribute object
  
  enabled boolean Required
- _source object
  Hide _source attributes Show _source attributes object
  
  compress boolean
  
  compress_threshold string
  
  enabled boolean
  
  excludes array[string]
  
  includes array[string]
  
  mode string
  
  Values are disabled, stored, or synthetic.
- runtime object
  Hide runtime attribute Show runtime attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  fields object
  
  For type composite
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  Hide * attribute Show * attribute object
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.
  
  fetch_fields array[object]
  
  For type lookup
  
  Hide fetch_fields attributes Show fetch_fields attributes object
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  format string
  
  format string
  
  A custom format for date type runtime fields.
  
  input_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_index string
  
  script object
  
  Hide script attributes Show script attributes object
  
  source string | object
  
  One of:
  ScriptSource string ScriptSource object
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  lang string
  
  Any of:
  ScriptLanguage string ScriptLanguage string
  
  Values are painless, expression, mustache, or java.
  
  options object
  
  Hide options attribute Show options attribute object
  
  * string Additional properties
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.
- enabled boolean
- subobjects string
  
  Values are true, false, or auto.
- _data_stream_timestamp object
  Hide _data_stream_timestamp attribute Show _data_stream_timestamp attribute object
  
  enabled boolean Required
pipeline_substitutions object

Pipelines to test. If you don’t specify the pipeline request path parameter, this parameter is required. If you specify both this and the request path parameter, the API only uses the request path parameter.
Hide pipeline_substitutions attribute Show pipeline_substitutions attribute object
- * object Additional properties
  Hide * attributes Show * attributes object
  
  description string
  
  Description of the ingest pipeline.
  
  on_failure array[object]
  
  Processors to run immediately after a processor failure.
  
  Hide on_failure attributes Show on_failure attributes object
  
  append object
  
  Hide append attributes Show append attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  value
  
  allow_duplicates boolean
  
  If false, the processor does not append values already present in the field.
  
  attachment object
  
  Hide attachment attributes Show attachment attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  indexed_chars number
  
  The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.
  
  indexed_chars_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  remove_binary boolean
  
  If true, the binary field will be removed from the document
  
  resource_name string
  
  Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.
  
  bytes object
  
  Hide bytes attributes Show bytes attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  circle object
  
  Hide circle attributes Show circle attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  error_distance number Required
  
  The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  shape_type string Required
  
  Values are geo_shape or shape.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  community_id object
  
  Hide community_id attributes Show community_id attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  source_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  iana_number string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_type string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_code string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  transport string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  seed number
  
  Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  convert object
  
  Hide convert attributes Show convert attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  type string Required
  
  Values are integer, long, double, float, boolean, ip, string, or auto.
  
  csv object
  
  Hide csv attributes Show csv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  empty_value object
  
  Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  quote string
  
  Quote used in CSV, has to be single character string.
  
  separator string
  
  Separator used in CSV, has to be single character string.
  
  target_fields string | array[string] Required
  
  trim boolean
  
  Trim whitespaces in unquoted fields.
  
  date object
  
  Hide date attributes Show date attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  formats array[string] Required
  
  An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  locale string
  
  The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  timezone string
  
  The timezone to use when parsing the date. Supports template snippets.
  
  output_format string
  
  The format to use when writing the date to target_field. Must be a valid java time pattern.
  
  date_index_name object
  
  Hide date_index_name attributes Show date_index_name attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  date_formats array[string]
  
  An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  date_rounding string Required
  
  How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  index_name_format string
  
  The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.
  
  index_name_prefix string
  
  A prefix of the index name to be prepended before the printed date. Supports template snippets.
  
  locale string
  
  The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.
  
  timezone string
  
  The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.
  
  dissect object
  
  Hide dissect attributes Show dissect attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  append_separator string
  
  The character(s) that separate the appended fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to apply to the field.
  
  dot_expander object
  
  Hide dot_expander attributes Show dot_expander attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  override boolean
  
  Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.
  
  path string
  
  The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.
  
  drop object
  
  Hide drop attributes Show drop attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  enrich object
  
  Hide enrich attributes Show enrich attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  max_matches number
  
  The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.
  
  override boolean
  
  If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  policy_name string Required
  
  The name of the enrich policy to use.
  
  shape_relation string
  
  Values are intersects, disjoint, within, or contains.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  fail object
  
  Hide fail attributes Show fail attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  message string Required
  
  The error message thrown by the processor. Supports template snippets.
  
  fingerprint object
  
  Hide fingerprint attributes Show fingerprint attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  fields string | array[string] Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  salt string
  
  Salt value for the hash function.
  
  method string
  
  Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.
  
  ignore_missing boolean
  
  If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.
  
  foreach object
  
  Hide foreach attributes Show foreach attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true, the processor silently exits without changing the document if the field is null or missing.
  
  processor object Required
  
  ip_location object
  
  Hide ip_location attributes Show ip_location attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found IP location data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the IP location lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  geo_grid object
  
  Hide geo_grid attributes Show geo_grid attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to interpret as a geo-tile.= The field format is determined by the tile_type.
  
  tile_type string Required
  
  Values are geotile, geohex, or geohash.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  parent_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  non_children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  precision_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_format string
  
  Values are geojson or wkt.
  
  geoip object
  
  Hide geoip attributes Show geoip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found geoip data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the geoip lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  grok object
  
  Hide grok attributes Show grok attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  ecs_compatibility string
  
  Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern_definitions object
  
  A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.
  
  patterns array[string] Required
  
  An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.
  
  trace_match boolean
  
  When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.
  
  gsub object
  
  Hide gsub attributes Show gsub attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to be replaced.
  
  replacement string Required
  
  The string to replace the matching patterns with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  html_strip object
  
  Hide html_strip attributes Show html_strip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document,
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  inference object
  
  Hide inference attributes Show inference attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  model_id string Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_map object
  
  Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.
  
  inference_config object
  
  input_output
  
  ignore_missing boolean
  
  If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.
  
  join object
  
  Hide join attributes Show join attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  separator string Required
  
  The separator character.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  json object
  
  Hide json attributes Show json attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  add_to_root boolean
  
  Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.
  
  add_to_root_conflict_strategy string
  
  Values are replace or merge.
  
  allow_duplicate_keys boolean
  
  When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  kv object
  
  Hide kv attributes Show kv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  exclude_keys array[string]
  
  List of keys to exclude from document.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_split string Required
  
  Regex pattern to use for splitting key-value pairs.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  include_keys array[string]
  
  List of keys to filter and insert into document. Defaults to including all keys.
  
  prefix string
  
  Prefix to be added to extracted keys.
  
  strip_brackets boolean
  
  If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  trim_key string
  
  String of characters to trim from extracted keys.
  
  trim_value string
  
  String of characters to trim from extracted values.
  
  value_split string Required
  
  Regex pattern to use for splitting the key from the value within a key-value pair.
  
  lowercase object
  
  Hide lowercase attributes Show lowercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  network_direction object
  
  Hide network_direction attributes Show network_direction attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  internal_networks array[string]
  
  List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.
  
  internal_networks_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  pipeline object
  
  Hide pipeline attributes Show pipeline attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  name string Required
  
  ignore_missing_pipeline boolean
  
  Whether to ignore missing pipelines instead of failing.
  
  redact object
  
  Hide redact attributes Show redact attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  patterns array[string] Required
  
  A list of grok expressions to match and redact named captures with
  
  pattern_definitions object
  
  prefix string
  
  Start a redacted section with this token
  
  suffix string
  
  End a redacted section with this token
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  skip_if_unlicensed boolean
  
  If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document
  
  trace_redact boolean
  
  If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted
  
  registered_domain object
  
  Hide registered_domain attributes Show registered_domain attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  remove object
  
  Hide remove attributes Show remove attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string | array[string] Required
  
  keep string | array[string]
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  rename object
  
  Hide rename attributes Show rename attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  reroute object
  
  Hide reroute attributes Show reroute attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  destination string
  
  A static value for the target. Can’t be set when the dataset or namespace option is set.
  
  dataset
  
  namespace
  
  script object
  
  Hide script attributes Show script attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  id string
  
  lang
  
  params object
  
  Object containing parameters for the script.
  
  source
  
  set object
  
  Hide set attributes Show set attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  copy_from string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_empty_value boolean
  
  If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.
  
  media_type string
  
  The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.
  
  override boolean
  
  If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  value object
  
  The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.
  
  set_security_user object
  
  Hide set_security_user attributes Show set_security_user attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what user related properties are added to the field.
  
  sort object
  
  Hide sort attributes Show sort attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  order string
  
  Values are asc or desc.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  split object
  
  Hide split attributes Show split attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  preserve_trailing boolean
  
  Preserves empty trailing fields, if any.
  
  separator string Required
  
  A regex which matches the separator, for example, , or \s+.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  terminate object
  
  Hide terminate attributes Show terminate attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  trim object
  
  Hide trim attributes Show trim attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uppercase object
  
  Hide uppercase attributes Show uppercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  urldecode object
  
  Hide urldecode attributes Show urldecode attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uri_parts object
  
  Hide uri_parts attributes Show uri_parts attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  keep_original boolean
  
  If true, the processor copies the unparsed URI to <target_field>.original.
  
  remove_if_successful boolean
  
  If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  user_agent object
  
  Hide user_agent attributes Show user_agent attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  regex_file string
  
  The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what properties are added to target_field.
  
  Values are name, os, device, original, or version.
  
  extract_device_type boolean Beta
  
  Extracts device type from the user agent string on a best-effort basis.
  
  processors array[object]
  
  Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.
  
  Hide processors attributes Show processors attributes object
  
  append object
  
  Hide append attributes Show append attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  value
  
  allow_duplicates boolean
  
  If false, the processor does not append values already present in the field.
  
  attachment object
  
  Hide attachment attributes Show attachment attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  indexed_chars number
  
  The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.
  
  indexed_chars_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  remove_binary boolean
  
  If true, the binary field will be removed from the document
  
  resource_name string
  
  Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.
  
  bytes object
  
  Hide bytes attributes Show bytes attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  circle object
  
  Hide circle attributes Show circle attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  error_distance number Required
  
  The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  shape_type string Required
  
  Values are geo_shape or shape.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  community_id object
  
  Hide community_id attributes Show community_id attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  source_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_port string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  iana_number string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_type string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  icmp_code string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  transport string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  seed number
  
  Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  convert object
  
  Hide convert attributes Show convert attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  type string Required
  
  Values are integer, long, double, float, boolean, ip, string, or auto.
  
  csv object
  
  Hide csv attributes Show csv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  empty_value object
  
  Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  quote string
  
  Quote used in CSV, has to be single character string.
  
  separator string
  
  Separator used in CSV, has to be single character string.
  
  target_fields string | array[string] Required
  
  trim boolean
  
  Trim whitespaces in unquoted fields.
  
  date object
  
  Hide date attributes Show date attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  formats array[string] Required
  
  An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  locale string
  
  The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  timezone string
  
  The timezone to use when parsing the date. Supports template snippets.
  
  output_format string
  
  The format to use when writing the date to target_field. Must be a valid java time pattern.
  
  date_index_name object
  
  Hide date_index_name attributes Show date_index_name attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  date_formats array[string]
  
  An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.
  
  date_rounding string Required
  
  How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  index_name_format string
  
  The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.
  
  index_name_prefix string
  
  A prefix of the index name to be prepended before the printed date. Supports template snippets.
  
  locale string
  
  The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.
  
  timezone string
  
  The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.
  
  dissect object
  
  Hide dissect attributes Show dissect attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  append_separator string
  
  The character(s) that separate the appended fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to apply to the field.
  
  dot_expander object
  
  Hide dot_expander attributes Show dot_expander attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  override boolean
  
  Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.
  
  path string
  
  The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.
  
  drop object
  
  Hide drop attributes Show drop attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  enrich object
  
  Hide enrich attributes Show enrich attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  max_matches number
  
  The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.
  
  override boolean
  
  If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  policy_name string Required
  
  The name of the enrich policy to use.
  
  shape_relation string
  
  Values are intersects, disjoint, within, or contains.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  fail object
  
  Hide fail attributes Show fail attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  message string Required
  
  The error message thrown by the processor. Supports template snippets.
  
  fingerprint object
  
  Hide fingerprint attributes Show fingerprint attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  fields string | array[string] Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  salt string
  
  Salt value for the hash function.
  
  method string
  
  Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.
  
  ignore_missing boolean
  
  If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.
  
  foreach object
  
  Hide foreach attributes Show foreach attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true, the processor silently exits without changing the document if the field is null or missing.
  
  processor object Required
  
  ip_location object
  
  Hide ip_location attributes Show ip_location attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found IP location data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the IP location lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  geo_grid object
  
  Hide geo_grid attributes Show geo_grid attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  The field to interpret as a geo-tile.= The field format is determined by the tile_type.
  
  tile_type string Required
  
  Values are geotile, geohex, or geohash.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  parent_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  non_children_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  precision_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_format string
  
  Values are geojson or wkt.
  
  geoip object
  
  Hide geoip attributes Show geoip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  database_file string
  
  The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  first_only boolean
  
  If true, only the first found geoip data will be returned, even if the field contains an array.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  properties array[string]
  
  Controls what properties are added to the target_field based on the geoip lookup.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  download_database_on_pipeline_creation boolean
  
  If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.
  
  grok object
  
  Hide grok attributes Show grok attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  ecs_compatibility string
  
  Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern_definitions object
  
  A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.
  
  patterns array[string] Required
  
  An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.
  
  trace_match boolean
  
  When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.
  
  gsub object
  
  Hide gsub attributes Show gsub attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  pattern string Required
  
  The pattern to be replaced.
  
  replacement string Required
  
  The string to replace the matching patterns with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  html_strip object
  
  Hide html_strip attributes Show html_strip attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document,
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  inference object
  
  Hide inference attributes Show inference attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  model_id string Required
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_map object
  
  Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.
  
  inference_config object
  
  input_output
  
  ignore_missing boolean
  
  If true and any of the input fields defined in input_ouput are missing then those missing fields are quietly ignored, otherwise a missing field causes a failure. Only applies when using input_output configurations to explicitly list the input fields.
  
  join object
  
  Hide join attributes Show join attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  separator string Required
  
  The separator character.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  json object
  
  Hide json attributes Show json attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  add_to_root boolean
  
  Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.
  
  add_to_root_conflict_strategy string
  
  Values are replace or merge.
  
  allow_duplicate_keys boolean
  
  When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  kv object
  
  Hide kv attributes Show kv attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  exclude_keys array[string]
  
  List of keys to exclude from document.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field_split string Required
  
  Regex pattern to use for splitting key-value pairs.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  include_keys array[string]
  
  List of keys to filter and insert into document. Defaults to including all keys.
  
  prefix string
  
  Prefix to be added to extracted keys.
  
  strip_brackets boolean
  
  If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  trim_key string
  
  String of characters to trim from extracted keys.
  
  trim_value string
  
  String of characters to trim from extracted values.
  
  value_split string Required
  
  Regex pattern to use for splitting the key from the value within a key-value pair.
  
  lowercase object
  
  Hide lowercase attributes Show lowercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  network_direction object
  
  Hide network_direction attributes Show network_direction attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  source_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  destination_ip string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  internal_networks array[string]
  
  List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.
  
  internal_networks_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  pipeline object
  
  Hide pipeline attributes Show pipeline attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  name string Required
  
  ignore_missing_pipeline boolean
  
  Whether to ignore missing pipelines instead of failing.
  
  redact object
  
  Hide redact attributes Show redact attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  patterns array[string] Required
  
  A list of grok expressions to match and redact named captures with
  
  pattern_definitions object
  
  prefix string
  
  Start a redacted section with this token
  
  suffix string
  
  End a redacted section with this token
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  skip_if_unlicensed boolean
  
  If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document
  
  trace_redact boolean
  
  If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted
  
  registered_domain object
  
  Hide registered_domain attributes Show registered_domain attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and any required fields are missing, the processor quietly exits without modifying the document.
  
  remove object
  
  Hide remove attributes Show remove attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string | array[string] Required
  
  keep string | array[string]
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  rename object
  
  Hide rename attributes Show rename attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  reroute object
  
  Hide reroute attributes Show reroute attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  destination string
  
  A static value for the target. Can’t be set when the dataset or namespace option is set.
  
  dataset
  
  namespace
  
  script object
  
  Hide script attributes Show script attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  id string
  
  lang
  
  params object
  
  Object containing parameters for the script.
  
  source
  
  set object
  
  Hide set attributes Show set attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  copy_from string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_empty_value boolean
  
  If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.
  
  media_type string
  
  The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.
  
  override boolean
  
  If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.
  
  value object
  
  The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.
  
  set_security_user object
  
  Hide set_security_user attributes Show set_security_user attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what user related properties are added to the field.
  
  sort object
  
  Hide sort attributes Show sort attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  order string
  
  Values are asc or desc.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  split object
  
  Hide split attributes Show split attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  preserve_trailing boolean
  
  Preserves empty trailing fields, if any.
  
  separator string Required
  
  A regex which matches the separator, for example, , or \s+.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  terminate object
  
  Hide terminate attributes Show terminate attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  trim object
  
  Hide trim attributes Show trim attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uppercase object
  
  Hide uppercase attributes Show uppercase attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  urldecode object
  
  Hide urldecode attributes Show urldecode attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist or is null, the processor quietly exits without modifying the document.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  uri_parts object
  
  Hide uri_parts attributes Show uri_parts attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  keep_original boolean
  
  If true, the processor copies the unparsed URI to <target_field>.original.
  
  remove_if_successful boolean
  
  If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  user_agent object
  
  Hide user_agent attributes Show user_agent attributes object
  
  description string
  
  Description of the processor. Useful for describing the purpose of the processor or its configuration.
  
  if object
  
  ignore_failure boolean
  
  Ignore failures for the processor.
  
  on_failure array[object]
  
  Handle failures for the processor.
  
  tag string
  
  Identifier for the processor. Useful for debugging and metrics.
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  ignore_missing boolean
  
  If true and field does not exist, the processor quietly exits without modifying the document.
  
  regex_file string
  
  The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  properties array[string]
  
  Controls what properties are added to target_field.
  
  Values are name, os, device, original, or version.
  
  extract_device_type boolean Beta
  
  Extracts device type from the user agent string on a best-effort basis.
  
  version number
  
  deprecated boolean
  
  Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.
  
  _meta object
  
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties

Responses

200 application/json
Hide response attribute Show response attribute object
- docs array[object] Required
  
  Hide docs attribute Show docs attribute object
  
  doc object
  
  Hide doc attributes Show doc attributes object
  
  _id string Required
  
  _index string Required
  
  _source object Required
  
  JSON body for the document.
  
  Hide _source attribute Show _source attribute object
  
  * object Additional properties
  
  _version number | string Required
  
  Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.
  
  Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.
  
  One of:
  VersionNumber number StringifiedVersionNumber string
  
  executed_pipelines array[string] Required
  
  A list of the names of the pipelines executed on this document.
  
  ignored_fields array[object]
  
  A list of the fields that would be ignored at the indexing step. For example, a field whose value is larger than the allowed limit would make it through all of the pipelines, but would not be indexed into Elasticsearch.
  
  Hide ignored_fields attribute Show ignored_fields attribute object
  
  * string Additional properties
  
  error object
  
  Hide error attributes Show error attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]

GET /_ingest/{index}/_simulate

curl \
 --request GET 'http://api.example.com/_ingest/{index}/_simulate' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"docs\": [\n    {\n      \"_id\": 123,\n      \"_index\": \"my-index\",\n      \"_source\": {\n        \"foo\": \"bar\"\n      }\n    },\n    {\n      \"_id\": 456,\n      \"_index\": \"my-index\",\n      \"_source\": {\n        \"foo\": \"rab\"\n      }\n    }\n  ]\n}"'

Request examples

In this example the index `my-index` has a default pipeline called `my-pipeline` and a final pipeline called `my-final-pipeline`. Since both documents are being ingested into `my-index`, both pipelines are run using the pipeline definitions that are already in the system.

{
  "docs": [
    {
      "_id": 123,
      "_index": "my-index",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_id": 456,
      "_index": "my-index",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}

In this example the index `my-index` has a default pipeline called `my-pipeline` and a final pipeline called `my-final-pipeline`. But a substitute definition of `my-pipeline` is provided in `pipeline_substitutions`. The substitute `my-pipeline` will be used in place of the `my-pipeline` that is in the system, and then the `my-final-pipeline` that is already defined in the system will run.

{
  "docs": [
    {
      "_index": "my-index",
      "_id": 123,
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "my-index",
      "_id": 456,
      "_source": {
        "foo": "rab"
      }
    }
  ],
  "pipeline_substitutions": {
    "my-pipeline": {
      "processors": [
        {
          "uppercase": {
            "field": "foo"
          }
        }
      ]
    }
  }
}

In this example, imagine that the index `my-index` has a strict mapping with only the `foo` keyword field defined. Say that field mapping came from a component template named `my-mappings-template`. You want to test adding a new field, `bar`. So a substitute definition of `my-mappings-template` is provided in `component_template_substitutions`. The substitute `my-mappings-template` will be used in place of the existing mapping for `my-index` and in place of the `my-mappings-template` that is in the system.

{
  "docs": [
    {
      "_index": "my-index",
      "_id": "123",
      "_source": {
        "foo": "foo"
      }
    },
    {
      "_index": "my-index",
      "_id": "456",
      "_source": {
        "bar": "rab"
      }
    }
  ],
  "component_template_substitutions": {
    "my-mappings_template": {
      "template": {
        "mappings": {
          "dynamic": "strict",
          "properties": {
            "foo": {
              "type": "keyword"
            },
            "bar": {
              "type": "keyword"
            }
          }
        }
      }
    }
  }
}

The pipeline, component template, and index template substitutions replace the existing pipeline details for the duration of this request.

{
  "docs": [
    {
      "_id": "id",
      "_index": "my-index",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_id": "id",
      "_index": "my-index",
      "_source": {
        "foo": "rab"
      }
    }
  ],
  "pipeline_substitutions": {
    "my-pipeline": {
      "processors": [
        {
          "set": {
            "field": "field3",
            "value": "value3"
          }
        }
      ]
    }
  },
  "component_template_substitutions": {
    "my-component-template": {
      "template": {
        "mappings": {
          "dynamic": true,
          "properties": {
            "field3": {
              "type": "keyword"
            }
          }
        },
        "settings": {
          "index": {
            "default_pipeline": "my-pipeline"
          }
        }
      }
    }
  },
  "index_template_substitutions": {
    "my-index-template": {
      "index_patterns": [
        "my-index-*"
      ],
      "composed_of": [
        "component_template_1",
        "component_template_2"
      ]
    }
  },
  "mapping_addition": {
    "dynamic": "strict",
    "properties": {
      "foo": {
        "type": "keyword"
      }
    }
  }
}

Response examples (200)

A successful response when the simulation uses pipeline definitions that are already in the system.

{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "bar"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": "-3,",
      "_source": {
        "field1": "value1",
        "field2": "value2",
        "foo": "rab"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}

A successful response when the simulation uses pipeline substitutions.

{
  "docs": [
    {
      "doc": null,
      "_id": 123,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "BAR"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    },
    {
      "doc": null,
      "_id": 456,
      "_index": "my-index",
      "_version": -3,
      "_source": {
        "field2": "value2",
        "foo": "RAB"
      },
      "executed_pipelines": [
        "my-pipeline",
        "my-final-pipeline"
      ]
    }
  ]
}

A successful response when the simulation uses pipeline substitutions.

{
  "docs": [
    {
      "doc": {
        "_id": "123",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "foo": "foo"
        },
        "executed_pipelines": []
      }
    },
    {
      "doc": {
        "_id": "456",
        "_index": "my-index",
        "_version": -3,
        "_source": {
          "bar": "rab"
        },
      "executed_pipelines": []
      }
    }
  ]
}

Get data frame analytics job configuration info Added in 7.3.0

GET /_ml/data_frame/analytics/{id}

Api key auth Basic auth Bearer auth

You can get information for multiple data frame analytics jobs in a single API request by using a comma-separated list of data frame analytics jobs or a wildcard expression.

Path parameters

id string Required

Identifier for the data frame analytics job. If you do not specify this option, the API returns information for the first hundred data frame analytics jobs.

Query parameters

allow_no_match boolean
Specifies what to do when the request:
1. Contains wildcard expressions and there are no data frame analytics jobs that match.
2. Contains the _all string or no identifiers and there are no matches.
3. Contains wildcard expressions and there are only partial matches.
The default value returns an empty data_frame_analytics array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.
from number

Skips the specified number of data frame analytics jobs.
size number

Specifies the maximum number of data frame analytics jobs to obtain.
exclude_generated boolean

Indicates if certain fields should be removed from the configuration on retrieval. This allows the configuration to be in an acceptable format to be retrieved and then added to another cluster.

Responses

200 application/json
Hide response attributes Show response attributes object
- count number Required
- data_frame_analytics array[object] Required
  
  An array of data frame analytics job resources, which are sorted by the id value in ascending order.
  
  Hide data_frame_analytics attributes Show data_frame_analytics attributes object
  
  allow_lazy_start boolean
  
  analysis object Required
  
  Hide analysis attributes Show analysis attributes object
  
  classification object
  
  Hide classification attributes Show classification attributes object
  
  alpha number
  
  Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.
  
  dependent_variable string Required
  
  Defines which field of the document is to be predicted. It must match one of the fields in the index being used to train. If this field is missing from a document, then that document will not be used for training, but a prediction with the trained model will be generated for it. It is also known as continuous target variable. For classification analysis, the data type of the field must be numeric (integer, short, long, byte), categorical (ip or keyword), or boolean. There must be no more than 30 different values in this field. For regression analysis, the data type of the field must be numeric.
  
  downsample_factor number
  
  Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.
  
  early_stopping_enabled boolean
  
  Advanced configuration option. Specifies whether the training process should finish if it is not finding any better performing models. If disabled, the training process can take significantly longer and the chance of finding a better performing model is unremarkable.
  
  eta number
  
  Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.
  
  eta_growth_rate_per_tree number
  
  Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.
  
  feature_bag_fraction number
  
  Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.
  
  feature_processors array[object]
  
  Advanced configuration option. A collection of feature preprocessors that modify one or more included fields. The analysis uses the resulting one or more features instead of the original document field. However, these features are ephemeral; they are not stored in the destination index. Multiple feature_processors entries can refer to the same document fields. Automatic categorical feature encoding still occurs for the fields that are unprocessed by a custom processor or that have categorical values. Use this property only if you want to override the automatic feature encoding of the specified fields.
  
  gamma number
  
  Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.
  
  lambda number
  
  Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.
  
  max_optimization_rounds_per_hyperparameter number
  
  Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.
  
  max_trees number
  
  Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.
  
  num_top_feature_importance_values number
  
  Advanced configuration option. Specifies the maximum number of feature importance values per document to return. By default, no feature importance calculation occurs.
  
  prediction_field_name string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  randomize_seed number
  
  Defines the seed for the random generator that is used to pick training data. By default, it is randomly generated. Set it to a specific value to use the same training data each time you start a job (assuming other related parameters such as source and analyzed_fields are the same).
  
  soft_tree_depth_limit number
  
  Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.
  
  soft_tree_depth_tolerance number
  
  Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.
  
  training_percent
  
  class_assignment_objective string
  
  num_top_classes number
  
  Defines the number of categories for which the predicted probabilities are reported. It must be non-negative or -1. If it is -1 or greater than the total number of categories, probabilities are reported for all categories; if you have a large number of categories, there could be a significant effect on the size of your destination index. NOTE: To use the AUC ROC evaluation method, num_top_classes must be set to -1 or a value greater than or equal to the total number of categories.
  
  outlier_detection object
  
  Hide outlier_detection attributes Show outlier_detection attributes object
  
  compute_feature_influence boolean
  
  Specifies whether the feature influence calculation is enabled.
  
  feature_influence_threshold number
  
  The minimum outlier score that a document needs to have in order to calculate its feature influence score. Value range: 0-1.
  
  method string
  
  The method that outlier detection uses. Available methods are lof, ldof, distance_kth_nn, distance_knn, and ensemble. The default value is ensemble, which means that outlier detection uses an ensemble of different methods and normalises and combines their individual outlier scores to obtain the overall outlier score.
  
  n_neighbors number
  
  Defines the value for how many nearest neighbors each method of outlier detection uses to calculate its outlier score. When the value is not set, different values are used for different ensemble members. This default behavior helps improve the diversity in the ensemble; only override it if you are confident that the value you choose is appropriate for the data set.
  
  outlier_fraction number
  
  The proportion of the data set that is assumed to be outlying prior to outlier detection. For example, 0.05 means it is assumed that 5% of values are real outliers and 95% are inliers.
  
  standardization_enabled boolean
  
  If true, the following operation is performed on the columns before computing outlier scores: (x_i - mean(x_i)) / sd(x_i).
  
  regression object
  
  Hide regression attributes Show regression attributes object
  
  alpha number
  
  Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.
  
  dependent_variable string Required
  
  Defines which field of the document is to be predicted. It must match one of the fields in the index being used to train. If this field is missing from a document, then that document will not be used for training, but a prediction with the trained model will be generated for it. It is also known as continuous target variable. For classification analysis, the data type of the field must be numeric (integer, short, long, byte), categorical (ip or keyword), or boolean. There must be no more than 30 different values in this field. For regression analysis, the data type of the field must be numeric.
  
  downsample_factor number
  
  Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.
  
  early_stopping_enabled boolean
  
  Advanced configuration option. Specifies whether the training process should finish if it is not finding any better performing models. If disabled, the training process can take significantly longer and the chance of finding a better performing model is unremarkable.
  
  eta number
  
  Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.
  
  eta_growth_rate_per_tree number
  
  Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.
  
  feature_bag_fraction number
  
  Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.
  
  feature_processors array[object]
  
  Advanced configuration option. A collection of feature preprocessors that modify one or more included fields. The analysis uses the resulting one or more features instead of the original document field. However, these features are ephemeral; they are not stored in the destination index. Multiple feature_processors entries can refer to the same document fields. Automatic categorical feature encoding still occurs for the fields that are unprocessed by a custom processor or that have categorical values. Use this property only if you want to override the automatic feature encoding of the specified fields.
  
  gamma number
  
  Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.
  
  lambda number
  
  Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.
  
  max_optimization_rounds_per_hyperparameter number
  
  Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.
  
  max_trees number
  
  Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.
  
  num_top_feature_importance_values number
  
  Advanced configuration option. Specifies the maximum number of feature importance values per document to return. By default, no feature importance calculation occurs.
  
  prediction_field_name string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  randomize_seed number
  
  Defines the seed for the random generator that is used to pick training data. By default, it is randomly generated. Set it to a specific value to use the same training data each time you start a job (assuming other related parameters such as source and analyzed_fields are the same).
  
  soft_tree_depth_limit number
  
  Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.
  
  soft_tree_depth_tolerance number
  
  Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.
  
  training_percent
  
  loss_function string
  
  The loss function used during regression. Available options are mse (mean squared error), msle (mean squared logarithmic error), huber (Pseudo-Huber loss).
  
  loss_function_parameter number
  
  A positive number that is used as a parameter to the loss_function.
  
  analyzed_fields object
  
  Hide analyzed_fields attributes Show analyzed_fields attributes object
  
  includes array[string]
  
  An array of strings that defines the fields that will be excluded from the analysis. You do not need to add fields with unsupported data types to excludes, these fields are excluded from the analysis automatically.
  
  excludes array[string]
  
  An array of strings that defines the fields that will be included in the analysis.
  
  authorization object
  
  Hide authorization attributes Show authorization attributes object
  
  api_key object
  
  Hide api_key attributes Show api_key attributes object
  
  id string Required
  
  The identifier for the API key.
  
  name string Required
  
  The name of the API key.
  
  roles array[string]
  
  If a user ID was used for the most recent update to the job, its roles at the time of the update are listed in the response.
  
  service_account string
  
  If a service account was used for the most recent update to the job, the account name is listed in the response.
  
  create_time number
  
  Time unit for milliseconds
  
  description string
  
  dest object Required
  
  Hide dest attributes Show dest attributes object
  
  index string Required
  
  results_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  id string Required
  
  max_num_threads number
  
  model_memory_limit string
  
  source object Required
  
  Hide source attributes Show source attributes object
  
  index string | array[string] Required
  
  runtime_mappings object
  
  Hide runtime_mappings attribute Show runtime_mappings attribute object
  
  * object Additional properties
  
  Hide * attributes Show * attributes object
  
  fields object
  
  For type composite
  
  fetch_fields array[object]
  
  For type lookup
  
  format string
  
  A custom format for date type runtime fields.
  
  input_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_index string
  
  script object
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, geo_shape, ip, keyword, long, or lookup.
  
  _source object
  
  Hide _source attributes Show _source attributes object
  
  includes array[string]
  
  An array of strings that defines the fields that will be excluded from the analysis. You do not need to add fields with unsupported data types to excludes, these fields are excluded from the analysis automatically.
  
  excludes array[string]
  
  An array of strings that defines the fields that will be included in the analysis.
  
  query object
  
  The Elasticsearch query domain-specific language (DSL). This value corresponds to the query object in an Elasticsearch search POST body. All the options that are supported by Elasticsearch can be used, as this object is passed verbatim to Elasticsearch. By default, this property has the following value: {"match_all": {}}.
  
  Query DSL
  
  version string
  
  _meta object
  
  Hide _meta attribute Show _meta attribute object
  
  * object Additional properties

GET /_ml/data_frame/analytics/{id}

curl \
 --request GET 'http://api.example.com/_ml/data_frame/analytics/{id}' \
 --header "Authorization: $API_KEY"

Response examples (200)

{
  "count": 42.0,
  "data_frame_analytics": [
    {
      "allow_lazy_start": true,
      "analysis": {
        "classification": {
          "alpha": 42.0,
          "dependent_variable": "string",
          "downsample_factor": 42.0,
          "early_stopping_enabled": true,
          "eta": 42.0,
          "eta_growth_rate_per_tree": 42.0,
          "feature_bag_fraction": 42.0,
          "feature_processors": [
            {}
          ],
          "gamma": 42.0,
          "lambda": 42.0,
          "max_optimization_rounds_per_hyperparameter": 42.0,
          "max_trees": 42.0,
          "num_top_feature_importance_values": 42.0,
          "prediction_field_name": "string",
          "randomize_seed": 42.0,
          "soft_tree_depth_limit": 42.0,
          "soft_tree_depth_tolerance": 42.0,
          "class_assignment_objective": "string",
          "num_top_classes": 42.0
        },
        "outlier_detection": {
          "compute_feature_influence": true,
          "feature_influence_threshold": 42.0,
          "method": "string",
          "n_neighbors": 42.0,
          "outlier_fraction": 42.0,
          "standardization_enabled": true
        },
        "regression": {
          "alpha": 42.0,
          "dependent_variable": "string",
          "downsample_factor": 42.0,
          "early_stopping_enabled": true,
          "eta": 42.0,
          "eta_growth_rate_per_tree": 42.0,
          "feature_bag_fraction": 42.0,
          "feature_processors": [
            {}
          ],
          "gamma": 42.0,
          "lambda": 42.0,
          "max_optimization_rounds_per_hyperparameter": 42.0,
          "max_trees": 42.0,
          "num_top_feature_importance_values": 42.0,
          "prediction_field_name": "string",
          "randomize_seed": 42.0,
          "soft_tree_depth_limit": 42.0,
          "soft_tree_depth_tolerance": 42.0,
          "loss_function": "string",
          "loss_function_parameter": 42.0
        }
      },
      "analyzed_fields": {
        "includes": [
          "string"
        ],
        "excludes": [
          "string"
        ]
      },
      "authorization": {
        "api_key": {
          "id": "string",
          "name": "string"
        },
        "roles": [
          "string"
        ],
        "service_account": "string"
      },
      "": 42.0,
      "description": "string",
      "dest": {
        "index": "string",
        "results_field": "string"
      },
      "id": "string",
      "max_num_threads": 42.0,
      "model_memory_limit": "string",
      "source": {
        "index": "string",
        "runtime_mappings": {
          "additionalProperty1": {
            "fields": {},
            "fetch_fields": [
              {}
            ],
            "format": "string",
            "input_field": "string",
            "target_field": "string",
            "target_index": "string",
            "script": {},
            "type": "boolean"
          },
          "additionalProperty2": {
            "fields": {},
            "fetch_fields": [
              {}
            ],
            "format": "string",
            "input_field": "string",
            "target_field": "string",
            "target_index": "string",
            "script": {},
            "type": "boolean"
          }
        },
        "_source": {
          "includes": [
            "string"
          ],
          "excludes": [
            "string"
          ]
        },
        "query": {}
      },
      "version": "string",
      "_meta": {
        "additionalProperty1": {},
        "additionalProperty2": {}
      }
    }
  ]
}

Search rolled-up data Deprecated Technical preview

GET /{index}/_rollup_search

Api key auth Basic auth Bearer auth

The rollup search endpoint is needed because, internally, rolled-up documents utilize a different document structure than the original data. It rewrites standard Query DSL into a format that matches the rollup documents then takes the response and rewrites it back to what a client would expect given the original query.

The request body supports a subset of features from the regular search API. The following functionality is not available:

size: Because rollups work on pre-aggregated data, no search hits can be returned and so size must be set to zero or omitted entirely. highlighter, suggestors, post_filter, profile, explain: These are similarly disallowed.

Searching both historical rollup and non-rollup data

The rollup search API has the capability to search across both "live" non-rollup data and the aggregated rollup data. This is done by simply adding the live indices to the URI. For example:

GET sensor-1,sensor_rollup/_rollup_search
{
  "size": 0,
  "aggregations": {
     "max_temperature": {
      "max": {
        "field": "temperature"
      }
    }
  }
}

The rollup search endpoint does two things when the search runs:

The original request is sent to the non-rollup index unaltered.
A rewritten version of the original request is sent to the rollup index.

When the two responses are received, the endpoint rewrites the rollup response and merges the two together. During the merging process, if there is any overlap in buckets between the two responses, the buckets from the non-rollup index are used.

Path parameters

index string | array[string] Required
A comma-separated list of data streams and indices used to limit the request. This parameter has the following rules:
- At least one data stream, index, or wildcard expression must be specified. This target can include a rollup or non-rollup index. For data streams, the stream's backing indices can only serve as non-rollup indices. Omitting the parameter or using _all are not permitted.
- Multiple non-rollup indices may be specified.
- Only one rollup index may be specified. If more than one are supplied, an exception occurs.
- Wildcard expressions (*) may be used. If they match more than one rollup index, an exception occurs. However, you can use an expression to match multiple non-rollup indices or data streams.

Query parameters

rest_total_hits_as_int boolean

Indicates whether hits.total should be rendered as an integer or an object in the rest search response
typed_keys boolean

Specify whether aggregation and suggester names should be prefixed by their respective types in the response

application/json

Body Required

aggregations object

Specifies aggregations.

External documentation
query object

An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

External documentation
size number

Must be zero if set, as rollups work on pre-aggregated data.

Responses

200 application/json
Hide response attributes Show response attributes object
- took number Required
- timed_out boolean Required
- terminated_early boolean
- _shards object Required
  
  Hide _shards attributes Show _shards attributes object
  
  failed number Required
  
  successful number Required
  
  total number Required
  
  failures array[object]
  
  Hide failures attributes Show failures attributes object
  
  index string
  
  node string
  
  reason object Required
  
  Hide reason attributes Show reason attributes object
  
  type string Required
  
  The type of error
  
  reason string
  
  A human-readable explanation of the error, in English.
  
  stack_trace string
  
  The server stack trace. Present only if the error_trace=true parameter was sent with the request.
  
  caused_by object
  
  root_cause array[object]
  
  suppressed array[object]
  
  shard number Required
  
  status string
  
  skipped number
- hits object Required
  
  Hide hits attributes Show hits attributes object
  
  total object | number
  
  Total hit count information, present only if track_total_hits wasn't false in the search request.
  
  One of:
  TotalHits object number-2 number
  
  Hide attributes Show attributes
  
  relation string Required
  
  Values are eq or gte.
  
  value number Required
  
  hits array[object] Required
  
  Hide hits attributes Show hits attributes object
  
  _index string Required
  
  _id string
  
  _score number | string | null
  
  One of:
  number-1 number string-2 string | null
  
  _explanation object
  
  Hide _explanation attributes Show _explanation attributes object
  
  description string Required
  
  details array[object] Required
  
  value number Required
  
  fields object
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  highlight object
  
  Hide highlight attribute Show highlight attribute object
  
  * array[string] Additional properties
  
  inner_hits object
  
  Hide inner_hits attribute Show inner_hits attribute object
  
  * object Additional properties
  
  Hide * attribute Show * attribute object
  
  hits object Required
  
  matched_queries array[string] | object
  
  One of:
  array-1 array[string] object-2 object
  
  _nested object
  
  Hide _nested attributes Show _nested attributes object
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  offset number Required
  
  _nested object
  
  _ignored array[string]
  
  ignored_field_values object
  
  Hide ignored_field_values attribute Show ignored_field_values attribute object
  
  * array[object] Additional properties
  
  _shard string
  
  _node string
  
  _routing string
  
  _source object
  
  _rank number
  
  _seq_no number
  
  _primary_term number
  
  _version number
  
  sort array[number | string | boolean | null]
  
  A field value.
  
  max_score number | string | null
  
  One of:
  number-1 number string-2 string | null
- aggregations object

GET /{index}/_rollup_search

curl \
 --request GET 'http://api.example.com/{index}/_rollup_search' \
 --header "Authorization: $API_KEY" \
 --header "Content-Type: application/json" \
 --data '"{\n  \"size\": 0,\n  \"aggregations\": {\n    \"max_temperature\": {\n      \"max\": {\n        \"field\": \"temperature\"\n      }\n    }\n  }\n}"'

Request example

Search rolled up data stored in `sensor_rollup` with `GET /sensor_rollup/_rollup_search`

{
  "size": 0,
  "aggregations": {
    "max_temperature": {
      "max": {
        "field": "temperature"
      }
    }
  }
}

Response examples (200)

An abbreviated response from `GET /sensor_rollup/_rollup_search` with a `max` aggregation on a `temperature` field. The response provides some metadata about the request (`took`, `_shards`), the search hits (which is always empty for rollup searches), and the aggregation response.

{
  "took" : 102,
  "timed_out" : false,
  "terminated_early" : false,
  "_shards" : {} ,
  "hits" : {
    "total" : {
        "value": 0,
        "relation": "eq"
    },
    "max_score" : 0.0,
    "hits" : [ ]
  },
  "aggregations" : {
    "max_temperature" : {
      "value" : 202.0
    }
  }
}

Run a retention policy Added in 7.5.0

POST /_slm/_execute_retention

Api key auth Basic auth Bearer auth

Manually apply the retention policy to force immediate removal of snapshots that are expired according to the snapshot lifecycle policy retention rules. The retention policy is normally applied according to its schedule.

Query parameters

master_timeout string

The period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.
timeout string

The period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Responses

200 application/json
Hide response attribute Show response attribute object
- acknowledged boolean Required
  
  For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_slm/_execute_retention

curl \
 --request POST 'http://api.example.com/_slm/_execute_retention' \
 --header "Authorization: $API_KEY"

Response examples (200)

{
  "acknowledged": true
}

start_time string | number

stop_time string | number

files_percent string | number

bytes_percent string | number

translog_ops_percent string | number

Body Required

source string | object

lang string

Validate a query Added in 1.3.0

Get lifecycle policies Added in 6.6.0

modified_date string | number Required

Simulate data ingestion Technical preview

Body Required

routing_path string | array[string]

number_of_shards number | string

number_of_replicas number | string

routing_partition_size number | string

hidden boolean | string

auto_expand_replicas string | null

verified_before_close boolean | string

format string | number

priority number | string

routing_path string | array[string]

order string | array[string]

mode string | array[string]

missing string | array[string]

number_of_shards number | string

number_of_replicas number | string

routing_partition_size number | string

hidden boolean | string

auto_expand_replicas string | null

read_only boolean | string

read_only_allow_delete boolean | string

read boolean | string

write boolean | string

metadata boolean | string

max_token_count number | string

indexing_complete boolean | string

prefer_ilm boolean | string

creation_date number | string

creation_date_string string | number

verified_before_close boolean | string

format string | number

flush_threshold_size number | string

lenient boolean | string Required

priority number | string

ignore_malformed boolean | string

type string Required

source string | object

lang string

_version number | string Required

Get data frame analytics job configuration info Added in 7.3.0

Search rolled-up data Deprecated Technical preview

Body Required

total object | number

_score number | string | null

matched_queries array[string] | object

max_score number | string | null

Run a retention policy Added in 7.5.0