Update the connector configuration Beta

PUT /_connector/{connector_id}/_configuration

Update the configuration field in the connector document.

Path parameters

  • connector_id string Required

    The unique identifier of the connector to be updated

application/json

Body Required

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • result string Required

      Values are created, updated, deleted, not_found, or noop.

PUT /_connector/{connector_id}/_configuration
curl \
 --request PUT http://api.example.com/_connector/{connector_id}/_configuration \
 --header "Content-Type: application/json" \
 --data '"{\n    \"values\": {\n        \"tenant_id\": \"my-tenant-id\",\n        \"tenant_name\": \"my-sharepoint-site\",\n        \"client_id\": \"foo\",\n        \"secret_value\": \"bar\",\n        \"site_collections\": \"*\"\n    }\n}"'
{
    "values": {
        "tenant_id": "my-tenant-id",
        "tenant_name": "my-sharepoint-site",
        "client_id": "foo",
        "secret_value": "bar",
        "site_collections": "*"
    }
}
{
    "values": {
        "secret_value": "foo-bar"
    }
}
Response examples (200)
{
  "result": "updated"
}

































































Get data streams Added in 7.9.0

GET /_data_stream

Get information about one or more data streams.

Query parameters

  • expand_wildcards string | array[string]

    Type of data stream that wildcard patterns can match. Supports comma-separated values, such as open,hidden.

  • If true, returns all relevant default configurations for the index template.

  • Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • verbose boolean

    Whether the maximum timestamp for each data stream should be calculated and returned.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • data_streams array[object] Required
      Hide data_streams attributes Show data_streams attributes object
      • _meta object
        Hide _meta attribute Show _meta attribute object
        • * object Additional properties

          Additional properties are allowed.

      • If true, the data stream allows custom routing on write request.

      • Additional properties are allowed.

        Hide failure_store attributes Show failure_store attributes object
      • generation number Required

        Current generation for the data stream. This number acts as a cumulative count of the stream’s rollovers, starting at 1.

      • hidden boolean Required

        If true, the data stream is hidden.

      • Values are Index Lifecycle Management, Data stream lifecycle, or Unmanaged.

      • prefer_ilm boolean Required

        Indicates if ILM should take precedence over DSL in case both are configured to managed this data stream.

      • indices array[object] Required

        Array of objects containing information about the data stream’s backing indices. The last item in this array contains information about the stream’s current write index.

        Hide indices attributes Show indices attributes object
      • Additional properties are allowed.

        Hide lifecycle attributes Show lifecycle attributes object
        • A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.

        • Additional properties are allowed.

          Hide downsampling attribute Show downsampling attribute object
          • rounds array[object] Required

            The list of downsampling rounds to execute as part of this downsampling configuration

        • enabled boolean

          If defined, it turns data stream lifecycle on/off (true/false) for this data stream. A data stream lifecycle that's disabled (enabled: false) will have no effect on the data stream.

        • rollover object

          Additional properties are allowed.

          Hide rollover attributes Show rollover attributes object
      • name string Required
      • replicated boolean

        If true, the data stream is created and managed by cross-cluster replication and the local cluster can not write into this data stream or change its mappings.

      • rollover_on_write boolean Required

        If true, the next write to this data stream will trigger a rollover first and the document will be indexed in the new backing index. If the rollover fails the indexing request will fail too.

      • status string Required

        Values are green, GREEN, yellow, YELLOW, red, or RED.

      • system boolean

        If true, the data stream is created and managed by an Elastic stack component and cannot be modified through normal user interaction.

      • template string Required
      • timestamp_field object Required

        Additional properties are allowed.

        Hide timestamp_field attribute Show timestamp_field attribute object
        • name string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

GET /_data_stream
curl \
 --request GET http://api.example.com/_data_stream
Response examples (200)
A successful response for retrieving information about a data stream.
{
  "data_streams": [
    {
      "name": "my-data-stream",
      "timestamp_field": {
        "name": "@timestamp"
      },
      "indices": [
        {
          "index_name": ".ds-my-data-stream-2099.03.07-000001",
          "index_uuid": "xCEhwsp8Tey0-FLNFYVwSg",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        },
        {
          "index_name": ".ds-my-data-stream-2099.03.08-000002",
          "index_uuid": "PA_JquKGSiKcAKBA8DJ5gw",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        }
      ],
      "generation": 2,
      "_meta": {
        "my-meta-field": "foo"
      },
      "status": "GREEN",
      "next_generation_managed_by": "Index Lifecycle Management",
      "prefer_ilm": true,
      "template": "my-index-template",
      "ilm_policy": "my-lifecycle-policy",
      "hidden": false,
      "system": false,
      "allow_custom_routing": false,
      "replicated": false,
      "rollover_on_write": false
    },
    {
      "name": "my-data-stream-two",
      "timestamp_field": {
        "name": "@timestamp"
      },
      "indices": [
        {
          "index_name": ".ds-my-data-stream-two-2099.03.08-000001",
          "index_uuid": "3liBu2SYS5axasRt6fUIpA",
          "prefer_ilm": true,
          "ilm_policy": "my-lifecycle-policy",
          "managed_by": "Index Lifecycle Management"
        }
      ],
      "generation": 1,
      "_meta": {
        "my-meta-field": "foo"
      },
      "status": "YELLOW",
      "next_generation_managed_by": "Index Lifecycle Management",
      "prefer_ilm": true,
      "template": "my-index-template",
      "ilm_policy": "my-lifecycle-policy",
      "hidden": false,
      "system": false,
      "allow_custom_routing": false,
      "replicated": false,
      "rollover_on_write": false
    }
  ]
}




Update data streams Added in 7.16.0

POST /_data_stream/_modify

Performs one or more data stream modification actions in a single atomic operation.

application/json

Body Required

  • actions array[object] Required

    Actions to perform.

    Hide actions attributes Show actions attributes object

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

POST /_data_stream/_modify
curl \
 --request POST http://api.example.com/_data_stream/_modify \
 --header "Content-Type: application/json" \
 --data '{"actions":[{"add_backing_index":{"data_stream":"string","index":"string"},"remove_backing_index":{"data_stream":"string","index":"string"}}]}'



































































































































































ES|QL

The Elasticsearch Query Language (ES|QL) provides a powerful way to filter, transform, and analyze data stored in Elasticsearch, and in the future in other runtimes.































Check component templates Added in 7.8.0

HEAD /_component_template/{name}

Returns information about whether a particular component template exists.

Path parameters

  • name string | array[string] Required

    Comma-separated list of component template names used to limit the request. Wildcard (*) expressions are supported.

Query parameters

  • Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • local boolean

    If true, the request retrieves information from the local node only. Defaults to false, which means information is retrieved from the master node.

Responses

HEAD /_component_template/{name}
curl \
 --request HEAD http://api.example.com/_component_template/{name}
































Delete indices

DELETE /{index}

Deleting an index deletes its documents, shards, and metadata. It does not delete related Kibana components, such as data views, visualizations, or dashboards.

You cannot delete the current write index of a data stream. To delete the index, you must roll over the data stream so a new write index is created. You can then use the delete index API to delete the previous write index.

Path parameters

  • index string | array[string] Required

    Comma-separated list of indices to delete. You cannot specify index aliases. By default, this parameter does not support wildcards (*) or _all. To use wildcards or _all, set the action.destructive_requires_name cluster setting to false.

Query parameters

  • If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

  • If false, the request returns an error if it targets a missing or closed index.

  • Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

    • _shards object

      Additional properties are allowed.

      Hide _shards attributes Show _shards attributes object
DELETE /{index}
curl \
 --request DELETE http://api.example.com/{index}
























































































Update field mappings

POST /{index}/_mapping

Add new fields to an existing data stream or index. You can also use this API to change the search settings of existing fields and add new properties to existing object fields. For data streams, these changes are applied to all backing indices by default.

Add multi-fields to an existing field

Multi-fields let you index the same field in different ways. You can use this API to update the fields mapping parameter and enable multi-fields for an existing field. WARNING: If an index (or data stream) contains documents when you add a multi-field, those documents will not have values for the new multi-field. You can populate the new multi-field with the update by query API.

Change supported mapping parameters for an existing field

The documentation for each mapping parameter indicates whether you can update it for an existing field using this API. For example, you can use the update mapping API to update the ignore_above parameter.

Change the mapping of an existing field

Except for supported mapping parameters, you can't change the mapping or field type of an existing field. Changing an existing field could invalidate data that's already indexed.

If you need to change the mapping of a field in a data stream's backing indices, refer to documentation about modifying data streams. If you need to change the mapping of a field in other indices, create a new index with the correct mapping and reindex your data into that index.

Rename a field

Renaming a field would invalidate data already indexed under the old field name. Instead, add an alias field to create an alternate field name.

Path parameters

  • index string | array[string] Required

    A comma-separated list of index names the mapping should be added to (supports wildcards); use _all or omit to add the mapping on all indices.

Query parameters

  • If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

  • If false, the request returns an error if it targets a missing or closed index.

  • Period to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error.

  • timeout string

    Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.

  • If true, the mappings are applied only to the current write index for the target.

application/json

Body Required

  • Controls whether dynamic date detection is enabled.

  • dynamic string

    Values are strict, runtime, true, or false.

  • If date detection is enabled then new string fields are checked against 'dynamic_date_formats' and if the value matches then a new date field is added instead of string.

  • dynamic_templates object | array[object]

    Specify dynamic templates for the mapping.

  • Additional properties are allowed.

    Hide _field_names attribute Show _field_names attribute object
  • _meta object
    Hide _meta attribute Show _meta attribute object
    • * object Additional properties

      Additional properties are allowed.

  • Automatically map strings into numeric data types for all fields.

  • Mapping for a field. For new fields, this mapping can include:

    • Field name
    • Field data type
    • Mapping parameters
  • _routing object

    Additional properties are allowed.

    Hide _routing attribute Show _routing attribute object
  • _source object

    Additional properties are allowed.

    Hide _source attributes Show _source attributes object
  • runtime object
    Hide runtime attribute Show runtime attribute object
    • * object Additional properties

      Additional properties are allowed.

      Hide * attributes Show * attributes object
      • fields object

        For type composite

        Hide fields attribute Show fields attribute object
        • * object Additional properties

          Additional properties are allowed.

          Hide * attribute Show * attribute object
          • type string Required

            Values are boolean, composite, date, double, geo_point, ip, keyword, long, or lookup.

      • fetch_fields array[object]

        For type lookup

        Hide fetch_fields attributes Show fetch_fields attributes object
        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • format string
      • format string

        A custom format for date type runtime fields.

      • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • script object

        Additional properties are allowed.

        Hide script attributes Show script attributes object
        • source string

          The script source.

        • id string
        • params object

          Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          Hide params attribute Show params attribute object
          • * object Additional properties

            Additional properties are allowed.

        • lang string

          Any of:

          Values are painless, expression, mustache, or java.

        • options object
          Hide options attribute Show options attribute object
          • * string Additional properties
      • type string Required

        Values are boolean, composite, date, double, geo_point, ip, keyword, long, or lookup.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • acknowledged boolean Required

      For a successful response, this value is always true. On failure, an exception is returned instead.

    • _shards object

      Additional properties are allowed.

      Hide _shards attributes Show _shards attributes object
POST /{index}/_mapping
curl \
 --request POST http://api.example.com/{index}/_mapping \
 --header "Content-Type: application/json" \
 --data '{"properties":{"user":{"properties":{"name":{"type":"keyword"}}}}}'
Request example
The update mapping API can be applied to multiple data streams or indices with a single request. For example, run `PUT /my-index-000001,my-index-000002/_mapping` to update mappings for the `my-index-000001` and `my-index-000002` indices at the same time.
{
  "properties": {
    "user": {
      "properties": {
        "name": {
          "type": "keyword"
        }
      }
    }
  }
}








































































Validate a query Added in 1.3.0

POST /_validate/query

Validates a query without running it.

Query parameters

  • If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices.

  • all_shards boolean

    If true, the validation is executed on all shards instead of one random shard per index.

  • analyzer string

    Analyzer to use for the query string. This parameter can only be used when the q query string parameter is specified.

  • If true, wildcard and prefix queries are analyzed.

  • The default operator for query string query: AND or OR.

    Values are and, AND, or, or OR.

  • df string

    Field to use as default where no field prefix is given in the query string. This parameter can only be used when the q query string parameter is specified.

  • expand_wildcards string | array[string]

    Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are: all, open, closed, hidden, none.

  • explain boolean

    If true, the response returns detailed information if an error has occurred.

  • If false, the request returns an error if it targets a missing or closed index.

  • lenient boolean

    If true, format-based query failures (such as providing text to a numeric field) in the query string will be ignored.

  • rewrite boolean

    If true, returns a more detailed explanation showing the actual Lucene query that will be executed.

  • q string

    Query in the Lucene query string syntax.

application/json

Body

  • query object

    An Elasticsearch Query DSL (Domain Specific Language) object that defines a query.

    Additional properties are allowed.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
POST /_validate/query
curl \
 --request POST http://api.example.com/_validate/query \
 --header "Content-Type: application/json" \
 --data '{"query":{}}'






















































































Simulate a pipeline Added in 5.0.0

GET /_ingest/pipeline/{id}/_simulate

Run an ingest pipeline against a set of provided documents. You can either specify an existing pipeline to use with the provided documents or supply a pipeline definition in the body of the request.

Path parameters

  • id string Required

    The pipeline to test. If you don't specify a pipeline in the request body, this parameter is required.

Query parameters

  • verbose boolean

    If true, the response includes output data for each processor in the executed pipeline.

application/json

Body Required

  • docs array[object] Required

    Sample documents to test in the pipeline.

    Hide docs attributes Show docs attributes object
    • _id string
    • _index string
    • _source object Required

      JSON body for the document.

      Additional properties are allowed.

  • pipeline object Additional properties

    Additional properties are allowed.

    Hide pipeline attributes Show pipeline attributes object
    • Description of the ingest pipeline.

    • on_failure array[object]

      Processors to run immediately after a processor failure.

      Hide on_failure attributes Show on_failure attributes object
      • append object

        Additional properties are allowed.

        Hide append attributes Show append attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • value object | array[object] Required

          The value to be appended. Supports template snippets.

          One of:

          Additional properties are allowed.

        • If false, the processor does not append values already present in the field.

      • Additional properties are allowed.

        Hide attachment attributes Show attachment attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true, the binary field will be removed from the document

        • Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

      • bytes object

        Additional properties are allowed.

        Hide bytes attributes Show bytes attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • circle object

        Additional properties are allowed.

        Hide circle attributes Show circle attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • error_distance number Required

          The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • shape_type string Required

          Values are geo_shape or shape.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide community_id attributes Show community_id attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • seed number

          Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

        • If true and any required fields are missing, the processor quietly exits without modifying the document.

      • convert object

        Additional properties are allowed.

        Hide convert attributes Show convert attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • type string Required

          Values are integer, long, double, float, boolean, ip, string, or auto.

      • csv object

        Additional properties are allowed.

        Hide csv attributes Show csv attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

          Additional properties are allowed.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • quote string

          Quote used in CSV, has to be single character string.

        • Separator used in CSV, has to be single character string.

        • target_fields string | array[string] Required
        • trim boolean

          Trim whitespaces in unquoted fields.

      • date object

        Additional properties are allowed.

        Hide date attributes Show date attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • formats array[string] Required

          An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • locale string

          The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • timezone string

          The timezone to use when parsing the date. Supports template snippets.

        • The format to use when writing the date to target_field. Must be a valid java time pattern.

      • Additional properties are allowed.

        Hide date_index_name attributes Show date_index_name attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • date_formats array[string]

          An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • date_rounding string Required

          How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

        • A prefix of the index name to be prepended before the printed date. Supports template snippets.

        • locale string

          The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

        • timezone string

          The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

      • dissect object

        Additional properties are allowed.

        Hide dissect attributes Show dissect attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • The character(s) that separate the appended fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to apply to the field.

      • Additional properties are allowed.

        Hide dot_expander attributes Show dot_expander attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • override boolean

          Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

        • path string

          The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

      • drop object

        Additional properties are allowed.

        Hide drop attributes Show drop attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • enrich object

        Additional properties are allowed.

        Hide enrich attributes Show enrich attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

        • override boolean

          If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • policy_name string Required

          The name of the enrich policy to use.

        • Values are intersects, disjoint, within, or contains.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • fail object

        Additional properties are allowed.

        Hide fail attributes Show fail attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • message string Required

          The error message thrown by the processor. Supports template snippets.

      • Additional properties are allowed.

        Hide fingerprint attributes Show fingerprint attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • fields string | array[string] Required
        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • salt string

          Salt value for the hash function.

        • method string

          Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

        • If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

      • foreach object

        Additional properties are allowed.

        Hide foreach attributes Show foreach attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true, the processor silently exits without changing the document if the field is null or missing.

        • processor object Required

          Additional properties are allowed.

      • Additional properties are allowed.

        Hide ip_location attributes Show ip_location attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found IP location data will be returned, even if the field contains an array.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the IP location lookup.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • geo_grid object

        Additional properties are allowed.

        Hide geo_grid attributes Show geo_grid attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          The field to interpret as a geo-tile.= The field format is determined by the tile_type.

        • tile_type string Required

          Values are geotile, geohex, or geohash.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • Values are geojson or wkt.

      • geoip object

        Additional properties are allowed.

        Hide geoip attributes Show geoip attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found geoip data will be returned, even if the field contains an array.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the geoip lookup.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • grok object

        Additional properties are allowed.

        Hide grok attributes Show grok attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • patterns array[string] Required

          An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

        • When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

      • gsub object

        Additional properties are allowed.

        Hide gsub attributes Show gsub attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to be replaced.

        • replacement string Required

          The string to replace the matching patterns with.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide html_strip attributes Show html_strip attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document,

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide inference attributes Show inference attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • model_id string Required
        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          Hide field_map attribute Show field_map attribute object
          • * object Additional properties

            Additional properties are allowed.

        • Additional properties are allowed.

          Hide inference_config attributes Show inference_config attributes object
      • join object

        Additional properties are allowed.

        Hide join attributes Show join attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • separator string Required

          The separator character.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • json object

        Additional properties are allowed.

        Hide json attributes Show json attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

        • Values are replace or merge.

        • When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • kv object

        Additional properties are allowed.

        Hide kv attributes Show kv attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • exclude_keys array[string]

          List of keys to exclude from document.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_split string Required

          Regex pattern to use for splitting key-value pairs.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • include_keys array[string]

          List of keys to filter and insert into document. Defaults to including all keys.

        • prefix string

          Prefix to be added to extracted keys.

        • If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • trim_key string

          String of characters to trim from extracted keys.

        • String of characters to trim from extracted values.

        • value_split string Required

          Regex pattern to use for splitting the key from the value within a key-value pair.

      • Additional properties are allowed.

        Hide lowercase attributes Show lowercase attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide network_direction attributes Show network_direction attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • internal_networks array[string]

          List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and any required fields are missing, the processor quietly exits without modifying the document.

      • pipeline object

        Additional properties are allowed.

        Hide pipeline attributes Show pipeline attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • name string Required
        • Whether to ignore missing pipelines instead of failing.

      • redact object

        Additional properties are allowed.

        Hide redact attributes Show redact attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • patterns array[string] Required

          A list of grok expressions to match and redact named captures with

        • Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • prefix string

          Start a redacted section with this token

        • suffix string

          End a redacted section with this token

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

        • If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

      • Additional properties are allowed.

        Hide registered_domain attributes Show registered_domain attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and any required fields are missing, the processor quietly exits without modifying the document.

      • remove object

        Additional properties are allowed.

        Hide remove attributes Show remove attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string | array[string] Required
        • keep string | array[string]
        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • rename object

        Additional properties are allowed.

        Hide rename attributes Show rename attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • reroute object

        Additional properties are allowed.

        Hide reroute attributes Show reroute attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • A static value for the target. Can’t be set when the dataset or namespace option is set.

        • dataset string | array[string]

          Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.dataset}}

        • namespace string | array[string]

          Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.namespace}}

      • script object

        Additional properties are allowed.

        Hide script attributes Show script attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • id string
        • lang string

          Script language.

        • params object

          Object containing parameters for the script.

          Hide params attribute Show params attribute object
          • * object Additional properties

            Additional properties are allowed.

        • source string

          Inline script. If no id is specified, this parameter is required.

      • set object

        Additional properties are allowed.

        Hide set attributes Show set attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

        • The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

        • override boolean

          If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • value object

          The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

          Additional properties are allowed.

      • Additional properties are allowed.

        Hide set_security_user attributes Show set_security_user attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what user related properties are added to the field.

      • sort object

        Additional properties are allowed.

        Hide sort attributes Show sort attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • order string

          Values are asc or desc.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • split object

        Additional properties are allowed.

        Hide split attributes Show split attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • Preserves empty trailing fields, if any.

        • separator string Required

          A regex which matches the separator, for example, , or \s+.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide terminate attributes Show terminate attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • trim object

        Additional properties are allowed.

        Hide trim attributes Show trim attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide uppercase attributes Show uppercase attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide urldecode attributes Show urldecode attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide uri_parts attributes Show uri_parts attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • If true, the processor copies the unparsed URI to <target_field>.original.

        • If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide user_agent attributes Show user_agent attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what properties are added to target_field.

          Values are name, os, device, original, or version.

        • Extracts device type from the user agent string on a best-effort basis.

    • processors array[object]

      Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.

      Hide processors attributes Show processors attributes object
      • append object

        Additional properties are allowed.

        Hide append attributes Show append attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • value object | array[object] Required

          The value to be appended. Supports template snippets.

          One of:

          Additional properties are allowed.

        • If false, the processor does not append values already present in the field.

      • Additional properties are allowed.

        Hide attachment attributes Show attachment attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • The number of chars being used for extraction to prevent huge fields. Use -1 for no limit.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Array of properties to select to be stored. Can be content, title, name, author, keywords, date, content_type, content_length, language.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true, the binary field will be removed from the document

        • Field containing the name of the resource to decode. If specified, the processor passes this resource name to the underlying Tika library to enable Resource Name Based Detection.

      • bytes object

        Additional properties are allowed.

        Hide bytes attributes Show bytes attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • circle object

        Additional properties are allowed.

        Hide circle attributes Show circle attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • error_distance number Required

          The difference between the resulting inscribed distance from center to side and the circle’s radius (measured in meters for geo_shape, unit-less for shape).

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • shape_type string Required

          Values are geo_shape or shape.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide community_id attributes Show community_id attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • seed number

          Seed for the community ID hash. Must be between 0 and 65535 (inclusive). The seed can prevent hash collisions between network domains, such as a staging and production network that use the same addressing scheme.

        • If true and any required fields are missing, the processor quietly exits without modifying the document.

      • convert object

        Additional properties are allowed.

        Hide convert attributes Show convert attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • type string Required

          Values are integer, long, double, float, boolean, ip, string, or auto.

      • csv object

        Additional properties are allowed.

        Hide csv attributes Show csv attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • Value used to fill empty fields. Empty fields are skipped if this is not provided. An empty field is one with no value (2 consecutive separators) or empty quotes ("").

          Additional properties are allowed.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • quote string

          Quote used in CSV, has to be single character string.

        • Separator used in CSV, has to be single character string.

        • target_fields string | array[string] Required
        • trim boolean

          Trim whitespaces in unquoted fields.

      • date object

        Additional properties are allowed.

        Hide date attributes Show date attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • formats array[string] Required

          An array of the expected date formats. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • locale string

          The locale to use when parsing the date, relevant when parsing month names or week days. Supports template snippets.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • timezone string

          The timezone to use when parsing the date. Supports template snippets.

        • The format to use when writing the date to target_field. Must be a valid java time pattern.

      • Additional properties are allowed.

        Hide date_index_name attributes Show date_index_name attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • date_formats array[string]

          An array of the expected date formats for parsing dates / timestamps in the document being preprocessed. Can be a java time pattern or one of the following formats: ISO8601, UNIX, UNIX_MS, or TAI64N.

        • date_rounding string Required

          How to round the date when formatting the date into the index name. Valid values are: y (year), M (month), w (week), d (day), h (hour), m (minute) and s (second). Supports template snippets.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • The format to be used when printing the parsed date into the index name. A valid java time pattern is expected here. Supports template snippets.

        • A prefix of the index name to be prepended before the printed date. Supports template snippets.

        • locale string

          The locale to use when parsing the date from the document being preprocessed, relevant when parsing month names or week days.

        • timezone string

          The timezone to use when parsing the date and when date math index supports resolves expressions into concrete index names.

      • dissect object

        Additional properties are allowed.

        Hide dissect attributes Show dissect attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • The character(s) that separate the appended fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to apply to the field.

      • Additional properties are allowed.

        Hide dot_expander attributes Show dot_expander attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • override boolean

          Controls the behavior when there is already an existing nested object that conflicts with the expanded field. When false, the processor will merge conflicts by combining the old and the new values into an array. When true, the value from the expanded field will overwrite the existing value.

        • path string

          The field that contains the field to expand. Only required if the field to expand is part another object field, because the field option can only understand leaf fields.

      • drop object

        Additional properties are allowed.

        Hide drop attributes Show drop attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • enrich object

        Additional properties are allowed.

        Hide enrich attributes Show enrich attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • The maximum number of matched documents to include under the configured target field. The target_field will be turned into a json array if max_matches is higher than 1, otherwise target_field will become a json object. In order to avoid documents getting too large, the maximum allowed value is 128.

        • override boolean

          If processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • policy_name string Required

          The name of the enrich policy to use.

        • Values are intersects, disjoint, within, or contains.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • fail object

        Additional properties are allowed.

        Hide fail attributes Show fail attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • message string Required

          The error message thrown by the processor. Supports template snippets.

      • Additional properties are allowed.

        Hide fingerprint attributes Show fingerprint attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • fields string | array[string] Required
        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • salt string

          Salt value for the hash function.

        • method string

          Values are MD5, SHA-1, SHA-256, SHA-512, or MurmurHash3.

        • If true, the processor ignores any missing fields. If all fields are missing, the processor silently exits without modifying the document.

      • foreach object

        Additional properties are allowed.

        Hide foreach attributes Show foreach attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true, the processor silently exits without changing the document if the field is null or missing.

        • processor object Required

          Additional properties are allowed.

      • Additional properties are allowed.

        Hide ip_location attributes Show ip_location attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found IP location data will be returned, even if the field contains an array.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the IP location lookup.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • geo_grid object

        Additional properties are allowed.

        Hide geo_grid attributes Show geo_grid attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          The field to interpret as a geo-tile.= The field format is determined by the tile_type.

        • tile_type string Required

          Values are geotile, geohex, or geohash.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • Values are geojson or wkt.

      • geoip object

        Additional properties are allowed.

        Hide geoip attributes Show geoip attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • The database filename referring to a database the module ships with (GeoLite2-City.mmdb, GeoLite2-Country.mmdb, or GeoLite2-ASN.mmdb) or a custom database in the ingest-geoip config directory.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • first_only boolean

          If true, only the first found geoip data will be returned, even if the field contains an array.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • properties array[string]

          Controls what properties are added to the target_field based on the geoip lookup.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true (and if ingest.geoip.downloader.eager.download is false), the missing database is downloaded when the pipeline is created. Else, the download is triggered by when the pipeline is used as the default_pipeline or final_pipeline in an index.

      • grok object

        Additional properties are allowed.

        Hide grok attributes Show grok attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • Must be disabled or v1. If v1, the processor uses patterns with Elastic Common Schema (ECS) field names.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • A map of pattern-name and pattern tuples defining custom patterns to be used by the current processor. Patterns matching existing names will override the pre-existing definition.

          Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • patterns array[string] Required

          An ordered list of grok expression to match and extract named captures with. Returns on the first expression in the list that matches.

        • When true, _ingest._grok_match_index will be inserted into your matched document’s metadata with the index into the pattern found in patterns that matched.

      • gsub object

        Additional properties are allowed.

        Hide gsub attributes Show gsub attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • pattern string Required

          The pattern to be replaced.

        • replacement string Required

          The string to replace the matching patterns with.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide html_strip attributes Show html_strip attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document,

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide inference attributes Show inference attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • model_id string Required
        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Maps the document field names to the known field names of the model. This mapping takes precedence over any default mappings provided in the model configuration.

          Hide field_map attribute Show field_map attribute object
          • * object Additional properties

            Additional properties are allowed.

        • Additional properties are allowed.

          Hide inference_config attributes Show inference_config attributes object
      • join object

        Additional properties are allowed.

        Hide join attributes Show join attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • separator string Required

          The separator character.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • json object

        Additional properties are allowed.

        Hide json attributes Show json attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • Flag that forces the parsed JSON to be added at the top level of the document. target_field must not be set when this option is chosen.

        • Values are replace or merge.

        • When set to true, the JSON parser will not fail if the JSON contains duplicate keys. Instead, the last encountered value for any duplicate key wins.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • kv object

        Additional properties are allowed.

        Hide kv attributes Show kv attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • exclude_keys array[string]

          List of keys to exclude from document.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field_split string Required

          Regex pattern to use for splitting key-value pairs.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • include_keys array[string]

          List of keys to filter and insert into document. Defaults to including all keys.

        • prefix string

          Prefix to be added to extracted keys.

        • If true. strip brackets (), <>, [] as well as quotes ' and " from extracted values.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • trim_key string

          String of characters to trim from extracted keys.

        • String of characters to trim from extracted values.

        • value_split string Required

          Regex pattern to use for splitting the key from the value within a key-value pair.

      • Additional properties are allowed.

        Hide lowercase attributes Show lowercase attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide network_direction attributes Show network_direction attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • internal_networks array[string]

          List of internal networks. Supports IPv4 and IPv6 addresses and ranges in CIDR notation. Also supports the named ranges listed below. These may be constructed with template snippets. Must specify only one of internal_networks or internal_networks_field.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and any required fields are missing, the processor quietly exits without modifying the document.

      • pipeline object

        Additional properties are allowed.

        Hide pipeline attributes Show pipeline attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • name string Required
        • Whether to ignore missing pipelines instead of failing.

      • redact object

        Additional properties are allowed.

        Hide redact attributes Show redact attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • patterns array[string] Required

          A list of grok expressions to match and redact named captures with

        • Hide pattern_definitions attribute Show pattern_definitions attribute object
          • * string Additional properties
        • prefix string

          Start a redacted section with this token

        • suffix string

          End a redacted section with this token

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • If true and the current license does not support running redact processors, then the processor quietly exits without modifying the document

        • If true then ingest metadata _ingest._redact._is_redacted is set to true if the document has been redacted

      • Additional properties are allowed.

        Hide registered_domain attributes Show registered_domain attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and any required fields are missing, the processor quietly exits without modifying the document.

      • remove object

        Additional properties are allowed.

        Hide remove attributes Show remove attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string | array[string] Required
        • keep string | array[string]
        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

      • rename object

        Additional properties are allowed.

        Hide rename attributes Show rename attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • target_field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • reroute object

        Additional properties are allowed.

        Hide reroute attributes Show reroute attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • A static value for the target. Can’t be set when the dataset or namespace option is set.

        • dataset string | array[string]

          Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain - and must be no longer than 100 characters. Example values are nginx.access and nginx.error.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.dataset}}

        • namespace string | array[string]

          Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters.

          Supports field references with a mustache-like syntax (denoted as {{double}} or {{{triple}}} curly braces). When resolving field references, the processor replaces invalid characters with _. Uses the part of the index name as a fallback if all field references resolve to a null, missing, or non-string value.

          default {{data_stream.namespace}}

      • script object

        Additional properties are allowed.

        Hide script attributes Show script attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • id string
        • lang string

          Script language.

        • params object

          Object containing parameters for the script.

          Hide params attribute Show params attribute object
          • * object Additional properties

            Additional properties are allowed.

        • source string

          Inline script. If no id is specified, this parameter is required.

      • set object

        Additional properties are allowed.

        Hide set attributes Show set attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and value is a template snippet that evaluates to null or the empty string, the processor quietly exits without modifying the document.

        • The media type for encoding value. Applies only when value is a template snippet. Must be one of application/json, text/plain, or application/x-www-form-urlencoded.

        • override boolean

          If true processor will update fields with pre-existing non-null-valued field. When set to false, such fields will not be touched.

        • value object

          The value to be set for the field. Supports template snippets. May specify only one of value or copy_from.

          Additional properties are allowed.

      • Additional properties are allowed.

        Hide set_security_user attributes Show set_security_user attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what user related properties are added to the field.

      • sort object

        Additional properties are allowed.

        Hide sort attributes Show sort attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • order string

          Values are asc or desc.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • split object

        Additional properties are allowed.

        Hide split attributes Show split attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • Preserves empty trailing fields, if any.

        • separator string Required

          A regex which matches the separator, for example, , or \s+.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide terminate attributes Show terminate attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

      • trim object

        Additional properties are allowed.

        Hide trim attributes Show trim attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide uppercase attributes Show uppercase attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide urldecode attributes Show urldecode attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist or is null, the processor quietly exits without modifying the document.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide uri_parts attributes Show uri_parts attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • If true, the processor copies the unparsed URI to <target_field>.original.

        • If true, the processor removes the field after parsing the URI string. If parsing fails, the processor does not remove the field.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

      • Additional properties are allowed.

        Hide user_agent attributes Show user_agent attributes object
        • Description of the processor. Useful for describing the purpose of the processor or its configuration.

        • if object

          Additional properties are allowed.

          Hide if attributes Show if attributes object
          • source string

            The script source.

          • id string
          • params object

            Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.

          • options object
        • Ignore failures for the processor.

        • on_failure array[object]

          Handle failures for the processor.

          Additional properties are allowed.

        • tag string

          Identifier for the processor. Useful for debugging and metrics.

        • field string Required

          Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • If true and field does not exist, the processor quietly exits without modifying the document.

        • The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with.

        • Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.

        • properties array[string]

          Controls what properties are added to target_field.

          Values are name, os, device, original, or version.

        • Extracts device type from the user agent string on a best-effort basis.

    • version number
    • deprecated boolean

      Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.

    • _meta object
      Hide _meta attribute Show _meta attribute object
      • * object Additional properties

        Additional properties are allowed.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
    • docs array[object] Required
      Hide docs attributes Show docs attributes object
      • doc object

        Additional properties are allowed.

        Hide doc attributes Show doc attributes object
        • _id string Required
        • _index string Required
        • _ingest object Required

          Additional properties are allowed.

          Hide _ingest attributes Show _ingest attributes object
        • _routing string

          Value used to send the document to a specific primary shard.

        • _source object Required

          JSON body for the document.

          Hide _source attribute Show _source attribute object
          • * object Additional properties

            Additional properties are allowed.

        • _version number | string

          Some APIs will return values such as numbers also as a string (notably epoch timestamps). This behavior is used to capture this behavior while keeping the semantics of the field type.

          Depending on the target language, code generators can keep the union or remove it and leniently parse strings to the target type.

        • Values are internal, external, external_gte, or force.

      • error object

        Additional properties are allowed.

        Hide error attributes Show error attributes object
        • type string Required

          The type of error

        • reason string

          A human-readable explanation of the error, in English.

        • The server stack trace. Present only if the error_trace=true parameter was sent with the request.

        • Additional properties are allowed.

        • root_cause array[object]

          Additional properties are allowed.

        • suppressed array[object]

          Additional properties are allowed.

      • processor_results array[object]
        Hide processor_results attributes Show processor_results attributes object
        • doc object

          Additional properties are allowed.

          Hide doc attributes Show doc attributes object
        • tag string
        • status string

          Values are success, failure, simulated, or throttled.

        • Additional properties are allowed.

          Hide ignored_error attributes Show ignored_error attributes object
          • type string Required

            The type of error

          • reason string

            A human-readable explanation of the error, in English.

          • The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • Additional properties are allowed.

          • root_cause array[object]
          • suppressed array[object]
        • error object

          Additional properties are allowed.

          Hide error attributes Show error attributes object
          • type string Required

            The type of error

          • reason string

            A human-readable explanation of the error, in English.

          • The server stack trace. Present only if the error_trace=true parameter was sent with the request.

          • Additional properties are allowed.

          • root_cause array[object]
          • suppressed array[object]
GET /_ingest/pipeline/{id}/_simulate
curl \
 --request GET http://api.example.com/_ingest/pipeline/{id}/_simulate \
 --header "Content-Type: application/json" \
 --data '"{\n  \"pipeline\" :\n  {\n    \"description\": \"_description\",\n    \"processors\": [\n      {\n        \"set\" : {\n          \"field\" : \"field2\",\n          \"value\" : \"_value\"\n        }\n      }\n    ]\n  },\n  \"docs\": [\n    {\n      \"_index\": \"index\",\n      \"_id\": \"id\",\n      \"_source\": {\n        \"foo\": \"bar\"\n      }\n    },\n    {\n      \"_index\": \"index\",\n      \"_id\": \"id\",\n      \"_source\": {\n        \"foo\": \"rab\"\n      }\n    }\n  ]\n}"'
Request example
You can specify the used pipeline either in the request body or as a path parameter.
{
  "pipeline" :
  {
    "description": "_description",
    "processors": [
      {
        "set" : {
          "field" : "field2",
          "value" : "_value"
        }
      }
    ]
  },
  "docs": [
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "bar"
      }
    },
    {
      "_index": "index",
      "_id": "id",
      "_source": {
        "foo": "rab"
      }
    }
  ]
}
Response examples (200)
A successful response for running an ingest pipeline against a set of provided documents.
{
   "docs": [
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "bar"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.187Z"
            }
         }
      },
      {
         "doc": {
            "_id": "id",
            "_index": "index",
            "_version": "-3",
            "_source": {
               "field2": "_value",
               "foo": "rab"
            },
            "_ingest": {
               "timestamp": "2017-05-04T22:30:03.188Z"
            }
         }
      }
   ]
}




























































































































































































































Get data frame analytics jobs usage info Added in 7.3.0

GET /_ml/data_frame/analytics/_stats

Query parameters

  • Specifies what to do when the request:

    1. Contains wildcard expressions and there are no data frame analytics jobs that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    The default value returns an empty data_frame_analytics array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

  • from number

    Skips the specified number of data frame analytics jobs.

  • size number

    Specifies the maximum number of data frame analytics jobs to obtain.

  • verbose boolean

    Defines whether the stats response should be verbose.

Responses

  • 200 application/json
    Hide response attributes Show response attributes object
    • count number Required
    • data_frame_analytics array[object] Required

      An array of objects that contain usage information for data frame analytics jobs, which are sorted by the id value in ascending order.

      Hide data_frame_analytics attributes Show data_frame_analytics attributes object
      • Additional properties are allowed.

        Hide analysis_stats attributes Show analysis_stats attributes object
        • Additional properties are allowed.

          Hide classification_stats attributes Show classification_stats attributes object
          • hyperparameters object Required

            Additional properties are allowed.

            Hide hyperparameters attributes Show hyperparameters attributes object
            • alpha number

              Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

            • lambda number

              Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

            • gamma number

              Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

            • eta number

              Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

            • Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

            • Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

            • Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

            • If the algorithm fails to determine a non-trivial tree (more than a single leaf), this parameter determines how many of such consecutive failures are tolerated. Once the number of attempts exceeds the threshold, the forest training stops.

            • Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

            • Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

            • The maximum number of folds for the cross-validation procedure.

            • Determines the maximum number of splits for every feature that can occur in a decision tree when the tree is trained.

            • Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

            • Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

          • iteration number Required

            The number of iterations on the analysis.

          • Time unit for milliseconds

          • timing_stats object Required

            Additional properties are allowed.

            Hide timing_stats attributes Show timing_stats attributes object
          • validation_loss object Required

            Additional properties are allowed.

            Hide validation_loss attributes Show validation_loss attributes object
            • fold_values array[string] Required

              Validation loss values for every added decision tree during the forest growing procedure.

            • loss_type string Required

              The type of the loss metric. For example, binomial_logistic.

        • Additional properties are allowed.

          Hide outlier_detection_stats attributes Show outlier_detection_stats attributes object
          • parameters object Required

            Additional properties are allowed.

            Hide parameters attributes Show parameters attributes object
            • Specifies whether the feature influence calculation is enabled.

            • The minimum outlier score that a document needs to have in order to calculate its feature influence score. Value range: 0-1

            • method string

              The method that outlier detection uses. Available methods are lof, ldof, distance_kth_nn, distance_knn, and ensemble. The default value is ensemble, which means that outlier detection uses an ensemble of different methods and normalises and combines their individual outlier scores to obtain the overall outlier score.

            • Defines the value for how many nearest neighbors each method of outlier detection uses to calculate its outlier score. When the value is not set, different values are used for different ensemble members. This default behavior helps improve the diversity in the ensemble; only override it if you are confident that the value you choose is appropriate for the data set.

            • The proportion of the data set that is assumed to be outlying prior to outlier detection. For example, 0.05 means it is assumed that 5% of values are real outliers and 95% are inliers.

            • If true, the following operation is performed on the columns before computing outlier scores: (x_i - mean(x_i)) / sd(x_i).

          • Time unit for milliseconds

          • timing_stats object Required

            Additional properties are allowed.

            Hide timing_stats attributes Show timing_stats attributes object
        • Additional properties are allowed.

          Hide regression_stats attributes Show regression_stats attributes object
          • hyperparameters object Required

            Additional properties are allowed.

            Hide hyperparameters attributes Show hyperparameters attributes object
            • alpha number

              Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This parameter affects loss calculations by acting as a multiplier of the tree depth. Higher alpha values result in shallower trees and faster training times. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to zero.

            • lambda number

              Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies an L2 regularization term which applies to leaf weights of the individual trees in the forest. A high lambda value causes training to favor small leaf weights. This behavior makes the prediction function smoother at the expense of potentially not being able to capture relevant relationships between the features and the dependent variable. A small lambda value results in large individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

            • gamma number

              Advanced configuration option. Regularization parameter to prevent overfitting on the training data set. Multiplies a linear penalty associated with the size of individual trees in the forest. A high gamma value causes training to prefer small trees. A small gamma value results in larger individual trees and slower training. By default, this value is calculated during hyperparameter optimization. It must be a nonnegative value.

            • eta number

              Advanced configuration option. The shrinkage applied to the weights. Smaller values result in larger forests which have a better generalization error. However, larger forests cause slower training. By default, this value is calculated during hyperparameter optimization. It must be a value between 0.001 and 1.

            • Advanced configuration option. Specifies the rate at which eta increases for each new tree that is added to the forest. For example, a rate of 1.05 increases eta by 5% for each extra tree. By default, this value is calculated during hyperparameter optimization. It must be between 0.5 and 2.

            • Advanced configuration option. Defines the fraction of features that will be used when selecting a random bag for each candidate split. By default, this value is calculated during hyperparameter optimization.

            • Advanced configuration option. Controls the fraction of data that is used to compute the derivatives of the loss function for tree training. A small value results in the use of a small fraction of the data. If this value is set to be less than 1, accuracy typically improves. However, too small a value may result in poor convergence for the ensemble and so require more trees. By default, this value is calculated during hyperparameter optimization. It must be greater than zero and less than or equal to 1.

            • If the algorithm fails to determine a non-trivial tree (more than a single leaf), this parameter determines how many of such consecutive failures are tolerated. Once the number of attempts exceeds the threshold, the forest training stops.

            • Advanced configuration option. A multiplier responsible for determining the maximum number of hyperparameter optimization steps in the Bayesian optimization procedure. The maximum number of steps is determined based on the number of undefined hyperparameters times the maximum optimization rounds per hyperparameter. By default, this value is calculated during hyperparameter optimization.

            • Advanced configuration option. Defines the maximum number of decision trees in the forest. The maximum value is 2000. By default, this value is calculated during hyperparameter optimization.

            • The maximum number of folds for the cross-validation procedure.

            • Determines the maximum number of splits for every feature that can occur in a decision tree when the tree is trained.

            • Advanced configuration option. Machine learning uses loss guided tree growing, which means that the decision trees grow where the regularized loss decreases most quickly. This soft limit combines with the soft_tree_depth_tolerance to penalize trees that exceed the specified depth; the regularized loss increases quickly beyond this depth. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.

            • Advanced configuration option. This option controls how quickly the regularized loss increases when the tree depth exceeds soft_tree_depth_limit. By default, this value is calculated during hyperparameter optimization. It must be greater than or equal to 0.01.

          • iteration number Required

            The number of iterations on the analysis.

          • Time unit for milliseconds

          • timing_stats object Required

            Additional properties are allowed.

            Hide timing_stats attributes Show timing_stats attributes object
          • validation_loss object Required

            Additional properties are allowed.

            Hide validation_loss attributes Show validation_loss attributes object
            • fold_values array[string] Required

              Validation loss values for every added decision tree during the forest growing procedure.

            • loss_type string Required

              The type of the loss metric. For example, binomial_logistic.

      • For running jobs only, contains messages relating to the selection of a node to run the job.

      • data_counts object Required

        Additional properties are allowed.

        Hide data_counts attributes Show data_counts attributes object
        • skipped_docs_count number Required

          The number of documents that are skipped during the analysis because they contained values that are not supported by the analysis. For example, outlier detection does not support missing fields so it skips documents with missing fields. Likewise, all types of analysis skip documents that contain arrays with more than one element.

        • test_docs_count number Required

          The number of documents that are not used for training the model and can be used for testing.

        • training_docs_count number Required

          The number of documents that are used for training the model.

      • id string Required
      • memory_usage object Required

        Additional properties are allowed.

        Hide memory_usage attributes Show memory_usage attributes object
        • This value is present when the status is hard_limit and it is a new estimate of how much memory the job needs.

        • peak_usage_bytes number Required

          The number of bytes used at the highest peak of memory usage.

        • status string Required

          The memory usage status.

        • Time unit for milliseconds

      • progress array[object] Required

        The progress report of the data frame analytics job by phase.

        Hide progress attributes Show progress attributes object
        • phase string Required

          Defines the phase of the data frame analytics job.

        • progress_percent number Required

          The progress that the data frame analytics job has made expressed in percentage.

      • state string Required

        Values are started, stopped, starting, stopping, or failed.

GET /_ml/data_frame/analytics/_stats
curl \
 --request GET http://api.example.com/_ml/data_frame/analytics/_stats
























Stop data frame analytics jobs Added in 7.3.0

POST /_ml/data_frame/analytics/{id}/_stop

A data frame analytics job can be started and stopped multiple times throughout its lifecycle.

Path parameters

  • id string Required

    Identifier for the data frame analytics job. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

Query parameters

  • Specifies what to do when the request:

    1. Contains wildcard expressions and there are no data frame analytics jobs that match.
    2. Contains the _all string or no identifiers and there are no matches.
    3. Contains wildcard expressions and there are only partial matches.

    The default value is true, which returns an empty data_frame_analytics array when there are no matches and the subset of results when there are partial matches. If this parameter is false, the request returns a 404 status code when there are no matches or only partial matches.

  • force boolean

    If true, the data frame analytics job is stopped forcefully.

  • timeout string

    Controls the amount of time to wait until the data frame analytics job stops. Defaults to 20 seconds.

Responses

  • 200 application/json
    Hide response attribute Show response attribute object
POST /_ml/data_frame/analytics/{id}/_stop
curl \
 --request POST http://api.example.com/_ml/data_frame/analytics/{id}/_stop