Create a datafeed | Elasticsearch Serverless API

Create a datafeed Added in 5.4.0

PUT /_ml/datafeeds/{datafeed_id}

Datafeeds retrieve data from Elasticsearch for analysis by an anomaly detection job. You can associate only one datafeed with each anomaly detection job. The datafeed contains a query that runs at a defined interval (frequency). If you are concerned about delayed data, you can add a delay (query_delay) at each interval. When Elasticsearch security features are enabled, your datafeed remembers which roles the user who created it had at the time of creation and runs the query using those same roles. If you provide secondary authorization headers, those credentials are used instead. You must use Kibana, this API, or the create anomaly detection jobs API to create a datafeed. Do not add a datafeed directly to the .ml-config index. Do not give users write privileges on the .ml-config index.

Path parameters

datafeed_id string Required

A numerical character string that uniquely identifies the datafeed. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

Query parameters

allow_no_indices boolean

If true, wildcard indices expressions that resolve into no concrete indices are ignored. This includes the _all string or when no indices are specified.
expand_wildcards string | array[string]

Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values.
ignore_throttled boolean Deprecated

If true, concrete, expanded, or aliased indices are ignored when frozen.
ignore_unavailable boolean

If true, unavailable indices (missing or closed) are ignored.

application/json

Body Required

aggregations object

If set, the datafeed performs aggregation searches. Support for aggregations is limited and should be used only with low cardinality data.
chunking_config object

Additional properties are allowed.
Hide chunking_config attributes Show chunking_config attributes object
- mode string Required
  
  Values are auto, manual, or off.
- time_span string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
delayed_data_check_config object

Additional properties are allowed.
Hide delayed_data_check_config attributes Show delayed_data_check_config attributes object
- check_window string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
- enabled boolean Required
  
  Specifies whether the datafeed periodically checks for delayed data.
frequency string

A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
indices string | array[string]
indices_options object

Additional properties are allowed.
Hide indices_options attributes Show indices_options attributes object
- allow_no_indices boolean
  
  If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
- expand_wildcards string | array[string]
- ignore_unavailable boolean
  
  If true, missing or closed indices are not included in the response.
- ignore_throttled boolean
  
  If true, concrete, expanded or aliased indices are ignored when frozen.
job_id string
max_empty_searches number

If a real-time datafeed has never seen any data (including during any initial training period), it automatically stops and closes the associated job after this many real-time searches return no documents. In other words, it stops after frequency times max_empty_searches of real-time operation. If not set, a datafeed with no end time that sees no data remains started until it is explicitly stopped. By default, it is not set.
query object

Additional properties are allowed.

External documentation
query_delay string

A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
runtime_mappings object
Hide runtime_mappings attribute Show runtime_mappings attribute object
- * object Additional properties
  
  Additional properties are allowed.
  Hide * attributes Show * attributes object
  
  fields object
  
  For type composite
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  Additional properties are allowed.
  
  Hide * attribute Show * attribute object
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, ip, keyword, long, or lookup.
  
  fetch_fields array[object]
  
  For type lookup
  
  Hide fetch_fields attributes Show fetch_fields attributes object
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  format string
  
  format string
  
  A custom format for date type runtime fields.
  
  input_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_index string
  
  script object
  
  Additional properties are allowed.
  
  Hide script attributes Show script attributes object
  
  source string
  
  The script source.
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  Additional properties are allowed.
  
  lang string
  
  Any of:
  _types:ScriptLanguage string _types:ScriptLanguage string
  
  Values are painless, expression, mustache, or java.
  
  options object
  
  Hide options attribute Show options attribute object
  
  * string Additional properties
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, ip, keyword, long, or lookup.
script_fields object

Specifies scripts that evaluate custom expressions and returns script fields to the datafeed. The detector configuration objects in a job can contain functions that use these script fields.
Hide script_fields attribute Show script_fields attribute object
- * object Additional properties
  
  Additional properties are allowed.
  Hide * attributes Show * attributes object
  
  script object Required
  
  Additional properties are allowed.
  
  Hide script attributes Show script attributes object
  
  source string
  
  The script source.
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  Additional properties are allowed.
  
  lang string
  
  Any of:
  _types:ScriptLanguage string _types:ScriptLanguage string
  
  Values are painless, expression, mustache, or java.
  
  options object
  
  Hide options attribute Show options attribute object
  
  * string Additional properties
  
  ignore_failure boolean
scroll_size number

The size parameter that is used in Elasticsearch searches when the datafeed does not use aggregations. The maximum value is the value of index.max_result_window, which is 10,000 by default.
headers object

Responses

200 application/json
Hide response attributes Show response attributes object
- aggregations object
- authorization object
  
  Additional properties are allowed.
  
  Hide authorization attributes Show authorization attributes object
  
  api_key object
  
  Additional properties are allowed.
  
  Hide api_key attributes Show api_key attributes object
  
  id string Required
  
  The identifier for the API key.
  
  name string Required
  
  The name of the API key.
  
  roles array[string]
  
  If a user ID was used for the most recent update to the datafeed, its roles at the time of the update are listed in the response.
  
  service_account string
  
  If a service account was used for the most recent update to the datafeed, the account name is listed in the response.
- chunking_config object Required
  
  Additional properties are allowed.
  
  Hide chunking_config attributes Show chunking_config attributes object
  
  mode string Required
  
  Values are auto, manual, or off.
  
  time_span string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
- delayed_data_check_config object
  
  Additional properties are allowed.
  
  Hide delayed_data_check_config attributes Show delayed_data_check_config attributes object
  
  check_window string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
  
  enabled boolean Required
  
  Specifies whether the datafeed periodically checks for delayed data.
- datafeed_id string Required
- frequency string
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
- indices array[string] Required
- job_id string Required
- indices_options object
  
  Additional properties are allowed.
  
  Hide indices_options attributes Show indices_options attributes object
  
  allow_no_indices boolean
  
  If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.
  
  expand_wildcards string | array[string]
  
  ignore_unavailable boolean
  
  If true, missing or closed indices are not included in the response.
  
  ignore_throttled boolean
  
  If true, concrete, expanded or aliased indices are ignored when frozen.
- max_empty_searches number
- query object Required
  
  Additional properties are allowed.
  
  External documentation
- query_delay string Required
  
  A duration. Units can be nanos, micros, ms (milliseconds), s (seconds), m (minutes), h (hours) and d (days). Also accepts "0" without a unit and "-1" to indicate an unspecified value.
- runtime_mappings object
  
  Hide runtime_mappings attribute Show runtime_mappings attribute object
  
  * object Additional properties
  
  Additional properties are allowed.
  
  Hide * attributes Show * attributes object
  
  fields object
  
  For type composite
  
  Hide fields attribute Show fields attribute object
  
  * object Additional properties
  
  Additional properties are allowed.
  
  Hide * attribute Show * attribute object
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, ip, keyword, long, or lookup.
  
  fetch_fields array[object]
  
  For type lookup
  
  Hide fetch_fields attributes Show fetch_fields attributes object
  
  field string Required
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  format string
  
  format string
  
  A custom format for date type runtime fields.
  
  input_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_field string
  
  Path to field or array of paths. Some API's support wildcards in the path to select multiple fields.
  
  target_index string
  
  script object
  
  Additional properties are allowed.
  
  Hide script attributes Show script attributes object
  
  source string
  
  The script source.
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  Additional properties are allowed.
  
  lang string
  
  Any of:
  _types:ScriptLanguage string _types:ScriptLanguage string
  
  Values are painless, expression, mustache, or java.
  
  options object
  
  Hide options attribute Show options attribute object
  
  * string Additional properties
  
  type string Required
  
  Values are boolean, composite, date, double, geo_point, ip, keyword, long, or lookup.
- script_fields object
  
  Hide script_fields attribute Show script_fields attribute object
  
  * object Additional properties
  
  Additional properties are allowed.
  
  Hide * attributes Show * attributes object
  
  script object Required
  
  Additional properties are allowed.
  
  Hide script attributes Show script attributes object
  
  source string
  
  The script source.
  
  id string
  
  params object
  
  Specifies any named parameters that are passed into the script as variables. Use parameters instead of hard-coded values to decrease compile time.
  
  Hide params attribute Show params attribute object
  
  * object Additional properties
  
  Additional properties are allowed.
  
  lang string
  
  Any of:
  _types:ScriptLanguage string _types:ScriptLanguage string
  
  Values are painless, expression, mustache, or java.
  
  options object
  
  Hide options attribute Show options attribute object
  
  * string Additional properties
  
  ignore_failure boolean
- scroll_size number Required

PUT /_ml/datafeeds/{datafeed_id}

curl \
 -X PUT http://api.example.com/_ml/datafeeds/{datafeed_id} \
 -H "Content-Type: application/json" \
 -d '{"aggregations":{},"chunking_config":{"mode":"auto","time_span":"string"},"delayed_data_check_config":{"check_window":"string","enabled":true},"frequency":"string","indices":"string","indices_options":{"allow_no_indices":true,"expand_wildcards":"string","ignore_unavailable":true,"ignore_throttled":true},"job_id":"string","max_empty_searches":42.0,"query":{},"query_delay":"string","runtime_mappings":{"additionalProperty1":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"},"additionalProperty2":{"fields":{"additionalProperty1":{"type":"boolean"},"additionalProperty2":{"type":"boolean"}},"fetch_fields":[{"field":"string","format":"string"}],"format":"string","input_field":"string","target_field":"string","target_index":"string","script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"type":"boolean"}},"script_fields":{"additionalProperty1":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true},"additionalProperty2":{"script":{"source":"string","id":"string","params":{"additionalProperty1":{},"additionalProperty2":{}},"":"painless","options":{"additionalProperty1":"string","additionalProperty2":"string"}},"ignore_failure":true}},"scroll_size":42.0,"headers":{}}'

Create a datafeed Added in 5.4.0

Path parameters

Query parameters

Body Required

lang string

lang string

Responses

lang string

lang string