Create a transform Added in 7.2.0
Creates a transform.
A transform copies data from source indices, transforms it, and persists it into an entity-centric destination index. You can also think of the destination index as a two-dimensional tabular data structure (known as a data frame). The ID for each document in the data frame is generated from a hash of the entity, so there is a unique row per entity.
You must choose either the latest or pivot method for your transform; you cannot use both in a single transform. If
you choose to use the pivot method for your transform, the entities are defined by the set of group_by fields in
the pivot object. If you choose to use the latest method, the entities are defined by the unique_key field values
in the latest object.
You must have create_index, index, and read privileges on the destination index and read and
view_index_metadata privileges on the source indices. When Elasticsearch security features are enabled, the
transform remembers which roles the user that created it had at the time of creation and uses those same roles. If
those roles do not have the required privileges on the source and destination indices, the transform fails when it
attempts unauthorized operations.
NOTE: You must use Kibana or this API to create a transform. Do not add a transform directly into any
.transform-internal* indices using the Elasticsearch index API. If Elasticsearch security features are enabled, do
not give users any privileges on .transform-internal* indices. If you used transforms prior to 7.5, also do not
give users any privileges on .data-frame-internal* indices.
Path parameters
-
Identifier for the transform. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It has a 64 character limit and must start and end with alphanumeric characters.
Query parameters
-
defer_validation boolean
When the transform is created, a series of validations occur to ensure its success. For example, there is a check for the existence of the source indices and a check that the destination index is not part of the source index pattern. You can use this parameter to skip the checks, for example when the source index does not exist until after the transform is created. The validations are always run when you start the transform, however, with the exception of privilege checks.
-
timeout string
Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error.
Body Required
-
Additional properties are allowed.
-
description string
Free text description of the transform.
-
frequency string
A duration. Units can be
nanos,micros,ms(milliseconds),s(seconds),m(minutes),h(hours) andd(days). Also accepts "0" without a unit and "-1" to indicate an unspecified value. -
latest object
Additional properties are allowed.
-
_meta object
-
pivot object
Additional properties are allowed.
-
retention_policy object
Additional properties are allowed.
-
settings object
Additional properties are allowed.
-
Additional properties are allowed.
-
sync object
Additional properties are allowed.
curl \
-X PUT http://api.example.com/_transform/{transform_id} \
-H "Content-Type: application/json" \
-d '{"dest":{"index":"kibana_sample_data_ecommerce_transform1","pipeline":"add_timestamp_pipeline"},"sync":{"time":{"delay":"60s","field":"order_date"}},"pivot":{"group_by":{"customer_id":{"terms":{"field":"customer_id","missing_bucket":true}}},"aggregations":{"max_price":{"max":{"field":"taxful_total_price"}}}},"source":{"index":"kibana_sample_data_ecommerce","query":{"term":{"geoip.continent_name":{"value":"Asia"}}}},"frequency":"5m","description":"Maximum priced ecommerce data by customer_id in Asia","retention_policy":{"time":{"field":"order_date","max_age":"30d"}}}'{
"dest": {
"index": "kibana_sample_data_ecommerce_transform1",
"pipeline": "add_timestamp_pipeline"
},
"sync": {
"time": {
"delay": "60s",
"field": "order_date"
}
},
"pivot": {
"group_by": {
"customer_id": {
"terms": {
"field": "customer_id",
"missing_bucket": true
}
}
},
"aggregations": {
"max_price": {
"max": {
"field": "taxful_total_price"
}
}
}
},
"source": {
"index": "kibana_sample_data_ecommerce",
"query": {
"term": {
"geoip.continent_name": {
"value": "Asia"
}
}
}
},
"frequency": "5m",
"description": "Maximum priced ecommerce data by customer_id in Asia",
"retention_policy": {
"time": {
"field": "order_date",
"max_age": "30d"
}
}
}{
"dest": {
"index": "kibana_sample_data_ecommerce_transform2"
},
"sync": {
"time": {
"delay": "60s",
"field": "order_date"
}
},
"latest": {
"sort": "order_date",
"unique_key": [
"customer_id"
]
},
"source": {
"index": "kibana_sample_data_ecommerce"
},
"frequency": "5m",
"description": "Latest order for each customer"
}{
"acknowledged": true
}