Preview transform API
editPreview transform API
editPreviews a transform.
Request
editGET _transform/<transform_id>/_preview
POST _transform/<transform_id>/_preview
GET _transform/_preview
POST _transform/_preview
Prerequisites
editRequires the following privileges:
-
cluster:
manage_transform
(thetransform_admin
built-in role grants this privilege) -
source indices:
read
,view_index_metadata
.
Description
editThis API generates a preview of the results that you will get when you run the create transforms API with the same configuration. It returns a maximum of 100 results. The calculations are based on all the current data in the source index.
It also generates a list of mappings and settings for the destination index. If the destination index does not exist when you start a transform, these are the mappings and settings that are used. These values are determined based on the field types of the source index and the transform aggregations.
There are some limitations that might result in poor mappings. As a work-around, create the destination index or an index template with your preferred mappings before you start the transform.
You must choose either the latest
or pivot
method for your transform; you
cannot use both in a single transform.
Path parameters
edit-
<transform_id>
-
(Optional, string) Id of the transform to preview.
If you provide the
<transform_id>
as a path parameter, you cannot provide transform configuration details in the request body.
Query parameters
edit-
timeout
-
(Optional, time)
Period to wait for a response. If no response is received before the timeout
expires, the request fails and returns an error. Defaults to
30s
.
Request body
edit-
description
- (Optional, string) Free text description of the transform.
-
dest
-
(Optional, object) The destination for the transform.
Properties of
dest
-
index
- (Optional, string) The destination index for the transform. The mappings of the destination index are deduced based on the source fields when possible. If alternate mappings are required, use the Create index API prior to starting the transform.
-
pipeline
- (Optional, string) The unique identifier for an ingest pipeline.
-
-
frequency
-
(Optional, time units)
The interval between checks for changes in the source indices when the
transform is running continuously. Also determines the retry interval in the
event of transient failures while the transform is searching or indexing. The
minimum value is
1s
and the maximum is1h
. The default value is1m
.
-
latest
-
(Required*, object) The
latest
method transforms the data by finding the latest document for each unique key.Properties of
latest
-
sort
- (Required, string) Specifies the date field that is used to identify the latest documents.
-
unique_key
- (Required, array of strings) Specifies an array of one or more fields that are used to group the data.
-
-
pivot
-
(Required, object) The
pivot
method transforms the data by aggregating and grouping it. These objects define thegroup by
fields and the aggregation to reduce the data.Properties of
pivot
-
aggregations
oraggs
-
(Required, object) Defines how to aggregate the grouped data. The following aggregations are currently supported:
-
group_by
-
(Required, object) Defines how to group the data. More than one grouping can be defined per pivot. The following groupings are currently supported:
-
max_page_search_size
-
(Optional, integer)
[7.8.0]
Deprecated in 7.8.0. Moved to
settings
.
-
-
retention_policy
-
(Optional, object) Defines a retention policy for the transform. Data that meets the defined criteria is deleted from the destination index.
Properties of
retention_policy
-
time
-
(Required, object) Specifies that the transform uses a time field to set the retention policy.
Properties of
time
-
field
- (Required, string) The date field that is used to calculate the age of the document.
-
max_age
- (Required, time units) Specifies the maximum age of a document in the destination index. Documents that are older than the configured value are removed from the destination index.
-
-
-
source
-
(Required, object) The source of the data for the transform.
Properties of
source
-
index
-
(Required, string or array) The source indices for the transform. It can be a single index, an index pattern (for example,
"my-index-*"
), an array of indices (for example,["my-index-000001", "my-index-000002"]
), or an array of index patterns (for example,["my-index-*", "my-other-index-*"]
. For remote indices use the syntax"remote_name:index_name"
.If any indices are in remote clusters then the master node and at least one transform node must have the
remote_cluster_client
node role. -
query
- (Optional, object) A query clause that retrieves a subset of data from the source index. See Query DSL.
-
runtime_mappings
- (Optional, object) Definitions of search-time runtime fields that can be used by the transform. For search runtime fields all data nodes, including remote nodes, must be 7.12 or later.
-
-
sync
-
(Optional, object) Defines the properties transforms require to run continuously.
Properties of
sync
-
time
-
(Optional, object) Specifies that the transform uses a time field to synchronize the source and destination indices.
Properties of
analysis_config
-
delay
-
(Optional, time units)
The time delay between the current time and the latest input data time. The
default value is
60s
. -
field
- (Optional, string) The date field that is used to identify new documents in the source.
-
-
-
settings
-
(Optional, object) Defines optional transform settings.
Properties of
settings
-
docs_per_second
-
(Optional, float)
Specifies a limit on the number of input documents per second. This setting
throttles the transform by adding a wait time between search requests. The
default value is
null
, which disables throttling. -
align_checkpoints
-
(Optional, boolean)
Specifies whether the transform checkpoint ranges should be optimized for performance.
Such optimization can align checkpoint ranges with date histogram interval when date histogram
is specified as a group source in the transform config. As an effect, less document updates in the
destination index will be performed thus improving overall performance.
The default value is
true
, which means the checkpoint ranges will be optimized if possible. -
max_page_search_size
-
(Optional, integer)
Defines the initial page size to use for the composite aggregation for each
checkpoint. If circuit breaker exceptions occur, the page size is dynamically
adjusted to a lower value. The minimum value is
10
and the maximum is65,536
. The default value is500
.
-
Response body
edit-
preview
- (array) An array of documents. In particular, they are the JSON representation of the documents that would be created in the destination index by the transform.
-
generated_dest_index
-
(object) Contains details about the destination index.
Properties of
generated_dest_index
-
aliases
- (object) The aliases for the destination index.
-
mappings
- (object) The mappings for each document in the destination index.
-
settings
- (object) The index settings for the destination index.
-
Examples
editPOST _transform/_preview { "source": { "index": "kibana_sample_data_ecommerce" }, "pivot": { "group_by": { "customer_id": { "terms": { "field": "customer_id" } } }, "aggregations": { "max_price": { "max": { "field": "taxful_total_price" } } } } }
The data that is returned for this example is as follows:
{ "preview" : [ { "max_price" : 171.0, "customer_id" : "10" }, { "max_price" : 233.0, "customer_id" : "11" }, { "max_price" : 200.0, "customer_id" : "12" } ... ], "generated_dest_index" : { "mappings" : { "_meta" : { "_transform" : { "transform" : "transform-preview", "version" : { "created" : "7.7.0" }, "creation_date_in_millis" : 1584738236757 }, "created_by" : "transform" }, "properties" : { "max_price" : { "type" : "half_float" }, "customer_id" : { "type" : "keyword" } } }, "settings" : { "index" : { "number_of_shards" : "1", "auto_expand_replicas" : "0-1" } }, "aliases" : { } } }