IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

« Transform APIs Delete transforms API »

› › ›

Create transforms API

edit

Create transforms API

edit

Instantiates a transform.

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

Request

edit

PUT _data_frame/transforms/<transform_id>

Prerequisites

edit

If the Elasticsearch security features are enabled, you must have manage_data_frame_transforms cluster privileges to use this API. The built-in data_frame_transforms_admin role has these privileges. You must also have read and view_index_metadata privileges on the source index and read, create_index, and index privileges on the destination index. For more information, see Security privileges and Built-in roles.

Description

edit

This API defines a transform, which copies data from source indices, transforms it, and persists it into an entity-centric destination index. The entities are defined by the set of group_by fields in the pivot object. You can also think of the destination index as a two-dimensional tabular data structure (known as a data frame). The ID for each document in the data frame is generated from a hash of the entity, so there is a unique row per entity. For more information, see Transforming data.

When the transform is created, a series of validations occur to ensure its success. For example, there is a check for the existence of the source indices and a check that the destination index is not part of the source index pattern.

You must use Kibana or this API to create a transform. Do not put a transform directly into any .data-frame-internal* indices using the Elasticsearch index API. If Elasticsearch security features are enabled, do not give users any privileges on .data-frame-internal* indices.

Path parameters

edit

<transform_id>: (Required, string) Identifier for the transform. This identifier can contain lowercase alphanumeric characters (a-z and 0-9), hyphens, and underscores. It must start and end with alphanumeric characters.

Request body

edit

description

(Optional, string) Free text description of the transform.

dest

(Required, object) Required. The destination configuration, which has the following properties:

index: (Required, string) The destination index for the transform.

pivot

(Required, object) Defines the pivot function group by fields and the aggregation to reduce the data. See Pivot objects.

source

(Required, object) The source configuration, which has the following properties:

index: (Required, string or array) The source indices for the transform. It can be a single index, an index pattern (for example, "myindex*"), or an array of indices (for example, ["index1", "index2"]).
query: (Optional, object) A query clause that retrieves a subset of data from the source index. See Query DSL.

Examples

edit

PUT _data_frame/transforms/ecommerce_transform
{
  "source": {
    "index": "kibana_sample_data_ecommerce",
    "query": {
      "term": {
        "geoip.continent_name": {
          "value": "Asia"
        }
      }
    }
  },
  "dest": {
    "index": "kibana_sample_data_ecommerce_transform"
  },
  "pivot": {
    "group_by": {
      "customer_id": {
        "terms": {
          "field": "customer_id"
        }
      }
    },
    "aggregations": {
      "max_price": {
        "max": {
          "field": "taxful_total_price"
        }
      }
    }
  },
  "description": "Maximum priced ecommerce data by customer_id in Asia"
}

When the transform is created, you receive the following results:

{
  "acknowledged" : true
}

« Transform APIs Delete transforms API »