Transform limitations

The following limitations and known problems apply to the 7.2 release of the Elastic data frame feature:

Data frames do not (yet) support fields containing arrays – in the UI or the API. If you try to create one, the UI will fail to show the source index table.

Cross-cluster search is not supported in 7.2 for transforms.

The Kibana Machine Learning > Data Frames page lists up to 100 transforms. You can work-around this limitation by calling the GET transforms API with the size parameter.

If you use a date histogram in the group_by object in the create or preview transform APIs, the defined interval and time format must have the same time fidelity. Otherwise, it might cause issues in the data frame.

For example, if you set the calendar_interval of the date histogram to one minute (1m), then make sure that the format is yyyy-MM-dd HH:mm instead of yyyy-MM-dd HH:00.

When you create a transform that uses a date_histogram as a group-by and set the interval to 1y, the date could be interpreted incorrectly in the generated date field of the destination index. The reason is that the yyyy value can be identified incorrectly as epoch_millis. As a workaround, using the API, you may define a custom destination index data format mapping prior to starting the transform. For example:

"mappings" : {
        "properties" : {
            "custom_date" : {
            "type" : "date",
            "format": "yyyy"
            }
          }
        }

Data frames use composite aggregations to transform data. In some cases, composite aggregations may return responses which are not compatible with the mappings set for the destination index. For example "NaN", "Infinity" or possibly a numeric overflow. Where possible, a null response has been substituted. Please, check Elasticsearch logs if you think this may have occurred. As a workaround, using the API, you may define custom destination index mappings prior to starting the transform.