Estimate memory usage API

edit

Estimates memory usage for the given data frame analytics config.

This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

Request

edit

POST _ml/data_frame/analytics/_estimate_memory_usage

Prerequisites

edit

Description

edit

This API estimates memory usage for the given data frame analytics config before the data frame analytics job is even created.

Serves as an advice on how to set model_memory_limit when creating data frame analytics job.

Request body

edit
data_frame_analytics_config
(Required, object) Intended configuration of data frame analytics job. For more information, see Data frame analytics job resources. Note that id and dest don’t need to be provided in the context of this API.

Response body

edit
expected_memory_without_disk
(string) Estimated memory usage under the assumption that the whole data frame analytics should happen in memory (i.e. without overflowing to disk).
expected_memory_with_disk
(string) Estimated memory usage under the assumption that overflowing to disk is allowed during data frame analytics. expected_memory_with_disk is usually smaller than expected_memory_without_disk as using disk allows to limit the main memory needed to perform data frame analytics.

Examples

edit
POST _ml/data_frame/analytics/_estimate_memory_usage
{
  "data_frame_analytics_config": {
    "source": {
      "index": "logdata"
    },
    "analysis": {
      "outlier_detection": {}
    }
  }
}

The API returns the following results:

{
  "expected_memory_without_disk": "128MB",
  "expected_memory_with_disk": "32MB"
}