Create and manage rollup jobs

In Management, you’ll find a UI for viewing, creating, starting, stopping, and deleting rollup jobs. A rollup job is a periodic task that summarizes data from indices specified by an index pattern and rolls it into a new index. To navigate to the UI, go to Management, and under Elasticsearch, click Rollup Jobs.

Kibana makes it easy for you to create a rollup job by walking you through the process step by step. The first step is to define the job logistics. These include the name of the rollup job, the index or indices to summarize, and the output rollup index.

The index pattern cannot match the name of the output rollup index. For example, if your index pattern is metricbeat-*, you cannot name your rollup index metricbeat-rollup. Otherwise, the job will attempt to capture the data in the rollup index.

You must set a schedule for the rollup job: how often to collect the data, the number of documents to roll up at a time, and the duration of its latency. The latency buffer field is provided to protect against the late arrival of data from Beats or other sources. By delaying the rollup for the specified amount of time from when the job starts, you allow for the inclusion of late-arriving data in the rollup.

In the subsequent phases, you define the Date Histogram aggregation for the job and optionally the Terms and Histogram aggregations.

The final step is to specify the fields for calculating metrics. For each selected field, you can collect any or all of the following: value count, average, sum, min, and max.

Before you save the rollup job, Kibana displays a summary of the rollup job for validation.

Selecting a job on the Rollup jobs page shows its details. The Manage menu in the lower right enables you to start, stop, and delete the rollup job. You must first stop a rollup job before deleting it.