Get rollup job capabilities API
editGet rollup job capabilities API
editDeprecated in 8.11.0.
Rollups will be removed in a future version. Use downsampling instead.
Returns the capabilities of any rollup jobs that have been configured for a specific index or index pattern.
Request
editGET _rollup/data/<index>
Prerequisites
edit-
If the Elasticsearch security features are enabled, you must have
monitor
,monitor_rollup
,manage
ormanage_rollup
cluster privileges to use this API. For more information, see Security privileges.
Description
editThis API is useful because a rollup job is often configured to rollup only a subset of fields from the source index. Furthermore, only certain aggregations can be configured for various fields, leading to a limited subset of functionality depending on that configuration.
This API enables you to inspect an index and determine:
- Does this index have associated rollup data somewhere in the cluster?
- If yes to the first question, what fields were rolled up, what aggregations can be performed, and where does the data live?
Path parameters
edit-
<index>
-
(string) Index, indices or index-pattern to return rollup capabilities for.
_all
may be used to fetch rollup capabilities from all jobs.
Examples
editImagine we have an index named sensor-1
full of raw data. We know that the
data will grow over time, so there will be a sensor-2
, sensor-3
, etc. Let’s
create a rollup job that targets the index pattern sensor-*
to accommodate
this future scaling:
resp = client.rollup.put_job( id="sensor", index_pattern="sensor-*", rollup_index="sensor_rollup", cron="*/30 * * * * ?", page_size=1000, groups={ "date_histogram": { "field": "timestamp", "fixed_interval": "1h", "delay": "7d" }, "terms": { "fields": [ "node" ] } }, metrics=[ { "field": "temperature", "metrics": [ "min", "max", "sum" ] }, { "field": "voltage", "metrics": [ "avg" ] } ], ) print(resp)
const response = await client.rollup.putJob({ id: "sensor", index_pattern: "sensor-*", rollup_index: "sensor_rollup", cron: "*/30 * * * * ?", page_size: 1000, groups: { date_histogram: { field: "timestamp", fixed_interval: "1h", delay: "7d", }, terms: { fields: ["node"], }, }, metrics: [ { field: "temperature", metrics: ["min", "max", "sum"], }, { field: "voltage", metrics: ["avg"], }, ], }); console.log(response);
PUT _rollup/job/sensor { "index_pattern": "sensor-*", "rollup_index": "sensor_rollup", "cron": "*/30 * * * * ?", "page_size": 1000, "groups": { "date_histogram": { "field": "timestamp", "fixed_interval": "1h", "delay": "7d" }, "terms": { "fields": [ "node" ] } }, "metrics": [ { "field": "temperature", "metrics": [ "min", "max", "sum" ] }, { "field": "voltage", "metrics": [ "avg" ] } ] }
We can then retrieve the rollup capabilities of that index pattern (sensor-*
)
via the following command:
resp = client.rollup.get_rollup_caps( id="sensor-*", ) print(resp)
response = client.rollup.get_rollup_caps( id: 'sensor-*' ) puts response
const response = await client.rollup.getRollupCaps({ id: "sensor-*", }); console.log(response);
GET _rollup/data/sensor-*
Which will yield the following response:
{ "sensor-*" : { "rollup_jobs" : [ { "job_id" : "sensor", "rollup_index" : "sensor_rollup", "index_pattern" : "sensor-*", "fields" : { "node" : [ { "agg" : "terms" } ], "temperature" : [ { "agg" : "min" }, { "agg" : "max" }, { "agg" : "sum" } ], "timestamp" : [ { "agg" : "date_histogram", "time_zone" : "UTC", "fixed_interval" : "1h", "delay": "7d" } ], "voltage" : [ { "agg" : "avg" } ] } } ] } }
The response that is returned contains information that is similar to the original rollup configuration, but formatted differently. First, there are some house-keeping details: the rollup job ID, the index that holds the rolled data, and the index pattern that the job was targeting.
Next it shows a list of fields that contain data eligible for rollup searches.
Here we see four fields: node
, temperature
, timestamp
and voltage
. Each
of these fields list the aggregations that are possible. For example, you can
use a min, max or sum aggregation on the temperature
field, but only a
date_histogram
on timestamp
.
Note that the rollup_jobs
element is an array; there can be multiple,
independent jobs configured for a single index or index pattern. Each of these
jobs may have different configurations, so the API returns a list of all the
various configurations available.
We could also retrieve the same information with a request to _all
:
resp = client.rollup.get_rollup_caps( id="_all", ) print(resp)
response = client.rollup.get_rollup_caps( id: '_all' ) puts response
const response = await client.rollup.getRollupCaps({ id: "_all", }); console.log(response);
GET _rollup/data/_all
But note that if we use the concrete index name (sensor-1
), we’ll retrieve no
rollup capabilities:
resp = client.rollup.get_rollup_caps( id="sensor-1", ) print(resp)
response = client.rollup.get_rollup_caps( id: 'sensor-1' ) puts response
const response = await client.rollup.getRollupCaps({ id: "sensor-1", }); console.log(response);
GET _rollup/data/sensor-1
{ }
Why is this? The original rollup job was configured against a specific index
pattern (sensor-*
) not a concrete index (sensor-1
). So while the index
belongs to the pattern, the rollup job is only valid across the entirety of
the pattern not just one of it’s containing indices. So for that reason, the
get rollup capabilities API only returns information based on the originally
configured index name or pattern.