Aggregations

[preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

An aggregation summarizes your data as metrics, statistics, or other analytics. Aggregations help you answer questions like:

Elasticsearch organizes aggregations into three categories:

You can run aggregations as part of a search by specifying the search API’s aggs parameter. The following search runs a terms aggregation on my-field:

curl "${ES_URL}/my-index/_search?pretty" \
-H "Authorization: ApiKey ${API_KEY}" \
-H "Content-Type: application/json" \
-d'
{
  "aggs": {
    "my-agg-name": {
      "terms": {
        "field": "my-field"
      }
    }
  }
}
'

Aggregation results are in the response’s aggregations object:

{
  "took": 78,
  "timed_out": false,
  "_shards": {...},
  "hits": {...},
  "aggregations": {
    "my-agg-name": {                             
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [...]
    }
  }
}

Use the query parameter to limit the documents on which an aggregation runs:

curl "${ES_URL}/my-index/_search?pretty" \
-H "Authorization: ApiKey ${API_KEY}" \
-H "Content-Type: application/json" \
-d'
{
  "query": {
    "range": {
      "@timestamp": {
        "gte": "now-1d/d",
        "lt": "now/d"
      }
    }
  },
  "aggs": {
    "my-agg-name": {
      "terms": {
        "field": "my-field"
      }
    }
  }
}
'

By default, searches containing an aggregation return both search hits and aggregation results. To return only aggregation results, set size to 0:

curl "${ES_URL}/my-index/_search?pretty" \
-H "Authorization: ApiKey ${API_KEY}" \
-H "Content-Type: application/json" \
-d'
{
  "size": 0,
  "aggs": {
    "my-agg-name": {
      "terms": {
        "field": "my-field"
      }
    }
  }
}
'

You can specify multiple aggregations in the same request:

curl "${ES_URL}/my-index/_search?pretty" \
-H "Authorization: ApiKey ${API_KEY}" \
-H "Content-Type: application/json" \
-d'
{
  "aggs": {
    "my-first-agg-name": {
      "terms": {
        "field": "my-field"
      }
    },
    "my-second-agg-name": {
      "avg": {
        "field": "my-other-field"
      }
    }
  }
}
'

Bucket aggregations support bucket or metric sub-aggregations. For example, a terms aggregation with an avg sub-aggregation calculates an average value for each bucket of documents. There is no level or depth limit for nesting sub-aggregations.

curl "${ES_URL}/my-index/_search?pretty" \
-H "Authorization: ApiKey ${API_KEY}" \
-H "Content-Type: application/json" \
-d'
{
  "aggs": {
    "my-agg-name": {
      "terms": {
        "field": "my-field"
      },
      "aggs": {
        "my-sub-agg-name": {
          "avg": {
            "field": "my-other-field"
          }
        }
      }
    }
  }
}
'

The response nests sub-aggregation results under their parent aggregation:

{
  ...
  "aggregations": {
    "my-agg-name": {                             
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": [
        {
          "key": "foo",
          "doc_count": 5,
          "my-sub-agg-name": {                   
            "value": 75.0
          }
        }
      ]
    }
  }
}

Use the meta object to associate custom metadata with an aggregation:

curl "${ES_URL}/my-index/_search?pretty" \
-H "Authorization: ApiKey ${API_KEY}" \
-H "Content-Type: application/json" \
-d'
{
  "aggs": {
    "my-agg-name": {
      "terms": {
        "field": "my-field"
      },
      "meta": {
        "my-metadata-field": "foo"
      }
    }
  }
}
'

The response returns the meta object in place:

{
  ...
  "aggregations": {
    "my-agg-name": {
      "meta": {
        "my-metadata-field": "foo"
      },
      "doc_count_error_upper_bound": 0,
      "sum_other_doc_count": 0,
      "buckets": []
    }
  }
}

By default, aggregation results include the aggregation’s name but not its type. To return the aggregation type, use the typed_keys query parameter.

curl "${ES_URL}/my-index/_search?typed_keys&pretty" \
-H "Authorization: ApiKey ${API_KEY}" \
-H "Content-Type: application/json" \
-d'
{
  "aggs": {
    "my-agg-name": {
      "histogram": {
        "field": "my-field",
        "interval": 1000
      }
    }
  }
}
'

The response returns the aggregation type as a prefix to the aggregation’s name.

{
  ...
  "aggregations": {
    "histogram#my-agg-name": {                   
      "buckets": []
    }
  }
}

When a field doesn’t exactly match the aggregation you need, you should aggregate on a runtime field:

curl "${ES_URL}/my-index/_search?pretty" \
-H "Authorization: ApiKey ${API_KEY}" \
-H "Content-Type: application/json" \
-d'
{
  "size": 0,
  "runtime_mappings": {
    "message.length": {
      "type": "long",
      "script": "emit(doc[\u0027message.keyword\u0027].value.length())"
    }
  },
  "aggs": {
    "message_length": {
      "histogram": {
        "interval": 10,
        "field": "message.length"
      }
    }
  }
}
'

Scripts calculate field values dynamically, which adds a little overhead to the aggregation. In addition to the time spent calculating, some aggregations like terms and filters can’t use some of their optimizations with runtime fields. In total, performance costs for using a runtime field varies from aggregation to aggregation.

For faster responses, Elasticsearch caches the results of frequently run aggregations in the shard request cache. To get cached results, use the same preference string for each search. If you don’t need search hits, set size to 0 to avoid filling the cache.

Elasticsearch routes searches with the same preference string to the same shards. If the shards' data doesn’t change between searches, the shards return cached aggregation results.

When running aggregations, Elasticsearch uses double values to hold and represent numeric data. As a result, aggregations on long numbers greater than 2^53 are approximate.