Top hits aggregation
editTop hits aggregation
editA top_hits
metric aggregator keeps track of the most relevant document being aggregated. This aggregator is intended
to be used as a sub aggregator, so that the top matching documents can be aggregated per bucket.
We do not recommend using top_hits
as a top-level aggregation. If you
want to group search hits, use the collapse
parameter instead.
The top_hits
aggregator can effectively be used to group result sets by certain fields via a bucket aggregator.
One or more bucket aggregators determines by which properties a result set get sliced into.
Options
edit-
from
- The offset from the first result you want to fetch. -
size
- The maximum number of top matching hits to return per bucket. By default the top three matching hits are returned. -
sort
- How the top matching hits should be sorted. By default the hits are sorted by the score of the main query.
Supported per hit features
editThe top_hits aggregation returns regular search hits, because of this many per hit features can be supported:
If you only need docvalue_fields
, size
, and sort
then
Top metrics might be a more efficient choice than the Top Hits Aggregation.
top_hits
does not support the rescore
parameter. Query rescoring
applies only to search hits, not aggregation results. To change the scores used
by aggregations, use a function_score
or
script_score
query.
Example
editIn the following example we group the sales by type and per type we show the last sale. For each sale only the date and price fields are being included in the source.
resp = client.search( index="sales", size="0", aggs={ "top_tags": { "terms": { "field": "type", "size": 3 }, "aggs": { "top_sales_hits": { "top_hits": { "sort": [ { "date": { "order": "desc" } } ], "_source": { "includes": [ "date", "price" ] }, "size": 1 } } } } }, ) print(resp)
response = client.search( index: 'sales', size: 0, body: { aggregations: { top_tags: { terms: { field: 'type', size: 3 }, aggregations: { top_sales_hits: { top_hits: { sort: [ { date: { order: 'desc' } } ], _source: { includes: [ 'date', 'price' ] }, size: 1 } } } } } } ) puts response
const response = await client.search({ index: "sales", size: 0, aggs: { top_tags: { terms: { field: "type", size: 3, }, aggs: { top_sales_hits: { top_hits: { sort: [ { date: { order: "desc", }, }, ], _source: { includes: ["date", "price"], }, size: 1, }, }, }, }, }, }); console.log(response);
POST /sales/_search?size=0 { "aggs": { "top_tags": { "terms": { "field": "type", "size": 3 }, "aggs": { "top_sales_hits": { "top_hits": { "sort": [ { "date": { "order": "desc" } } ], "_source": { "includes": [ "date", "price" ] }, "size": 1 } } } } } }
Possible response:
{ ... "aggregations": { "top_tags": { "doc_count_error_upper_bound": 0, "sum_other_doc_count": 0, "buckets": [ { "key": "hat", "doc_count": 3, "top_sales_hits": { "hits": { "total" : { "value": 3, "relation": "eq" }, "max_score": null, "hits": [ { "_index": "sales", "_id": "AVnNBmauCQpcRyxw6ChK", "_source": { "date": "2015/03/01 00:00:00", "price": 200 }, "sort": [ 1425168000000 ], "_score": null } ] } } }, { "key": "t-shirt", "doc_count": 3, "top_sales_hits": { "hits": { "total" : { "value": 3, "relation": "eq" }, "max_score": null, "hits": [ { "_index": "sales", "_id": "AVnNBmauCQpcRyxw6ChL", "_source": { "date": "2015/03/01 00:00:00", "price": 175 }, "sort": [ 1425168000000 ], "_score": null } ] } } }, { "key": "bag", "doc_count": 1, "top_sales_hits": { "hits": { "total" : { "value": 1, "relation": "eq" }, "max_score": null, "hits": [ { "_index": "sales", "_id": "AVnNBmatCQpcRyxw6ChH", "_source": { "date": "2015/01/01 00:00:00", "price": 150 }, "sort": [ 1420070400000 ], "_score": null } ] } } } ] } } }
Field collapse example
editField collapsing or result grouping is a feature that logically groups a result set into groups and per group returns
top documents. The ordering of the groups is determined by the relevancy of the first document in a group. In
Elasticsearch this can be implemented via a bucket aggregator that wraps a top_hits
aggregator as sub-aggregator.
In the example below we search across crawled webpages. For each webpage we store the body and the domain the webpage
belong to. By defining a terms
aggregator on the domain
field we group the result set of webpages by domain. The
top_hits
aggregator is then defined as sub-aggregator, so that the top matching hits are collected per bucket.
Also a max
aggregator is defined which is used by the terms
aggregator’s order feature to return the buckets by
relevancy order of the most relevant document in a bucket.
resp = client.search( index="sales", query={ "match": { "body": "elections" } }, aggs={ "top_sites": { "terms": { "field": "domain", "order": { "top_hit": "desc" } }, "aggs": { "top_tags_hits": { "top_hits": {} }, "top_hit": { "max": { "script": { "source": "_score" } } } } } }, ) print(resp)
response = client.search( index: 'sales', body: { query: { match: { body: 'elections' } }, aggregations: { top_sites: { terms: { field: 'domain', order: { top_hit: 'desc' } }, aggregations: { top_tags_hits: { top_hits: {} }, top_hit: { max: { script: { source: '_score' } } } } } } } ) puts response
const response = await client.search({ index: "sales", query: { match: { body: "elections", }, }, aggs: { top_sites: { terms: { field: "domain", order: { top_hit: "desc", }, }, aggs: { top_tags_hits: { top_hits: {}, }, top_hit: { max: { script: { source: "_score", }, }, }, }, }, }, }); console.log(response);
POST /sales/_search { "query": { "match": { "body": "elections" } }, "aggs": { "top_sites": { "terms": { "field": "domain", "order": { "top_hit": "desc" } }, "aggs": { "top_tags_hits": { "top_hits": {} }, "top_hit" : { "max": { "script": { "source": "_score" } } } } } } }
At the moment the max
(or min
) aggregator is needed to make sure the buckets from the terms
aggregator are
ordered according to the score of the most relevant webpage per domain. Unfortunately the top_hits
aggregator
can’t be used in the order
option of the terms
aggregator yet.
top_hits support in a nested or reverse_nested aggregator
editIf the top_hits
aggregator is wrapped in a nested
or reverse_nested
aggregator then nested hits are being returned.
Nested hits are in a sense hidden mini documents that are part of regular document where in the mapping a nested field type
has been configured. The top_hits
aggregator has the ability to un-hide these documents if it is wrapped in a nested
or reverse_nested
aggregator. Read more about nested in the nested type mapping.
If nested type has been configured a single document is actually indexed as multiple Lucene documents and they share
the same id. In order to determine the identity of a nested hit there is more needed than just the id, so that is why
nested hits also include their nested identity. The nested identity is kept under the _nested
field in the search hit
and includes the array field and the offset in the array field the nested hit belongs to. The offset is zero based.
Let’s see how it works with a real sample. Considering the following mapping:
resp = client.indices.create( index="sales", mappings={ "properties": { "tags": { "type": "keyword" }, "comments": { "type": "nested", "properties": { "username": { "type": "keyword" }, "comment": { "type": "text" } } } } }, ) print(resp)
response = client.indices.create( index: 'sales', body: { mappings: { properties: { tags: { type: 'keyword' }, comments: { type: 'nested', properties: { username: { type: 'keyword' }, comment: { type: 'text' } } } } } } ) puts response
const response = await client.indices.create({ index: "sales", mappings: { properties: { tags: { type: "keyword", }, comments: { type: "nested", properties: { username: { type: "keyword", }, comment: { type: "text", }, }, }, }, }, }); console.log(response);
PUT /sales { "mappings": { "properties": { "tags": { "type": "keyword" }, "comments": { "type": "nested", "properties": { "username": { "type": "keyword" }, "comment": { "type": "text" } } } } } }
And some documents:
resp = client.index( index="sales", id="1", refresh=True, document={ "tags": [ "car", "auto" ], "comments": [ { "username": "baddriver007", "comment": "This car could have better brakes" }, { "username": "dr_who", "comment": "Where's the autopilot? Can't find it" }, { "username": "ilovemotorbikes", "comment": "This car has two extra wheels" } ] }, ) print(resp)
response = client.index( index: 'sales', id: 1, refresh: true, body: { tags: [ 'car', 'auto' ], comments: [ { username: 'baddriver007', comment: 'This car could have better brakes' }, { username: 'dr_who', comment: "Where's the autopilot? Can't find it" }, { username: 'ilovemotorbikes', comment: 'This car has two extra wheels' } ] } ) puts response
const response = await client.index({ index: "sales", id: 1, refresh: "true", document: { tags: ["car", "auto"], comments: [ { username: "baddriver007", comment: "This car could have better brakes", }, { username: "dr_who", comment: "Where's the autopilot? Can't find it", }, { username: "ilovemotorbikes", comment: "This car has two extra wheels", }, ], }, }); console.log(response);
PUT /sales/_doc/1?refresh { "tags": [ "car", "auto" ], "comments": [ { "username": "baddriver007", "comment": "This car could have better brakes" }, { "username": "dr_who", "comment": "Where's the autopilot? Can't find it" }, { "username": "ilovemotorbikes", "comment": "This car has two extra wheels" } ] }
It’s now possible to execute the following top_hits
aggregation (wrapped in a nested
aggregation):
resp = client.search( index="sales", query={ "term": { "tags": "car" } }, aggs={ "by_sale": { "nested": { "path": "comments" }, "aggs": { "by_user": { "terms": { "field": "comments.username", "size": 1 }, "aggs": { "by_nested": { "top_hits": {} } } } } } }, ) print(resp)
response = client.search( index: 'sales', body: { query: { term: { tags: 'car' } }, aggregations: { by_sale: { nested: { path: 'comments' }, aggregations: { by_user: { terms: { field: 'comments.username', size: 1 }, aggregations: { by_nested: { top_hits: {} } } } } } } } ) puts response
const response = await client.search({ index: "sales", query: { term: { tags: "car", }, }, aggs: { by_sale: { nested: { path: "comments", }, aggs: { by_user: { terms: { field: "comments.username", size: 1, }, aggs: { by_nested: { top_hits: {}, }, }, }, }, }, }, }); console.log(response);
POST /sales/_search { "query": { "term": { "tags": "car" } }, "aggs": { "by_sale": { "nested": { "path": "comments" }, "aggs": { "by_user": { "terms": { "field": "comments.username", "size": 1 }, "aggs": { "by_nested": { "top_hits": {} } } } } } } }
Top hits response snippet with a nested hit, which resides in the first slot of array field comments
:
{ ... "aggregations": { "by_sale": { "by_user": { "buckets": [ { "key": "baddriver007", "doc_count": 1, "by_nested": { "hits": { "total" : { "value": 1, "relation": "eq" }, "max_score": 0.3616575, "hits": [ { "_index": "sales", "_id": "1", "_nested": { "field": "comments", "offset": 0 }, "_score": 0.3616575, "_source": { "comment": "This car could have better brakes", "username": "baddriver007" } } ] } } } ... ] } } } }
Name of the array field containing the nested hit |
|
Position if the nested hit in the containing array |
|
Source of the nested hit |
If _source
is requested then just the part of the source of the nested object is returned, not the entire source of the document.
Also stored fields on the nested inner object level are accessible via top_hits
aggregator residing in a nested
or reverse_nested
aggregator.
Only nested hits will have a _nested
field in the hit, non nested (regular) hits will not have a _nested
field.
The information in _nested
can also be used to parse the original source somewhere else if _source
isn’t enabled.
If there are multiple levels of nested object types defined in mappings then the _nested
information can also be hierarchical
in order to express the identity of nested hits that are two layers deep or more.
In the example below a nested hit resides in the first slot of the field nested_grand_child_field
which then resides in
the second slow of the nested_child_field
field:
... "hits": { "total" : { "value": 2565, "relation": "eq" }, "max_score": 1, "hits": [ { "_index": "a", "_id": "1", "_score": 1, "_nested" : { "field" : "nested_child_field", "offset" : 1, "_nested" : { "field" : "nested_grand_child_field", "offset" : 0 } } "_source": ... }, ... ] } ...
Use in pipeline aggregations
edittop_hits
can be used in pipeline aggregations that consume a single value per bucket, such as bucket_selector
that applies per bucket filtering, similar to using a HAVING clause in SQL. This requires setting size
to 1, and
specifying the right path for the value to be passed to the wrapping aggregator. The latter can be a _source
, a
_sort
or a _score
value. For example:
resp = client.search( index="sales", size="0", aggs={ "top_tags": { "terms": { "field": "type", "size": 3 }, "aggs": { "top_sales_hits": { "top_hits": { "sort": [ { "date": { "order": "desc" } } ], "_source": { "includes": [ "date", "price" ] }, "size": 1 } }, "having.top_salary": { "bucket_selector": { "buckets_path": { "tp": "top_sales_hits[_source.price]" }, "script": "params.tp < 180" } } } } }, ) print(resp)
const response = await client.search({ index: "sales", size: 0, aggs: { top_tags: { terms: { field: "type", size: 3, }, aggs: { top_sales_hits: { top_hits: { sort: [ { date: { order: "desc", }, }, ], _source: { includes: ["date", "price"], }, size: 1, }, }, "having.top_salary": { bucket_selector: { buckets_path: { tp: "top_sales_hits[_source.price]", }, script: "params.tp < 180", }, }, }, }, }, }); console.log(response);
POST /sales/_search?size=0 { "aggs": { "top_tags": { "terms": { "field": "type", "size": 3 }, "aggs": { "top_sales_hits": { "top_hits": { "sort": [ { "date": { "order": "desc" } } ], "_source": { "includes": [ "date", "price" ] }, "size": 1 } }, "having.top_salary": { "bucket_selector": { "buckets_path": { "tp": "top_sales_hits[_source.price]" }, "script": "params.tp < 180" } } } } } }
The bucket_path
uses the top_hits
name top_sales_hits
and a keyword for the field providing the aggregate value,
namely _source
field price
in the example above. Other options include top_sales_hits[_sort]
, for filtering on the
sort value date
above, and top_sales_hits[_score]
, for filtering on the score of the top hit.