Retrieve a runtime field
editRetrieve a runtime field
editUse the fields
parameter on the _search
API to retrieve
the values of runtime fields. Runtime fields won’t display in _source
, but
the fields
API works for all fields, even those that were not sent as part of
the original _source
.
Define a runtime field to calculate the day of week
editFor example, the following request adds a runtime field called day_of_week
.
The runtime field includes a script that calculates the day of the week based
on the value of the @timestamp
field. We’ll include "dynamic":"runtime"
in
the request so that new fields are added to the mapping as runtime fields.
PUT my-index-000001/ { "mappings": { "dynamic": "runtime", "runtime": { "day_of_week": { "type": "keyword", "script": { "source": "emit(doc['@timestamp'].value.dayOfWeekEnum.getDisplayName(TextStyle.FULL, Locale.ROOT))" } } }, "properties": { "@timestamp": {"type": "date"} } } }
Ingest some data
editLet’s ingest some sample data, which will result in two indexed fields:
@timestamp
and message
.
POST /my-index-000001/_bulk?refresh { "index": {}} { "@timestamp": "2020-06-21T15:00:01-05:00", "message" : "211.11.9.0 - - [2020-06-21T15:00:01-05:00] \"GET /english/index.html HTTP/1.0\" 304 0"} { "index": {}} { "@timestamp": "2020-06-21T15:00:01-05:00", "message" : "211.11.9.0 - - [2020-06-21T15:00:01-05:00] \"GET /english/index.html HTTP/1.0\" 304 0"} { "index": {}} { "@timestamp": "2020-04-30T14:30:17-05:00", "message" : "40.135.0.0 - - [2020-04-30T14:30:17-05:00] \"GET /images/hm_bg.jpg HTTP/1.0\" 200 24736"} { "index": {}} { "@timestamp": "2020-04-30T14:30:53-05:00", "message" : "232.0.0.0 - - [2020-04-30T14:30:53-05:00] \"GET /images/hm_bg.jpg HTTP/1.0\" 200 24736"} { "index": {}} { "@timestamp": "2020-04-30T14:31:12-05:00", "message" : "26.1.0.0 - - [2020-04-30T14:31:12-05:00] \"GET /images/hm_bg.jpg HTTP/1.0\" 200 24736"} { "index": {}} { "@timestamp": "2020-04-30T14:31:19-05:00", "message" : "247.37.0.0 - - [2020-04-30T14:31:19-05:00] \"GET /french/splash_inet.html HTTP/1.0\" 200 3781"} { "index": {}} { "@timestamp": "2020-04-30T14:31:27-05:00", "message" : "252.0.0.0 - - [2020-04-30T14:31:27-05:00] \"GET /images/hm_bg.jpg HTTP/1.0\" 200 24736"} { "index": {}} { "@timestamp": "2020-04-30T14:31:29-05:00", "message" : "247.37.0.0 - - [2020-04-30T14:31:29-05:00] \"GET /images/hm_brdl.gif HTTP/1.0\" 304 0"} { "index": {}} { "@timestamp": "2020-04-30T14:31:29-05:00", "message" : "247.37.0.0 - - [2020-04-30T14:31:29-05:00] \"GET /images/hm_arw.gif HTTP/1.0\" 304 0"} { "index": {}} { "@timestamp": "2020-04-30T14:31:32-05:00", "message" : "247.37.0.0 - - [2020-04-30T14:31:32-05:00] \"GET /images/nav_bg_top.gif HTTP/1.0\" 200 929"} { "index": {}} { "@timestamp": "2020-04-30T14:31:43-05:00", "message" : "247.37.0.0 - - [2020-04-30T14:31:43-05:00] \"GET /french/images/nav_venue_off.gif HTTP/1.0\" 304 0"}
Search for the calculated day of week
editThe following request uses the search API to retrieve the day_of_week
field
that the original request defined as a runtime field in the mapping. The value
for this field is calculated dynamically at query time without reindexing
documents or indexing the day_of_week
field. This flexibility allows you to
modify the mapping without changing any field values.
GET my-index-000001/_search { "fields": [ "@timestamp", "day_of_week" ], "_source": false }
The previous request returns the day_of_week
field for all matching documents.
We can define another runtime field called client_ip
that also operates on
the message
field and will further refine the query:
PUT /my-index-000001/_mapping { "runtime": { "client_ip": { "type": "ip", "script" : { "source" : "String m = doc[\"message\"].value; int end = m.indexOf(\" \"); emit(m.substring(0, end));" } } } }
Run another query, but search for a specific IP address using the client_ip
runtime field:
GET my-index-000001/_search { "size": 1, "query": { "match": { "client_ip": "211.11.9.0" } }, "fields" : ["*"] }
This time, the response includes only two hits. The value for day_of_week
(Sunday
) was calculated at query time using the runtime script defined in the
mapping, and the result includes only documents matching the 211.11.9.0
IP
address.
{ ... "hits" : { "total" : { "value" : 2, "relation" : "eq" }, "max_score" : 1.0, "hits" : [ { "_index" : "my-index-000001", "_type" : "_doc", "_id" : "oWs5KXYB-XyJbifr9mrz", "_score" : 1.0, "_source" : { "@timestamp" : "2020-06-21T15:00:01-05:00", "message" : "211.11.9.0 - - [2020-06-21T15:00:01-05:00] \"GET /english/index.html HTTP/1.0\" 304 0" }, "fields" : { "@timestamp" : [ "2020-06-21T20:00:01.000Z" ], "client_ip" : [ "211.11.9.0" ], "message" : [ "211.11.9.0 - - [2020-06-21T15:00:01-05:00] \"GET /english/index.html HTTP/1.0\" 304 0" ], "day_of_week" : [ "Sunday" ] } } ] } }