Using Datetime in Painless
editUsing Datetime in Painless
editDatetime API
editDatetimes in Painless use the standard Java libraries and are available through the Painless Shared API. Most of the classes from the following Java packages are available to use in Painless scripts:
Datetime Representation
editDatetimes in Painless are most commonly represented as a numeric value, a string value, or a complex value.
- numeric
- a datetime representation as a number from a starting offset called an epoch; in Painless this is typically a long as milliseconds since an epoch of 1970-01-01 00:00:00 Zulu Time
- string
- a datetime representation as a sequence of characters defined by a standard format or a custom format; in Painless this is typically a String of the standard format ISO 8601
- complex
- a datetime representation as a complex type (object) that abstracts away internal details of how the datetime is stored and often provides utilities for modification and comparison; in Painless this is typically a ZonedDateTime
Switching between different representations of datetimes is often necessary to achieve a script’s objective(s). A typical pattern in a script is to switch a numeric or string datetime to a complex datetime, modify or compare the complex datetime, and then switch it back to a numeric or string datetime for storage or to return a result.
Datetime Parsing and Formatting
editDatetime parsing is a switch from a string datetime to a complex datetime, and datetime formatting is a switch from a complex datetime to a string datetime.
A DateTimeFormatter is a complex type (object) that defines the allowed sequence of characters for a string datetime. Datetime parsing and formatting often require a DateTimeFormatter. For more information about how to use a DateTimeFormatter see the Java documentation.
Datetime Parsing Examples
edit-
parse from milliseconds
String milliSinceEpochString = "434931330000"; long milliSinceEpoch = Long.parseLong(milliSinceEpochString); Instant instant = Instant.ofEpochMilli(milliSinceEpoch); ZonedDateTime zdt = ZonedDateTime.ofInstant(instant, ZoneId.of('Z'));
-
parse from ISO 8601
-
parse from RFC 1123
-
parse from a custom format
Datetime Formatting Examples
edit-
format to ISO 8601
-
format to a custom format
Datetime Conversion
editDatetime conversion is a switch from a numeric datetime to a complex datetime and vice versa.
Datetime Conversion Examples
edit-
convert from milliseconds
long milliSinceEpoch = 434931330000L; Instant instant = Instant.ofEpochMilli(milliSinceEpoch); ZonedDateTime zdt = ZonedDateTime.ofInstant(instant, ZoneId.of('Z'));
-
convert to milliseconds
ZonedDateTime zdt = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 0, ZoneId.of('Z')); long milliSinceEpoch = zdt.toInstant().toEpochMilli();
Datetime Pieces
editDatetime representations often contain the data to extract individual datetime pieces such as year, hour, timezone, etc. Use individual pieces of a datetime to create a complex datetime, and use a complex datetime to extract individual pieces.
Datetime Pieces Examples
edit-
create a complex datetime from pieces
int year = 1983; int month = 10; int day = 13; int hour = 22; int minutes = 15; int seconds = 30; int nanos = 0; ZonedDateTime zdt = ZonedDateTime.of( year, month, day, hour, minutes, seconds, nanos, ZoneId.of('Z'));
-
extract pieces from a complex datetime
ZonedDateTime zdt = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 100, ZoneId.of(tz)); int year = zdt.getYear(); int month = zdt.getMonthValue(); int day = zdt.getDayOfMonth(); int hour = zdt.getHour(); int minutes = zdt.getMinute(); int seconds = zdt.getSecond(); int nanos = zdt.getNano();
Datetime Modification
editUse either a numeric datetime or a complex datetime to do modification such as adding several seconds to a datetime or subtracting several days from a datetime. Use standard numeric operators to modify a numeric datetime. Use methods (or fields) to modify a complex datetime. Note many complex datetimes are immutable so upon modification a new complex datetime is created that requires assignment or immediate use.
Datetime Modification Examples
edit-
Subtract three seconds from a numeric datetime in milliseconds
long milliSinceEpoch = 434931330000L; milliSinceEpoch = milliSinceEpoch - 1000L*3L;
-
Add three days to a complex datetime
ZonedDateTime zdt = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 0, ZoneId.of('Z')); ZonedDateTime updatedZdt = zdt.plusDays(3);
-
Subtract 125 minutes from a complex datetime
ZonedDateTime zdt = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 0, ZoneId.of('Z')); ZonedDateTime updatedZdt = zdt.minusMinutes(125);
-
Set the year on a complex datetime
ZonedDateTime zdt = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 0, ZoneId.of('Z')); ZonedDateTime updatedZdt = zdt.withYear(1976);
Datetime Difference (Elapsed Time)
editUse either two numeric datetimes or two complex datetimes to calculate the difference (elapsed time) between two different datetimes. Use subtraction to calculate the difference between two numeric datetimes of the same time unit such as milliseconds. For complex datetimes there is often a method or another complex type (object) available to calculate the difference. Use ChronoUnit to calculate the difference between two complex datetimes if supported.
Datetime Difference Examples
edit-
Difference in milliseconds between two numeric datetimes
long startTimestamp = 434931327000L; long endTimestamp = 434931330000L; long differenceInMillis = endTimestamp - startTimestamp;
-
Difference in milliseconds between two complex datetimes
ZonedDateTime zdt1 = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 11000000, ZoneId.of('Z')); ZonedDateTime zdt2 = ZonedDateTime.of(1983, 10, 13, 22, 15, 35, 0, ZoneId.of('Z')); long differenceInMillis = ChronoUnit.MILLIS.between(zdt1, zdt2);
-
Difference in days between two complex datetimes
ZonedDateTime zdt1 = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 11000000, ZoneId.of('Z')); ZonedDateTime zdt2 = ZonedDateTime.of(1983, 10, 17, 22, 15, 35, 0, ZoneId.of('Z')); long differenceInDays = ChronoUnit.DAYS.between(zdt1, zdt2);
Datetime Comparison
editUse either two numeric datetimes or two complex datetimes to do a datetime comparison. Use standard comparison operators to compare two numeric datetimes of the same time unit such as milliseconds. For complex datetimes there is often a method or another complex type (object) available to do the comparison.
Datetime Comparison Examples
edit-
Greater than comparison of two numeric datetimes in milliseconds
long timestamp1 = 434931327000L; long timestamp2 = 434931330000L; if (timestamp1 > timestamp2) { // handle condition }
-
Equality comparison of two complex datetimes
ZonedDateTime zdt1 = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 0, ZoneId.of('Z')); ZonedDateTime zdt2 = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 0, ZoneId.of('Z')); if (zdt1.equals(zdt2)) { // handle condition }
-
Less than comparison of two complex datetimes
ZonedDateTime zdt1 = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 0, ZoneId.of('Z')); ZonedDateTime zdt2 = ZonedDateTime.of(1983, 10, 17, 22, 15, 35, 0, ZoneId.of('Z')); if (zdt1.isBefore(zdt2)) { // handle condition }
-
Greater than comparison of two complex datetimes
ZonedDateTime zdt1 = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 0, ZoneId.of('Z')); ZonedDateTime zdt2 = ZonedDateTime.of(1983, 10, 17, 22, 15, 35, 0, ZoneId.of('Z')); if (zdt1.isAfter(zdt2)) { // handle condition }
Datetime Zone
editBoth string datetimes and complex datetimes have a timezone with a default of
UTC
. Numeric datetimes do not have enough explicit information to
have a timezone, so UTC
is always assumed. Use
methods (or fields) in
conjunction with a ZoneId to change
the timezone for a complex datetime. Parse a string datetime into a complex
datetime to change the timezone, and then format the complex datetime back into
a desired string datetime. Note many complex datetimes are immutable so upon
modification a new complex datetime is created that requires
assignment or immediate use.
Datetime Zone Examples
edit-
Modify the timezone for a complex datetime
ZonedDateTime utc = ZonedDateTime.of(1983, 10, 13, 22, 15, 30, 0, ZoneId.of('Z')); ZonedDateTime pst = utc.withZoneSameInstant(ZoneId.of('America/Los_Angeles'));
-
Modify the timezone for a string datetime
String gmtString = 'Thu, 13 Oct 1983 22:15:30 GMT'; ZonedDateTime gmtZdt = ZonedDateTime.parse(gmtString, DateTimeFormatter.RFC_1123_DATE_TIME); ZonedDateTime pstZdt = gmtZdt.withZoneSameInstant(ZoneId.of('America/Los_Angeles')); String pstString = pstZdt.format(DateTimeFormatter.RFC_1123_DATE_TIME);
Datetime Input
editThere are several common ways datetimes are used as input for a script determined by the Painless context. Typically, datetime input will be accessed from parameters specified by the user, from an original source document, or from an indexed document.
Datetime Input From User Parameters
editUse the params section
during script specification to pass in a numeric datetime or string datetime as
a script input. Access to user-defined parameters within a script is dependent
on the Painless context, though, the parameters are most commonly accessible
through an input called params
.
Examples
-
Parse a numeric datetime from user parameters to a complex datetime
-
Input:
... "script": { ... "params": { "input_datetime": 434931327000 } } ...
-
Script:
long inputDateTime = params['input_datetime']; Instant instant = Instant.ofEpochMilli(inputDateTime); ZonedDateTime zdt = ZonedDateTime.ofInstant(instant, ZoneId.of('Z'));
-
-
Parse a string datetime from user parameters to a complex datetime
-
Input:
... "script": { ... "params": { "input_datetime": "custom y 1983 m 10 d 13 22:15:30 Z" } } ...
-
Script:
-
Datetime Input From a Source Document
editUse an original source document as a script
input to access a numeric datetime or string datetime for a specific field
within that document. Access to an original source document within a script is
dependent on the Painless context and is not always available. An original
source document is most commonly accessible through an input called
ctx['_source']
or params['_source']
.
Examples
-
Parse a numeric datetime from a sourced document to a complex datetime
-
Parse a string datetime from a sourced document to a complex datetime
Datetime Input From an Indexed Document
editUse an indexed document as a script input to access a complex datetime for a
specific field within that document where the field is mapped as a
standard date or a nanosecond date.
Numeric datetime fields mapped as numeric and string
datetime fields mapped as keyword are accessible through an
indexed document as well. Access to an indexed document within a script is
dependent on the Painless context and is not always available. An indexed
document is most commonly accessible through an input called doc
.
Examples
-
Format a complex datetime from an indexed document to a string datetime
-
Assumptions:
-
The field
input_datetime
exists in all indexes as part of the query -
All indexed documents contain the field
input_datetime
-
The field
-
Mappings:
{ "mappings": { ... "properties": { ... "input_datetime": { "type": "date" } ... } ... } }
-
Script:
-
-
Find the difference between two complex datetimes from an indexed document
-
Assumptions:
-
The fields
start
andend
may not exist in all indexes as part of the query -
The fields
start
andend
may not have values in all indexed documents
-
The fields
-
Mappings:
{ "mappings": { ... "properties": { ... "start": { "type": "date" }, "end": { "type": "date" } ... } ... } }
-
Script:
if (doc.containsKey('start') && doc.containsKey('end')) { if (doc['start'].size() > 0 && doc['end'].size() > 0) { ZonedDateTime start = doc['start'].value; ZonedDateTime end = doc['end'].value; long differenceInMillis = ChronoUnit.MILLIS.between(start, end); // handle difference in times } else { // handle fields without values } } else { // handle index with missing fields }
When a query’s results span multiple indexes, some indexes may not contain a specific field. Use the
containsKey
method call on thedoc
input to ensure a field exists as part of the index for the current document.Some fields within a document may have no values. Use the
size
method call on a field within thedoc
input to ensure that field has at least one value for the current document.
-
Datetime Now
editUnder most Painless contexts the current datetime, now
, is not supported.
There are two primary reasons for this. The first is that scripts are often run once
per document, so each time the script is run a different now
is returned. The
second is that scripts are often run in a distributed fashion without a way to
appropriately synchronize now
. Instead, pass in a user-defined parameter with
either a string datetime or numeric datetime for now
. A numeric datetime is
preferred as there is no need to parse it for comparison.
Datetime Now Examples
edit-
Use a numeric datetime as
now
-
Assumptions:
-
The field
input_datetime
exists in all indexes as part of the query -
All indexed documents contain the field
input_datetime
-
The field
-
Mappings:
{ "mappings": { ... "properties": { ... "input_datetime": { "type": "date" } ... } ... } }
-
Input:
... "script": { ... "params": { "now": <generated numeric datetime in milliseconds since epoch> } } ...
-
Script:
long now = params['now']; ZonedDateTime inputDateTime = doc['input_datetime']; long millisDateTime = zdt.toInstant().toEpochMilli(); long elapsedTime = now - millisDateTime;
-
-
Use a string datetime as
now
-
Assumptions:
-
The field
input_datetime
exists in all indexes as part of the query -
All indexed documents contain the field
input_datetime
-
The field
-
Mappings:
{ "mappings": { ... "properties": { ... "input_datetime": { "type": "date" } ... } ... } }
-
Input:
... "script": { ... "params": { "now": "<generated string datetime in ISO-8601>" } } ...
-
Script:
-
Datetime Examples in Contexts
editLoad the Example Data
editRun the following curl commands to load the data necessary for the context examples into an Elasticsearch cluster:
-
Create mappings for the sample data.
PUT /messages { "mappings": { "properties": { "priority": { "type": "integer" }, "datetime": { "type": "date" }, "message": { "type": "text" } } } }
-
Load the sample data.
POST /_bulk { "index" : { "_index" : "messages", "_id" : "1" } } { "priority": 1, "datetime": "2019-07-17T12:13:14Z", "message": "m1" } { "index" : { "_index" : "messages", "_id" : "2" } } { "priority": 1, "datetime": "2019-07-24T01:14:59Z", "message": "m2" } { "index" : { "_index" : "messages", "_id" : "3" } } { "priority": 2, "datetime": "1983-10-14T00:36:42Z", "message": "m3" } { "index" : { "_index" : "messages", "_id" : "4" } } { "priority": 3, "datetime": "1983-10-10T02:15:15Z", "message": "m4" } { "index" : { "_index" : "messages", "_id" : "5" } } { "priority": 3, "datetime": "1983-10-10T17:18:19Z", "message": "m5" } { "index" : { "_index" : "messages", "_id" : "6" } } { "priority": 1, "datetime": "2019-08-03T17:19:31Z", "message": "m6" } { "index" : { "_index" : "messages", "_id" : "7" } } { "priority": 3, "datetime": "2019-08-04T17:20:00Z", "message": "m7" } { "index" : { "_index" : "messages", "_id" : "8" } } { "priority": 2, "datetime": "2019-08-04T18:01:01Z", "message": "m8" } { "index" : { "_index" : "messages", "_id" : "9" } } { "priority": 3, "datetime": "1983-10-10T19:00:45Z", "message": "m9" } { "index" : { "_index" : "messages", "_id" : "10" } } { "priority": 2, "datetime": "2019-07-23T23:39:54Z", "message": "m10" }
Day-of-the-Week Bucket Aggregation Example
editThe following example uses a terms aggregation as part of the bucket script aggregation context to display the number of messages from each day-of-the-week.
GET /messages/_search?pretty=true { "aggs": { "day-of-week-count": { "terms": { "script": "return doc[\"datetime\"].value.getDayOfWeekEnum();" } } } }
Morning/Evening Bucket Aggregation Example
editThe following example uses a terms aggregation as part of the bucket script aggregation context to display the number of messages received in the morning versus the evening.
GET /messages/_search?pretty=true { "aggs": { "am-pm-count": { "terms": { "script": "return doc[\"datetime\"].value.getHour() < 12 ? \"AM\" : \"PM\";" } } } }
Age of a Message Script Field Example
editThe following example uses a script field as part of the field context to display the elapsed time between "now" and when a message was received.
GET /_search?pretty=true { "query": { "match_all": {} }, "script_fields": { "message_age": { "script": { "source": "ZonedDateTime now = ZonedDateTime.ofInstant(Instant.ofEpochMilli(params[\"now\"]), ZoneId.of(\"Z\")); ZonedDateTime mdt = doc[\"datetime\"].value; String age; long years = mdt.until(now, ChronoUnit.YEARS); age = years + \"Y \"; mdt = mdt.plusYears(years); long months = mdt.until(now, ChronoUnit.MONTHS); age += months + \"M \"; mdt = mdt.plusMonths(months); long days = mdt.until(now, ChronoUnit.DAYS); age += days + \"D \"; mdt = mdt.plusDays(days); long hours = mdt.until(now, ChronoUnit.HOURS); age += hours + \"h \"; mdt = mdt.plusHours(hours); long minutes = mdt.until(now, ChronoUnit.MINUTES); age += minutes + \"m \"; mdt = mdt.plusMinutes(minutes); long seconds = mdt.until(now, ChronoUnit.SECONDS); age += hours + \"s\"; return age;", "params": { "now": 1574005645830 } } } } }
The following shows the script broken into multiple lines:
ZonedDateTime now = ZonedDateTime.ofInstant( Instant.ofEpochMilli(params['now']), ZoneId.of('Z')); ZonedDateTime mdt = doc['datetime'].value; String age; long years = mdt.until(now, ChronoUnit.YEARS); age = years + 'Y '; mdt = mdt.plusYears(years); long months = mdt.until(now, ChronoUnit.MONTHS); age += months + 'M '; mdt = mdt.plusMonths(months); long days = mdt.until(now, ChronoUnit.DAYS); age += days + 'D '; mdt = mdt.plusDays(days); long hours = mdt.until(now, ChronoUnit.HOURS); age += hours + 'h '; mdt = mdt.plusHours(hours); long minutes = mdt.until(now, ChronoUnit.MINUTES); age += minutes + 'm '; mdt = mdt.plusMinutes(minutes); long seconds = mdt.until(now, ChronoUnit.SECONDS); age += hours + 's'; return age;
Parse the datetime "now" as input from the user-defined params. |
|
Store the datetime the message was received as a |
|
Find the difference in years between "now" and the datetime the message was received. |
|
Add the difference in years later returned in the format
|
|
Add the years so only the remainder of the months, days, etc. remain as the difference between "now" and the datetime the message was received. Repeat this pattern until the desired granularity is reached (seconds in this example). |
|
Return the age of the message in the format
|