Date nanoseconds field type
editDate nanoseconds field type
editThis data type is an addition to the date data type. However there is an
important distinction between the two. The existing date data type stores
dates in millisecond resolution. The date_nanos data type stores dates
in nanosecond resolution, which limits its range of dates from roughly
1970 to 2262, as dates are still stored as a long representing nanoseconds
since the epoch.
Queries on nanoseconds are internally converted to range queries on this long representation, and the result of aggregations and stored fields is converted back to a string depending on the date format that is associated with the field.
Date formats can be customised, but if no format is specified then it uses
the default:
"strict_date_optional_time_nanos||epoch_millis"
For instance:
response = client.indices.create(
index: 'my-index-000001',
body: {
mappings: {
properties: {
date: {
type: 'date_nanos'
}
}
}
}
)
puts response
response = client.bulk(
index: 'my-index-000001',
refresh: true,
body: [
{
index: {
_id: '1'
}
},
{
date: '2015-01-01'
},
{
index: {
_id: '2'
}
},
{
date: '2015-01-01T12:10:30.123456789Z'
},
{
index: {
_id: '3'
}
},
{
date: 1_420_070_400_000
}
]
)
puts response
response = client.search(
index: 'my-index-000001',
body: {
sort: {
date: 'asc'
},
runtime_mappings: {
date_has_nanos: {
type: 'boolean',
script: "emit(doc['date'].value.nano != 0)"
}
},
fields: [
{
field: 'date',
format: 'strict_date_optional_time_nanos'
},
{
field: 'date_has_nanos'
}
]
}
)
puts response
PUT my-index-000001
{
"mappings": {
"properties": {
"date": {
"type": "date_nanos"
}
}
}
}
PUT my-index-000001/_bulk?refresh
{ "index" : { "_id" : "1" } }
{ "date": "2015-01-01" }
{ "index" : { "_id" : "2" } }
{ "date": "2015-01-01T12:10:30.123456789Z" }
{ "index" : { "_id" : "3" } }
{ "date": 1420070400000 }
GET my-index-000001/_search
{
"sort": { "date": "asc"},
"runtime_mappings": {
"date_has_nanos": {
"type": "boolean",
"script": "emit(doc['date'].value.nano != 0)"
}
},
"fields": [
{
"field": "date",
"format": "strict_date_optional_time_nanos"
},
{
"field": "date_has_nanos"
}
]
}
|
The |
|
|
This document uses a plain date. |
|
|
This document includes a time. |
|
|
This document uses milliseconds-since-the-epoch. |
|
|
Note that the |
|
|
Use |
|
|
You can specify the format when fetching data using the |
You can also specify multiple date formats separated by ||. The
same mapping parameters than with the date field can be used.
Date nanoseconds
will accept numbers with a decimal point like {"date": 1618249875.123456}
but there are some cases (#70085) where we’ll lose precision
on those dates so they should be avoided.
Limitations
editAggregations are still on millisecond resolution, even when using a date_nanos
field. This limitation also affects transforms.
Synthetic _source
editSynthetic _source is Generally Available only for TSDB indices
(indices that have index.mode set to time_series). For other indices
synthetic _source is in technical preview. Features in technical preview may
be changed or removed in a future release. Elastic will work to fix
any issues, but features in technical preview are not subject to the support SLA
of official GA features.
date_nanos fields support synthetic _source in their
default configuration. Synthetic _source cannot be used together with
copy_to, ignore_malformed set to true
or with doc_values disabled.
Synthetic source always sorts date_nanos fields. For example:
response = client.indices.create(
index: 'idx',
body: {
mappings: {
_source: {
mode: 'synthetic'
},
properties: {
date: {
type: 'date_nanos'
}
}
}
}
)
puts response
response = client.index(
index: 'idx',
id: 1,
body: {
date: [
'2015-01-01T12:10:30.000Z',
'2014-01-01T12:10:30.000Z'
]
}
)
puts response
PUT idx
{
"mappings": {
"_source": { "mode": "synthetic" },
"properties": {
"date": { "type": "date_nanos" }
}
}
}
PUT idx/_doc/1
{
"date": ["2015-01-01T12:10:30.000Z", "2014-01-01T12:10:30.000Z"]
}
Will become:
{
"date": ["2014-01-01T12:10:30.000Z", "2015-01-01T12:10:30.000Z"]
}