WARNING: Version 5.1 of Elasticsearch has passed its EOL date.
This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.
Mapping changes
editMapping changes
editstring
fields replaced by text
/keyword
fields
editThe string
field datatype has been replaced by the text
field for full
text analyzed content, and the keyword
field for not-analyzed exact string
values. For backwards compatibility purposes, during the 5.x series:
-
string
fields on pre-5.0 indices will function as before. -
New
string
fields can be added to pre-5.0 indices as before. -
text
andkeyword
fields can also be added to pre-5.0 indices. -
When adding a
string
field to a new index, the field mapping will be rewritten as atext
orkeyword
field if possible, otherwise an exception will be thrown. Certain configurations that were possible withstring
fields are no longer possible withtext
/keyword
fields such as enablingterm_vectors
on a not-analyzedkeyword
field.
Default string mappings
editString mappings now have the following default mappings:
{ "type": "text", "fields": { "keyword": { "type": "keyword", "ignore_above": 256 } } }
This allows to perform full-text search on the original field name and to sort and run aggregations on the sub keyword field.
Numeric fields
editNumeric fields are now indexed with a completely different data-structure, called BKD tree, that is expected to require less disk space and be faster for range queries than the previous way that numerics were indexed.
Term queries will return constant scores now, while they used to return higher scores for rare terms due to the contribution of the document frequency, which this new BKD structure does not record. If scoring is needed, then it is advised to map the numeric fields as `keyword`s too.
Note that this keyword
mapping do not need to replace the numeric
mapping. For instance if you need both sorting and scoring on your numeric field,
you could map it both as a number and a keyword
using fields
:
PUT my_index { "mappings": { "my_type": { "properties": { "my_number": { "type": "long", "fields": { "keyword": { "type": "keyword" } } } } } } }
Also the precision_step
parameter is now irrelevant and will be rejected on
indices that are created on or after 5.0.
geo_point
fields
editLike Numeric fields the Geo point field now uses the new BKD tree structure. Since
this structure is fundamentally designed for multi-dimension spatial data, the
following field parameters are no longer needed or supported: geohash
,
geohash_prefix
, geohash_precision
, lat_lon
. Geohashes are still supported from
an API perspective, and can still be accessed using the .geohash
field extension,
but they are no longer used to index geo point data.
_timestamp
and _ttl
editThe _timestamp
and _ttl
fields were deprecated and are now removed. As a
replacement for _timestamp
, you should populate a regular date field with the
current timestamp on application side. For _ttl
, you should either use
time-based indices when applicable, or cron a delete-by-query with a range
query on a timestamp field
index
property
editOn all field datatypes (except for the deprecated string
field), the index
property now only accepts true
/false
instead of not_analyzed
/no
. The
string
field still accepts analyzed
/not_analyzed
/no
.
Doc values on unindexed fields
editPreviously, setting a field to index:no
would also disable doc-values. Now,
doc-values are enabled by default on all types but text
and binary
,
regardless of the value of the index
property.
Floating points use float
instead of double
editWhen dynamically mapping a field containing a floating point number, the field
now defaults to using float
instead of double
. The reasoning is that
floats should be more than enough for most cases but would decrease storage
requirements significantly.
norms
editnorms
now take a boolean instead of an object. This boolean is the replacement
for norms.enabled
. There is no replacement for norms.loading
since eager
loading of norms is not useful anymore now that norms are disk-based.
fielddata.format
editSetting fielddata.format: doc_values
in the mappings used to implicitly
enable doc-values on a field. This no longer works: the only way to enable or
disable doc-values is by using the doc_values
property of mappings.
fielddata.filter.regex
editRegex filters are not supported anymore and will be dropped on upgrade.
Source-transform removed
editThe source transform
feature has been removed. Instead, use an ingest pipeline.
Field mapping limits
editTo prevent mapping explosions, the following limits are applied to indices created in 5.x:
- The maximum number of fields in an index is limited to 1000.
-
The maximum depth for a field (1 plus the number of
object
ornested
parents) is limited to 20. -
The maximum number of
nested
fields in an index is limited to 50.
See Settings to prevent mappings explosion for more.
_parent
field no longer indexed
editThe join between parent and child documents no longer relies on indexed fields
and therefore from 5.0.0 onwards the _parent
field is no longer indexed. In
order to find documents that refer to a specific parent id, the new
parent_id
query can be used. The GET response and hits inside the search
response still include the parent id under the _parent
key.
Source format
option
editThe _source
mapping no longer supports the format
option. It will still be
accepted for indices created before the upgrade to 5.0 for backwards
compatibility, but it will have no effect. Indices created on or after 5.0
will reject this option.
Object notation
editCore types no longer support the object notation, which was used to provide per document boosts as follows:
{ "value": "field_value", "boost": 42 }
Boost accuracy for queries on _all
editPer-field boosts on the _all
are now compressed into a single byte instead
of the 4 bytes used previously. While this will make the index much more
space-efficient, it also means that index time boosts will be less accurately
encoded.
_ttl
and _timestamp
cannot be created
editYou can no longer create indexes with _ttl
or _timestamp
enabled. Indexes
with them enabled created before 5.0 will continue to work.
You should replace _timestamp
in new indexes by adding a field to your source
either in the application producing the data or with an ingest pipline like
this one:
PUT _ingest/pipeline/timestamp { "description" : "Adds a timestamp field at the current time", "processors" : [ { "set" : { "field": "timestamp", "value": "{{_ingest.timestamp}}" } } ] } PUT newindex/type/1?pipeline=timestamp { "example": "data" } GET newindex/type/1
Which produces
{ "_source": { "example": "data", "timestamp": "2016-06-21T18:48:55.560+0000" }, ... }
If you have an old index created with 2.x that has _timestamp
enabled then
you can migrate it to a new index with the a timestamp
field in the source
with reindex:
POST _reindex { "source": { "index": "oldindex" }, "dest": { "index": "newindex" }, "script": { "lang": "painless", "inline": "ctx._source.timestamp = ctx._timestamp; ctx._timestamp = null" } }
You can replace _ttl
with time based index names (preferred) or by adding a
cron job which runs a delete-by-query on a timestamp field in the source
document. If you had documents like this:
POST index/type/_bulk {"index":{"_id":1}} {"example": "data", "timestamp": "2016-06-21T18:48:55.560+0000" } {"index":{"_id":2}} {"example": "data", "timestamp": "2016-04-21T18:48:55.560+0000" }
Then you could delete all of the documents from before June 1st with:
POST index/type/_delete_by_query { "query": { "range" : { "timestamp" : { "lt" : "2016-05-01" } } } }
Keep in mind that deleting documents from an index is very expensive
compared to deleting whole indexes. That is why time based indexes are
recommended over this sort of thing and why _ttl
was deprecated in the first
place.
Blank field names is not supported
editBlank field names in mappings is not allowed after 5.0.