Elasticsearch index engines
editElasticsearch index engines
editSince 8.2.0, Elastic Enterprise Search allows you to generate Engines based on existing Elasticsearch indices. With these engines you can use Enterprise Search features on data that has already been ingested into Elasticsearch.
Elasticsearch index engines features
editThe following table compares Enterprise Search managed engines with Elasticsearch index engines:
Feature |
Engine |
Elasticsearch Index Engine |
Get, List, Create, Update, Delete |
Get, List |
|
List, Update |
List |
|
Supported |
Supported for specific fields, see Elasticsearch engines text field conventions |
|
Usable as a source engine in meta engines |
Supported |
Supported |
Supported |
Supported |
|
Supported |
Supported |
|
Supported |
Supported |
|
Supported |
Supported |
|
Supported |
Supported |
|
Supported |
Supported |
|
Web Crawler |
Elasticsearch index engines supported field types
editApp Search engines do not support all Elasticsearch field types. Enterprise Search will ignore unsupported Elasticsearch field types.
Enterprise Search will not modify your data and index mappings.
Unsupported field types will not be removed or altered and will remain available for Elasticsearch features and APIs.
Enterprise Search supports the following Elasticsearch field types:
Elasticsearch field type |
Corresponding Enterprise Search field type |
keyword |
text |
constant_keyword |
text |
wildcard |
text |
text |
text |
geo_point |
geolocation |
date |
date |
date_nanos |
date |
integer |
number |
long |
number |
short |
number |
byte |
number |
double |
number |
float |
number |
half_float |
number |
scaled_float |
number |
unsigned_long |
number |
See Supported Features, By Field Type for a full App Search field types reference.
Unlike the text
field type in Enterprise Search, not all Elasticsearch text fields above have the same features as Enterprise Search text fields:
keyword
, constant_keyword
, wildcard
can be filtered, sorted, grouped and faceted. text
fields cannot.
Having a subfield for text
fields will provide those capabilities to the field.
Object fields
editElasticsearch has an object field type intended to support JSON inner objects.
In the following mapping the fields manager
and manager.name
are object fields:
{ "mappings": { "properties": { "manager": { "properties": { "age": { "type": "integer" }, "name": { "properties": { "first": { "type": "text" }, "last": { "type": "text" } } } } } } } }
When using such a mapping, subfields of the object fields are added to the schema of the Elasticsearch index engine:
{ "manager.age": "number", "manager.name.first": "text", "manager.name.last": "text" }
Object fields features
editUse subfields of an object just like any other standard field in Search API features, including in filters, facets, sorting, and groups.
Object fields in search results
editObject fields are rendered like any other field in search results:
{ "manager.age": { "raw": 30 }, "manager.name.first": { "raw": "John" }, "manager.name.last": { "raw": "Smith" } }
Elasticsearch flattens object field data.
Internally, Elasticsearch flattens object field data into multi-value fields.
Given the following document:
{ "manager": [ { "name": { "first" : "John", "last" : "Smith" } }, { "name": { "first" : "Alice", "last" : "White" } } ] }
Elasticsearch will store it internally this way:
{ "manager.name.first": ["John", "Alice"], "manager.mame.last": ["Smith", "White"] }
In order to stay consistent with how objects are internally stored, the App Search Search API returns flattened object fields:
"manager.name.first": { "raw": ["John", "Alice"] }, "manager.name.last": { "raw": ["Smith", "White"] }
To keep the independence of each object in the array and not lose the association between Alice
and White
, use the nested field type instead of the object field type.
Nested fields
editElasticsearch has a nested field type intended to support JSON inner objects. Unlike the object field type, Elasticsearch does not flatten nested object values into multi-value fields.
In the following mapping the field manager
is a nested object field:
{ "mappings": { "properties": { "manager": { "type": "nested", "properties": { "age": { "type": "integer" }, "name": { "properties": { "first": { "type": "text" }, "last": { "type": "text" } } } } } } } }
When using such a mapping, the nested field and its subfields are added to the Elasticsearch index engine schema in a flattened form:
{ "manager": "nested", "manager.age": "number", "manager.name.first": "text", "manager.name.last": "text" }
Nested fields filters
editIt is possible to use subfields of a nested field as filters in the Search API.
{ "query": "", "filters": { "manager.name.first": "Rich" } }
Nested fields limitations
editNested fields cannot be used as search_fields
in the Search API.
Nested fields do not contribute to full-text search.
Additionally, nested fields and their subfield are not usable with the following features of the Search API:
Searching, faceting, boosting, and sorting with nested fields
To include a nested field’s textual value in a full-text search query, you can copy the value of the subfield into the root of the document by remapping your Elasticsearch index using copy_to
.
{ "mappings": { "properties": { "manager_first_name": { "type": "text" }, "manager": { "type": "nested", "properties": { "age": { "type": "integer" }, "name": { "properties": { "first": { "type": "text", "copy_to": "manager_first_name" }, "last": { "type": "text" } } } } } } } }
By doing so, you create a new manager_first_name
field containing all the values of manager.name.first
.
Because the copied field exists outside of the nested object it can be used as a full-text search field.
You can use this technique to build facets, boosts, sort orders or groups from nested object subfields.
Nested fields in search results
editUnlike with the object field type, App Search does not flatten nested object values into multi-value fields in search results. App Search flattens objects but preserves the overall structure of the original document.
Given the following document:
{ "manager": [ { "name": { "first" : "John", "last" : "Smith" } }, { "name": { "first" : "Alice", "last" : "White" } } ] }
App Search will render it this way in Search API results:
{ "manager": [ { "name.first": { "raw": "John" }, "name.last": { "raw": "Smith" } }, { "name.first": { "raw": "Alice" }, "name.last": { "raw": "White" } } ] }
Because nested object subfields cannot be used in full-text search, using them as a snippet
in search results is not possible.
App Search will raise an error if you use a nested field or subfield as a snippet
.