What’s new in 7.11
editWhat’s new in 7.11
editHere are the highlights of what’s new and improved in Elasticsearch 7.11!
For detailed information about this release, see the Release notes and Migration guide.
Other versions: 7.10 | 7.9 | 7.8 | 7.7 | 7.6 | 7.5 | 7.4 | 7.3 | 7.2 | 7.1 | 7.0
Runtime fields
editTypically, you index data into Elasticsearch to promote faster search. However, indexing can be slow and requires more disk space, and you have to reindex your data to add fields to existing documents. With runtime fields, you can add fields to documents already indexed to Elasticsearch without reindexing your data.
You access runtime fields from the search API like any other field, and Elasticsearch sees runtime fields no differently. You can define runtime fields in the index mapping or in the search request. It’s your choice, which is part of the inherent flexibility of runtime fields.
See the runtime fields documentation to read more about their benefits and how to use them.
Speed improvements to the date histogram
editIn 7.10, we managed an 11% speed improvement in
the date histogram aggregation. In 7.11, we’ve
sped it up an additional 85% in some common cases - most
noteably when the date_histogram
is a top level aggregation and there aren’t
sub-aggregations. Kibana frequently uses date_histogram
in this way.
Cross-cluster replication (CCR) now supports data streams
editCCR is now aware of data streams. Followers now replicate full data streams, not just their backing indices.
New audit record for security configuration changes via API
editWe’ve added a new event.type
category for audit records, namely
the security_config_change
, in the existing audit trail. Events in this
category indicate that a security configuration has been set (e.g.,
a user or role was created or updated) or cleared (e.g., a user or role was deleted). The events
are emitted by default, but can be explicitly toggled by the
security_config_change
handler. The record contains all the change
details (e.g., the rules of the particular role mapping that has been
created or updated), but all credentials are redacted. The change
details are formatted as a JSON object and are part of audit record
structure. They are not JSON-escaped and put in a string field.
Sample audit log output:
{"type":"audit", "timestamp":"2020-09-26T12:58:13,369+0300", "node.id":"6BXu_9j6QPK7jvmOwzxqLQ", "event.type":"transport", "event.action":"access_granted", "user.name":"elastic", "user.realm":"reserved", "user.roles":["superuser"], "origin.type":"rest", "authentication.type":"REALM", "origin.address":"[::1]:50481", "request.id":"JLr1ftaoTuODAUZl-8g4Bg", "request.name":"PutUserRequest"} {"type":"audit", "timestamp":"2020-09-26T12:58:13,370+0300", "node.id":"6BXu_9j6QPK7jvmOwzxqLQ", "event.type":"security_config_change", "event.action":"put", "request.id":"JLr1ftaoTuODAUZl-8g4Bg", "config_change":{"put_user":{"username":"test_user2","roles":["superuser"],"full_name":"Joe Average","email":"joe.average@example.com","metadata":{"intelligence":7},"enabled":true,"password_hash":"<redacted>"}}} {"type":"audit", "timestamp":"2020-09-26T12:58:52,954+0300", "node.id":"6BXu_9j6QPK7jvmOwzxqLQ", "event.type":"transport", "event.action":"access_granted", "user.name":"elastic", "user.realm":"reserved", "user.roles":["superuser"], "origin.type":"rest", "authentication.type":"REALM", "origin.address":"[::1]:50482", "request.id":"i2XtJLCoRheGuwUdCXjDJw", "request.name":"PutRoleRequest"} {"type":"audit", "timestamp":"2020-09-26T12:58:52,955+0300", "node.id":"6BXu_9j6QPK7jvmOwzxqLQ", "event.type":"security_config_change", "event.action":"put", "request.id":"i2XtJLCoRheGuwUdCXjDJw", "config_change":{"put_role":{"name":"role_fls","cluster_privileges":["all"],"run_as":[],"indices_privileges":[{"names":["apm*"],"privileges":["read"],"field_security":{"grant":["granted"]},"query":"{\"term\": {\"service.name\": \"bar\"}}","allow_restricted_indices":false},{"names":["apm-all*"],"privileges":["all"],"query":"{\"term\": {\"service.name\": \"bar2\"}}","allow_restricted_indices":false}],"application_privileges":[],"metadata":{},"configurable_cluster_privileges":{}}}}
EQL: Wildcard and list lookup support for the :
operator
editEQL’s case-insensitive :
operator now supports the *
and ?
wildcards. The
*
wildcard matches zero or more characters. The ?
wildcard matches exactly
one character. You can also now use :
as a lookup operator to compare a string
value to a list of strings.
See the Wildcards and Lookup operators sections of the EQL syntax documentation.
New garbage collection defaults for small heaps
editOur benchmarks have demonstrated that when Elasticsearch is using a smaller heap size, it performs better with an alternative set of garbage collection options. Elasticsearch now ergonomically chooses different G1GC options for heap sizes smaller than, but not including, 8GB.
Data frame analytics is now beta!
editIn 7.11, we move data frame analytics from experimental to beta.
Data frame analytics enable you to perform different analyses of your data and annotate it with the results. Possible analysis types are outlier detection, regression, and classification. Data frame analytics evolved a lot while it was an experimental feature starting from 7.3. The aim is to move the feature to GA in the near future with the least possible breaking changes.
Latest document transform
edit
[beta]
This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.
As an alternative to the pivot
type of transform, you can now choose a
latest
type of transform. It enables you to copy the most recent documents
into a new index. You need to identify one or more fields as the unique key for
grouping your data, as well as a date field that sorts the data chronologically.
For example, you can use this type of transform to keep track of the latest
purchase for each customer or the latest event for each host.