6.5.0 release highlights

See also Elasticsearch 6.5.0 release notes.

By default, when you enable auditing, it uses the logfile audit output, which persists events to a <clustername>_audit.log file in the logs directory. This file is new to version 6.5 and prints audit entries as JSON documents.

For backwards compatibility purposes, a <clustername>_access.log with the old style of formatting is also generated. See also Settings changes.

[preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. The new find file structure API determines the structure of a text file. It also returns statistics about the most common values of the detected fields and mappings that you can use when you ingest the file into Elasticsearch. If you want to influence the output, you can specify optional parameters. For example, if the file contains semi-structured text, you can specify a Grok pattern or let the tool generate it. This API is also used by the new File Data Visualizer in Kibana.

If you use the partition_field_name or by_field_name parameters in a machine learning job (or the Split Data field in the Kibana multi-metric job wizard), it generates many simultaneous analyses that are modeled independently. In 6.5, we have decreased the influence of the anomaly scores in each partition on other partitions' scores. As a result, record- and influencer-level results produce a much wider range of scores, but the overall anomaly score still uses sophisticated aggregation to ensure rate-limited alerts for the job.

Sometimes events are not interesting or anomalous in the context of a single bucket. However, they become interesting when you take into consideration a sequence of buckets as a whole. In 6.5, there is a new machine learning multi-bucket analysis, which uses features from multiple contiguous buckets for anomaly detection. The final anomaly score is now a combination of values from both the “standard” single-bucket analysis and the new multi-bucket analysis. A new multi_bucket_impact property in the record results indicates how strongly either form of analysis influences the score. In Kibana, anomalies with medium or high multi-bucket impact are depicted in the Anomaly Explorer and the Single Metric Viewer with a cross symbol instead of a dot.

Source-only snapshots save space on disk by only saving stored fields and index metadata. This can result in up to 50% reduction in size. Index and doc values are not included, so a restored source-only snapshot is not searchable. You must reindex the data into a new index after it’s restored.

The conditional token filter enables you to use inline Painless scripts to filter out terms. Previously, conditionally applying token filters required creating and using custom analysis plugins.

Elasticsearch SQL now supports ODBC mode in addition to JDBC. The functionality is currently the same as JDBC. An alpha version of the ODBC client is now available for download.

If you enable the Elasticsearch security features, some realms now have the ability to perform authentication internally then delegate authorization to another realm. For example, you could authenticate using PKI then delegate to an LDAP realm for role information. The realms that support this feature have a new authorization_realms setting that you can configure in the elasticsearch.yml file. For more information, see Realm chains and Realm settings.

Cross-cluster replication enables you to replicate indices that exist in remote clusters to your local cluster. You create an index in your local cluster (known as a follower index) that follows an index (known as a leader index) in a remote cluster. You can also automatically follow indices in a remote cluster that match a pattern. The individual write operations that occur on the leader indices are then replayed on the follower indices. This functionality is useful for replicating data to a second cluster for disaster recovery purposes and for geo-proximity so that reads can be served locally.

For more information, see Cross-cluster replication and Cross-cluster replication APIs.

In 6.4 and later, you can use Metricbeat to collect data about Kibana and ship it directly to your monitoring cluster, rather than routing it through Elasticsearch. Now in 6.5, you can also use Metricbeat to collect and ship data about Elasticsearch. If you are monitoring Logstash or Beats, at this time you must still use exporters to route the data. See Collecting monitoring data with Metricbeat and How monitoring works.

^* This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Please try this functionality in your test and development environments and provide feedback in the Elastic community forums.