Breaking changes in 8.0

Here are the breaking changes for 8.0.

Logstash must establish a Secure Sockets Layer (SSL) connection before it can transfer data to an on-premise Elasticsearch cluster. Logstash must have a copy of the Elasticsearch CA that signed the cluster’s certificates. When a new Elasticsearch cluster is started up without dedicated certificates, it generates its own default self-signed Certificate Authority at startup.

Our hosted Elasticsearch Service simplifies safe, secure communication between Logstash and Elasticsearch. Elasticsearch Service uses certificates signed by standard publicly trusted certificate authorities, and therefore setting a cacert is not necessary.

For more information, see Elasticsearch security on by default.

Logstash requires Java 11 or later. By default, Logstash will run with the bundled JDK, which has been verified to work with each specific version of Logstash, and generally provides the best performance and reliability.

We’ve removed support for using JAVA_HOME to override the path to the JDK. Users who need to use a version other than the bundled JDK should set the value of LS_JAVA_HOME to the path of their preferred JDK. The value of JAVA_HOME will be ignored.

Many plugins can now be run in a mode that avoids implicit conflict with the Elastic Common Schema. This mode is controlled individually with each plugin’s ecs_compatibility option, which defaults to the value of the Logstash pipeline.ecs_compatibility setting. In Logstash 8, this compatibility mode will be on-by-default for all pipelines. #11623

If you wish to lock in a pipeline’s behaviour from Logstash 7.x before upgrading to Logstash 8, you can set pipeline.ecs_compatibility: disabled to its definition in pipelines.yml (or globally in logstash.yml).

The Java Execution Engine has been the default engine since Logstash 7.0, and works with plugins written in either Ruby or Java. Removal of the Ruby Execution Engine will not affect the ability to run existing pipelines. #12517

We have added support for UTF-16 and other multi-byte-character when reading log files. #9702

The Field Reference parser interprets references to fields in your pipelines and plugins. Its behavior was configurable in 6.x, and 7.x allowed only a single option: strict. 8.0 no longer recognizes the setting, but maintains the same behavior as the strict setting. Logstash rejects ambiguous and illegal inputs as standard behavior.