Logging configuration migration

The entire logging system changed to offer a more consistent logging behavior with Elasticsearch, which uses the popular log4j 2 for managing logs. The current system differs from the more simplistic, deprecated system and allows you the freedom to customize what, how, and where Kibana outputs log messages. The best place to learn about the new logging system is our logging service guide. However, there are some important considerations when you are migrating from the old to the new system:

Kibana supports mixed usage of the deprecated and current logging configuration styles, however, to reduce the risk of confusion that may arise from conflicting configurations, we recommend changing over to the new system all at once.

If you use log rotation, be careful when mixing deprecated and current configuration styles as it could lead to unexpected behavior. For example, one system could apply the log file rotation while the other still writes to the original file. Rather, switch over to the current system completely.

If you are using the systemd service scripts (kibana.service), be aware that the deprecated logging.dest configuration might be set via CLI options in the kibana.service declaration. When logging.dest is set in the service scripts, Kibana will continue to write log messages to the "old" file. To switch over to the current logging system, remove the logging.dest CLI argument from /etc/systemd/system/kibana.service and run systemctl daemon-reload.

Compatibility with the legacy logging system is assured until the end of the v7 version. All log messages handled by root context are forwarded to the legacy logging service. If you configure a custom appender for root, make sure to include the default appender in the appenders array to provide backward compatibility.

When you switch to the new logging configuration, you will start seeing duplicate log entries in both legacy and current formats. The legacy format will be removed when the default appender is no longer required. To override this behavior for specific log messages, configure an appender for the logger.

If you log large volumes to file, such as logging in verbose mode, be aware that by using the legacy logging system, your cluster may experience increased memory consumption. We recommend switching over to the current logging system to avoid out of memory issues. Refer to the public issue for more information.