Logs
editLogs
editElastic Java APM Agent provides the following log features:
- Log correlation : Automatically inject correlation IDs that allow navigation between logs, traces and services.
- Log reformatting (experimental) : Automatically reformat plaintext logs in ECS logging format.
-
Error capturing : Automatically captures exceptions for calls like
logger.error("message", exception). - Log sending (experimental) : Automatically send logs to APM Server without filebeat
Those features are part of Application log ingestion strategies.
The ecs-logging-java library can also be used to use the ECS logging format without an APM agent.
When deployed with the Java APM agent, the agent will provide log correlation IDs.
Log correlation
editLog correlation allows you to navigate to all logs belonging to a particular trace and vice-versa: for a specific log, see in which context it has been logged and which parameters the user provided.
Starting in APM agent version 1.30.0, log correlation is enabled by default.
In previous versions, log correlation must be explicitly enabled by setting
the enable_log_correlation configuration variable to true.
In order to correlate logs from your application with traces and errors captured by the Elastic APM Java Agent, the agent injects the following IDs into slf4j-MDC-equivalents of supported logging frameworks:
For frameworks that don’t provide an MDC like java.util.logging (JUL), correlation is only supported when using ECS logging library or
with Log reformatting.
For plain text logs, the pattern layout of your logging configuration needs to be modified to write the MDC values into
log files. If you are using Logback or log4j, add %X to the format to log all MDC values or %X{trace.id} to only log the trace id.
When the application is logging in ECS format (by using ecs-logging-java or log reformatting)
but does not provide the service fields, then the agent will automatically provide fallback values from its own configuration
to provide service-level correlation:
-
service.namevalue providedservice_namein agent config. -
service.versionvalue provided byservice_versionin agent config. -
service.environmentvalue provided byenvironmentin agent config.
Log reformatting (experimental)
editThe agent can automatically reformat application logs to ECS format, without adding a dependency to ecs-logging-java, modifying the application
logging configuration and making the application always use ECS log format. In short, it provides the benefits of ECS logging at runtime without any
change to the application.
Log reformatting is controlled by the log_ecs_reformatting configuration option, and is disabled by default.
The reformatted logs will include both the trace and service correlation IDs.
Error capturing
editThe agent automatically captures exceptions sent to loggers with calls like logger.error("message", exception).
When doing so, the error.id is added to the MDC as well for log correlation since 1.16.0.
As a result, when an exception is reported to the logger:
- The agent reports an error to APM server with the provided exception
-
An
error.idis generated and injected into logger MDC for the duration of the logger invocation -
Logger output will contain the
error.idif the log format allows it (plaintext still requires some configuration)
Please note we capture the exception, not the message passed to the logger.error.
To collect the message passed to the logger.error, you would need to ingest the logs of the application (see Log correlation).
Log sending (experimental)
editThe agent can automatically capture and send logs directly to APM Server, which allows to ingest log events without relying on filebeat.
Log sending is controlled by the log_sending configuration option and is disabled by default.