Application logs

Application logs provide valuable insight into events that have occurred within your services and applications.

Logs are typically produced as either plaintext or structured. Plaintext logs contain only text and have no special formatting, for example:

2019-08-06T12:09:12.375Z INFO:spring-petclinic: Tomcat started on port(s): 8080 (http) with context path, org.springframework.boot.web.embedded.tomcat.TomcatWebServer
2019-08-06T12:09:12.379Z INFO:spring-petclinic: Started PetClinicApplication in 7.095 seconds (JVM running for 9.082), org.springframework.samples.petclinic.PetClinicApplication
2019-08-06T14:08:40.199Z DEBUG:spring-petclinic: init find form, org.springframework.samples.petclinic.owner.OwnerController

Structured logs, on the other hand, follow a predefined, repeatable pattern or structure. This structure is applied at write time—​preventing the need for parsing at ingest time. The Elastic Common Schema (ECS) defines a common set of fields to use when structuring logs. This structure allows logs to be easily ingested, and provides the ability to correlate, search, and aggregate on individual fields within your logs.

For example, the same example shown above might look like this when structured with ECS-compatable JSON:

{"@timestamp":"2019-08-06T12:09:12.375Z", "log.level": "INFO", "message":"Tomcat started on port(s): 8080 (http) with context path ''", "service.name":"spring-petclinic","process.thread.name":"restartedMain","log.logger":"org.springframework.boot.web.embedded.tomcat.TomcatWebServer"}
{"@timestamp":"2019-08-06T12:09:12.379Z", "log.level": "INFO", "message":"Started PetClinicApplication in 7.095 seconds (JVM running for 9.082)", "service.name":"spring-petclinic","process.thread.name":"restartedMain","log.logger":"org.springframework.samples.petclinic.PetClinicApplication"}
{"@timestamp":"2019-08-06T14:08:40.199Z", "log.level":"DEBUG", "message":"init find form", "service.name":"spring-petclinic","process.thread.name":"http-nio-8080-exec-8","log.logger":"org.springframework.samples.petclinic.owner.OwnerController","transaction.id":"28b7fb8d5aba51f1","trace.id":"2869b25b5469590610fea49ac04af7da"}

To correlate your logs, you must annotate your logs with the APM identifier, service.name. Elastic APM integrates with the most popular logging frameworks in each programming language to accomplish this automatically.

Correlating your application logs with trace events allows you to:

Learn more about log correlation in the APM Guide: log correlation, or in any of the agent-specific ingestion guides:

There are four main ways to ingest application logs into the Elastic Stack. Your use case will help determine the method that’s right for you.

Use Filebeat to parse and ingest raw, plain-text application logs.

Requirements

Pros

Cons

Learn more: Plaintext logs with Filebeat.

Elastic Common Schema (ECS) loggers format your logs into ECS-compatible JSON, removing the need to manually parse logs.

Requirements

Pros

Cons

Learn more: ECS logging with Filebeat

Elastic APM agents can automatically reformat application logs to Elastic Common Schema (ECS) format without needing to add an ECS logger dependency or modify the application.

Requirements

Pros

All the benefits of using ECS logging, without having to modify the application or its configuration:

Cons

Supported APM agents/languages

Learn more: APM agent ECS reformatting with Filebeat

Elastic APM agents can automatically capture and send logs directly to APM Server—​enabling you to easily ingest log events without needing a separate log shipper like Filebeat.

Requirements

The Elastic APM agent for your programming language.

Pros

Cons

Supported APM agents/languages

Learn more: APM agent log sending