IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Application logs ECS logging with Filebeat »
Elastic Docs ›Observability Guide [8.6] ›Application logs

Plaintext logs with Filebeat

edit

Plaintext logs with Filebeat

edit

Use Filebeat to parse and ingest raw, plain-text application logs.

Requirements

  • (Optional) Elastic APM agent for your programming language (for log correlation)
  • Raw, plain-text application logs stored on the file system
  • Filebeat configured to monitor and capture application logs

Pros

  • All programming languages/frameworks are supported
  • Existing application logs can be ingested
  • Does not require modification of the application or its configuration, unless log correlation is required

Cons

  • Must parse application logs to be useful—meaning writing and maintaining Grok patterns and spending CPU cycles on parsing
  • Parsing is tied to the application log format, meaning it can differ per application and needs to be maintained over time
  • Log correlation requires modifying the application log format and inject IDs in log messages

Step 1: Use Filebeat to ingest logs

edit
  1. Follow the Filebeat quick start to learn how to install Filebeat and connect to the Elastic Stack.
  2. Configure filebeat.yaml file to start collecting log data.

  3. Add the following configuration to your filebeat.yaml file to start collecting log data.

    filebeat.yaml.

    filebeat.inputs:
- type: filestream 
  paths: /path/to/logs.log

    Reads lines from an active log file.

    A list of glob-based paths that will be crawled and fetched.

Step 2: Parse logs at ingest time

edit

A downside of plaintext logs is that you can’t aggregate or search on the fields within the logs. To enable these features, you’ll need to parse the contents of your logs into ECS-compatible fields.

To learn how to use the Grok processor to parse application logs before indexing, see Example: Parse logs in the Common Log Format.

Step 3: Correlate your logs

edit

Correlating your application logs with trace events allows you to:

  • view the context of a log and the parameters a user provided
  • view all logs belonging to a particular trace
  • easily move between logs and traces when debugging application issues in Kibana

Learn more about log correlation in the APM Guide: log correlation, or in any of the agent-specific ingestion guides:

Step 4: View your logs in Kibana

edit

Use the APM or Logs UI to search, filter, and visualize your logs.

Learn more

edit
« Application logs ECS logging with Filebeat »