Configure inputs
editConfigure inputs
editBy default, Journalbeat reads log events from the default systemd journals. To
specify other journal files, set the paths
option in
the journalbeat.inputs
section of the journalbeat.yml
file. Each path
can be a directory path (to collect events from all journals in a directory), or
a file path. For example:
journalbeat.inputs: - paths: - "/dev/log" - "/var/log/messages/my-journal-file.journal"
Within the configuration file, you can also specify options that control how Journalbeat reads the journal files and which fields are sent to the configured output. See Configuration options for a list of available options.
The following examples show how to configure Journalbeat for some common use cases.
Example 1: Monitor multiple journals under the same directoryThis example configures Journalbeat to read from multiple journals that are
stored under the same directory. Journalbeat merges all journals under the
directory into a single event stream and reads the events. With seek
set to
cursor
, Journalbeat starts reading at the beginning of the journal, but will
continue reading at the last known position after a reload or restart.
journalbeat.inputs: - paths: ["/path/to/journal/directory"] seek: cursor
Example 2: Fetch log events for Redis running on Docker (uses field names from systemd)This example configures Journalbeat to fetch log events for Redis running in a Docker container. The fields are matched using field names from the systemd journal.
journalbeat.inputs: - paths: [] include_matches: - "CONTAINER_TAG=redis" - "_COMM=redis"
Example 3: Fetch log events for Redis running on Docker (uses translated field names)This example also configures Journalbeat to fetch log events for Redis running in a Docker container. However, in this example the fields are matched using the translated field names provided by Journalbeat.
journalbeat.inputs: - paths: [] include_matches: - "container.image.tag=redis" - "process.name=redis"
Configuration options
editYou can specify the following options to configure how Journalbeat reads the journal files.
paths
editA list of paths that will be crawled and fetched. Each path can be a directory path (to collect events from all journals in a directory), or a file path. If you specify a directory, Journalbeat merges all journals under the directory into a single journal and reads them.
If no paths are specified, Journalbeat reads from the default journal.
backoff
editThe number of seconds to wait before trying to read again from journals. The default is 1s.
max_backoff
editThe maximum number of seconds to wait before attempting to read again from journals. The default is 60s.
seek
editThe position to start reading the journal from. Valid settings are:
-
head
: Starts reading at the beginning of the journal. After a restart, Journalbeat resends all log messages in the journal. -
tail
: Starts reading at the end of the journal. After a restart, Journalbeat resends the last message, which might result in duplicates. If multiple log messages are written to a journal while Journalbeat is down, only the last log message is sent on restart. -
cursor
: On first read, starts reading at the beginning of the journal. After a reload or restart, continues reading at the last known position.
When specified under paths
, the seek
setting applies to all journals under
the configured paths. When specified directly under the journalbeat
namespace, the setting applies to all journals read by Journalbeat.
If you have old log files and want to skip lines, start Journalbeat with
seek: tail
specified. Then stop Journalbeat, set seek: cursor
, and restart
Journalbeat.
include_matches
editA list of filter expressions used to match fields. The format of the expression
is field=value
. Journalbeat fetches all events that exactly match the
expressions. Pattern matching is not supported.
To reference fields, use one of the following:
-
The field name used by the systemd journal. For example,
CONTAINER_TAG=redis
(see a full example). -
The translated field name used by
Journalbeat. For example,
container.image.tag=redis
(see a full example). Journalbeat does not translate all fields from the journal. For custom fields, use the name specified in the systemd journal.
When specified under paths
, the include_matches
filter is applied to all
journals under the configured paths. When specified directly under the
journalbeat
namespace, the setting applies to all journals read by
Journalbeat.
Translated field names
editYou can use the following translated names in filter expressions to reference journald fields:
Journald field name |
Translated name |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The following translated fields for Docker are also available:
|
|
|
|
|
|
|
|
|
|
index
editIf present, this formatted string overrides the index for events from this input
(for elasticsearch outputs), or sets the raw_index
field of the event’s
metadata (for other outputs). This string can only refer to the agent name and
version and the event timestamp; for access to dynamic fields, use
output.elasticsearch.index
or a processor.
Example value: "%{[agent.name]}-myindex-%{+yyyy.MM.dd}"
might
expand to "journalbeat-myindex-2019.12.13"
.