This functionality is experimental and may be changed or removed completely in a future release. Elastic will take a best effort approach to fix any issues, but experimental features are not subject to the support SLA of official GA features.

« Journalbeat overview Set up and run Journalbeat »

›

Journalbeat quick start: installation and configuration

edit

Journalbeat quick start: installation and configuration

edit

This guide describes how to get started quickly with log data collection from systemd journals. You’ll learn how to:

install Journalbeat on each system you want to monitor
specify the location of your log files
parse log data into fields and send it to Elasticsearch
visualize the log data in Kibana

Before you begin

edit

You need Elasticsearch for storing and searching your data, and Kibana for visualizing and managing it.

To get started quickly, spin up a deployment of our hosted Elasticsearch Service. The Elasticsearch Service is available on AWS, GCP, and Azure. Try it out for free.

Step 1: Install Journalbeat

edit

Install Journalbeat on all the servers you want to monitor.

To download and install Journalbeat, use the commands that work with your system:

curl -L -O https://artifacts.elastic.co/downloads/beats/journalbeat/journalbeat-7.11.2-amd64.deb
sudo dpkg -i journalbeat-7.11.2-amd64.deb

curl -L -O https://artifacts.elastic.co/downloads/beats/journalbeat/journalbeat-7.11.2-x86_64.rpm
sudo rpm -vi journalbeat-7.11.2-x86_64.rpm

curl -L -O https://artifacts.elastic.co/downloads/beats/journalbeat/journalbeat-7.11.2-linux-x86_64.tar.gz
tar xzvf journalbeat-7.11.2-linux-x86_64.tar.gz

Other installation options

edit

Step 2: Connect to the Elastic Stack

edit

Connections to Elasticsearch and Kibana are required to set up Journalbeat.

Set the connection information in journalbeat.yml. To locate this configuration file, see Directory layout.

Specify the cloud.id of your Elasticsearch Service, and set cloud.auth to a user who is authorized to set up Journalbeat. For example:

cloud.id: "staging:dXMtZWFzdC0xLmF3cy5mb3VuZC5pbyRjZWM2ZjI2MWE3NGJmMjRjZTMzYmI4ODExYjg0Mjk0ZiRjNmMyY2E2ZDA0MjI0OWFmMGNjN2Q3YTllOTYyNTc0Mw=="
cloud.auth: "journalbeat_setup:YOUR_PASSWORD"

This examples shows a hard-coded password, but you should store sensitive values in the secrets keystore.

Set the host and port where Journalbeat can find the Elasticsearch installation, and set the username and password of a user who is authorized to set up Journalbeat. For example:
```
output.elasticsearch:
  hosts: ["myEShost:9200"]
  username: "journalbeat_internal"
  password: "YOUR_PASSWORD" 
```
This examples shows a hard-coded password, but you should store sensitive values in the secrets keystore.

If you plan to use our pre-built Kibana dashboards, configure the Kibana endpoint. Skip this step if Kibana is running on the same host as Elasticsearch.

  setup.kibana:
    host: "mykibanahost:5601" 
    username: "my_kibana_user"  
    password: "{pwd}"

	The hostname and port of the machine where Kibana is running, for example, `mykibanahost:5601`. If you specify a path after the port number, include the scheme and port: `http://mykibanahost:5601/path`.
	The `username` and `password` settings for Kibana are optional. If you don’t specify credentials for Kibana, Journalbeat uses the `username` and `password` specified for the Elasticsearch output.
	To use the pre-built Kibana dashboards, this user must be authorized to view dashboards or have the `kibana_admin` built-in role.

To learn more about required roles and privileges, see Grant users access to secured resources.

You can send data to other outputs, such as Logstash, but that requires additional configuration and setup.

Step 3: Configure Journalbeat

edit

Before running Journalbeat, specify the location of the systemd journal files and configure how you want the files to be read. If you accept the default configuration, Journalbeat reads from the local journal.

In journalbeat.yml, specify a list of paths to your systemd journal files. Each path can be a directory path (to collect events from all journals in a directory), or a file path. For example:
```
journalbeat.inputs:
- paths:
  - "/dev/log"
  - "/var/log/messages/my-journal-file.journal"
  seek: head
```
If no paths are specified, Journalbeat reads from the default journal.
Set the seek option to control the position where Journalbeat starts reading the journal. The available options are head, tail, and cursor. The default is cursor, which means that on first read, Journalbeat starts reading at the beginning of the file, but continues reading at the last known position after a reload or restart. For more detail about the settings, see the reference docs for the seek option.
(Optional) Set the include_matches option to filter entries in journald before collecting any log events. This reduces the number of events that Journalbeat needs to process. For example, to fetch only Redis events from a Docker container tagged as redis, use:
```
journalbeat.inputs:
- paths: []
  include_matches:
    - "CONTAINER_TAG=redis"
    - "_COMM=redis"
```

To test your configuration file, change to the directory where the Journalbeat binary is installed, and run Journalbeat in the foreground with the following options specified: ./journalbeat test config -e. Make sure your config files are in the path expected by Journalbeat (see Directory layout), or use the -c flag to specify the path to the config file.

For more information about configuring Journalbeat, also see:

Configure Journalbeat
Config file format
journalbeat.reference.yml: This reference configuration file shows all non-deprecated options. You’ll find it in the same location as journalbeat.yml.

Step 4: Set up assets

edit

Journalbeat comes with predefined assets for parsing, indexing, and visualizing your data. To load these assets:

Make sure the user specified in journalbeat.yml is authorized to set up Journalbeat.
From the installation directory, run:
```
journalbeat setup -e
```
```
journalbeat setup -e
```
```
./journalbeat setup -e
```
-e is optional and sends output to standard error instead of the configured log output.

This step loads the recommended index template for writing to Elasticsearch.

A connection to Elasticsearch (or Elasticsearch Service) is required to set up the initial environment. If you’re using a different output, such as Logstash, see Load the index template manually.

Step 5: Start Journalbeat

edit

Before starting Journalbeat, modify the user credentials in journalbeat.yml and specify a user who is authorized to publish events.

To start Journalbeat, run:

sudo service journalbeat start

If you use an init.d script to start Journalbeat, you can’t specify command line flags (see Command reference). To specify flags, start Journalbeat in the foreground.

Also see Journalbeat and systemd.

sudo service journalbeat start

If you use an init.d script to start Journalbeat, you can’t specify command line flags (see Command reference). To specify flags, start Journalbeat in the foreground.

Also see Journalbeat and systemd.

sudo chown root journalbeat.yml 
sudo ./journalbeat -e

You’ll be running Journalbeat as root, so you need to change ownership of the configuration file, or run Journalbeat with --strict.perms=false specified. See Config File Ownership and Permissions.

Journalbeat is now ready to send journal events to the Elasticsearch.

Step 6: View your data in Kibana

edit

There is currently no dashboard available for Journalbeat. To start exploring your data, go to the Discover app in Kibana. From there, you can submit search queries, filter the search results, and view document data.

To learn how to build visualizations and dashboards to view your data, see the Kibana User Guide.

What’s next?

edit

Now that you have your logs streaming into Elasticsearch, learn how to unify your logs, metrics, uptime, and application performance data.

Ingest data from other sources by installing and configuring other Elastic Beats:

Elastic Beats	To capture
Metricbeat	Infrastructure metrics
Filebeat	Logs
Winlogbeat	Windows event logs
Heartbeat	Uptime information
APM	Application performance metrics
Auditbeat	Audit events

Use the Observability apps in Kibana to search across all your data:

Elastic apps	Use to
Metrics app	Explore metrics about systems and services across your ecosystem
Logs app	Tail related log data in real time
Uptime app	Monitor availability issues across your apps and services
APM app	Monitor application performance
SIEM app	Analyze security events

The Logs app shows logs from filebeat-* indices by default. To show Journalbeat indices, configure the source to include journalbeat-*. You can do this in the Logs app when you configure the source, or you can modify the Kibana configuration. For example:

xpack.infra:
 sources:
   default:
     logAlias: "filebeat-*,journalbeat-*"

« Journalbeat overview Set up and run Journalbeat »