WARNING: Version 6.2 of Beats has passed its EOL date.
This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.
Install Logstash (Optional)
editInstall Logstash (Optional)
editThe simplest architecture for the Beats platform setup consists of one or more Beats, Elasticsearch, and Kibana. This architecture is easy to get started with and sufficient for networks with low traffic. It also uses the minimum amount of servers: a single machine running Elasticsearch and Kibana. The Beats insert the transactions directly into the Elasticsearch instance.
If you want to perform additional processing or buffering on the data, however, you’ll want to install Logstash.
An important advantage to this approach is that you can use Logstash to modify the data captured by Beats in any way you like. You can also use Logstash’s many output plugins to integrate with other systems.
To download and install Logstash, use the commands that work with your system:
deb:
sudo apt-get install openjdk-8-jre curl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-6.2.4.deb sudo dpkg -i logstash-6.2.4.deb
rpm:
sudo yum install java-1.8.0-openjdk curl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-6.2.4.rpm sudo rpm -i logstash-6.2.4.rpm
mac:
# install Java, e.g. from: https://www.java.com/en/download/manual.jsp curl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-6.2.4.zip unzip logstash-6.2.4.zip
win:
- If necessary, download and install the latest version of the Java from www.java.com.
- Download the Logstash 6.2.4 Windows zip file from the downloads page.
-
Extract the contents of the zip file to a directory on your computer, for example,
C:\Program Files
.
Don’t start Logstash yet. You need to set a couple of configuration options first.
Set up Logstash
editIn this setup, the Beat sends events to Logstash. Logstash receives these events by using the Beats input plugin for Logstash and then sends the transaction to Elasticsearch by using the Elasticsearch output plugin for Logstash. The Elasticsearch output plugin uses the bulk API, making indexing very efficient.
To set up Logstash, you create a Logstash pipeline configuration file that
configures Logstash to listen on port 5044 for incoming Beats connections
and to index into Elasticsearch. For example, you can save the following
example configuration to a file called logstash.conf
:
input { beats { port => 5044 } } # The filter part of this file is commented out to indicate that it is # optional. # filter { # # } output { elasticsearch { hosts => "localhost:9200" manage_template => false index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}" document_type => "%{[@metadata][type]}" } }
|
|
|
Starting with Logstash 6.0, the document_type
option is deprecated due to the
removal of types in Logstash 6.0.
It will be removed in the next major version of Logstash. If you are running
Logstash 6.0 or later, you do not need to set document_type
in your
configuration because Logstash sets the type to doc
by default.
When you run Logstash with this configuration, it indexes events into Elasticsearch in the same way that the Beat would, but you get access to other capabilities provided by Logstash for collecting, enriching, and transforming data. See the Logstash introduction for more information about these capabilities.
To use this setup, you’ll also need to configure your Beat to use Logstash. For more information, see the documentation for the Beat.
Update the Beats input plugin for Logstash
editPlugins have their own release cycle and are often released independent of Logstash’s core release cycle. To ensure that you have the latest version of the Beats input plugin for Logstash, run the following command from your Logstash installation:
deb, rpm, and mac:
./bin/logstash-plugin update logstash-input-beats
win:
bin\logstash-plugin update logstash-input-beats
Keep in mind that you can update to the latest version of the plugin without having to upgrade to a newer version of Logstash. More details about working with input plugins in Logstash are available here.
Start Logstash
editNow you can start Logstash. Use the command that works with your system. If you installed Logstash as a deb or rpm package, make sure the config file is in the expected directory.
deb:
sudo /etc/init.d/logstash start
rpm:
sudo service logstash start
mac:
./bin/logstash -f logstash.conf
win:
bin\logstash.bat -f logstash.conf
The default configuration for Beats and Logstash uses plain TCP. For encryption you must explicitly enable SSL when you configure Beats and Logstash.
You can learn more about installing, configuring, and running Logstash here.