- Logstash Reference: other versions:
- Logstash Introduction
- Getting Started with Logstash
- How Logstash Works
- Setting Up and Running Logstash
- Logstash Directory Layout
- Logstash Configuration Files
- logstash.yml
- Secrets keystore for secure settings
- Running Logstash from the Command Line
- Running Logstash as a Service on Debian or RPM
- Running Logstash on Docker
- Configuring Logstash for Docker
- Running Logstash on Kubernetes
- Running Logstash on Windows
- Logging
- Shutting Down Logstash
- Upgrading Logstash
- Creating a Logstash pipeline
- Secure your connection
- Advanced Logstash Configurations
- Logstash-to-Logstash communication
- Managing Logstash
- Using Logstash with Elastic Integrations
- Working with Logstash Modules
- Working with Filebeat Modules
- Working with Winlogbeat Modules
- Queues and data resiliency
- Transforming Data
- Deploying and Scaling Logstash
- Managing GeoIP Databases
- Performance tuning
- Monitoring Logstash with Elastic Agent
- Monitoring Logstash (legacy)
- Monitoring Logstash with APIs
- Working with plugins
- Integration plugins
- Input plugins
- azure_event_hubs
- beats
- cloudwatch
- couchdb_changes
- dead_letter_queue
- elastic_agent
- elastic_serverless_forwarder
- elasticsearch
- exec
- file
- ganglia
- gelf
- generator
- github
- google_cloud_storage
- google_pubsub
- graphite
- heartbeat
- http
- http_poller
- imap
- irc
- java_generator
- java_stdin
- jdbc
- jms
- jmx
- kafka
- kinesis
- logstash
- log4j
- lumberjack
- meetup
- pipe
- puppet_facter
- rabbitmq
- redis
- relp
- rss
- s3
- s3-sns-sqs
- salesforce
- snmp
- snmptrap
- sqlite
- sqs
- stdin
- stomp
- syslog
- tcp
- udp
- unix
- varnishlog
- websocket
- wmi
- xmpp
- Output plugins
- boundary
- circonus
- cloudwatch
- csv
- datadog
- datadog_metrics
- dynatrace
- elastic_app_search
- elastic_workplace_search
- elasticsearch
- exec
- file
- ganglia
- gelf
- google_bigquery
- google_cloud_storage
- google_pubsub
- graphite
- graphtastic
- http
- influxdb
- irc
- java_stdout
- juggernaut
- kafka
- librato
- logstash
- loggly
- lumberjack
- metriccatcher
- mongodb
- nagios
- nagios_nsca
- opentsdb
- pagerduty
- pipe
- rabbitmq
- redis
- redmine
- riak
- riemann
- s3
- sink
- sns
- solr_http
- sqs
- statsd
- stdout
- stomp
- syslog
- tcp
- timber
- udp
- webhdfs
- websocket
- xmpp
- zabbix
- Filter plugins
- age
- aggregate
- alter
- bytes
- cidr
- cipher
- clone
- csv
- date
- de_dot
- dissect
- dns
- drop
- elapsed
- elastic_integration
- elasticsearch
- environment
- extractnumbers
- fingerprint
- geoip
- grok
- http
- i18n
- java_uuid
- jdbc_static
- jdbc_streaming
- json
- json_encode
- kv
- memcached
- metricize
- metrics
- mutate
- prune
- range
- ruby
- sleep
- split
- syslog_pri
- threats_classifier
- throttle
- tld
- translate
- truncate
- urldecode
- useragent
- uuid
- wurfl_device_detection
- xml
- Codec plugins
- Tips and best practices
- Troubleshooting
- Contributing to Logstash
- How to write a Logstash input plugin
- How to write a Logstash codec plugin
- How to write a Logstash filter plugin
- How to write a Logstash output plugin
- Logstash Plugins Community Maintainer Guide
- Document your plugin
- Publish your plugin to RubyGems.org
- List your plugin
- Contributing a patch to a Logstash plugin
- Extending Logstash core
- Contributing a Java Plugin
- Breaking changes
- Release Notes
- Logstash 8.17.1 Release Notes
- Logstash 8.17.0 Release Notes
- Logstash 8.16.3 Release Notes
- Logstash 8.16.2 Release Notes
- Logstash 8.16.1 Release Notes
- Logstash 8.16.0 Release Notes
- Logstash 8.15.5 Release Notes
- Logstash 8.15.4 Release Notes
- Logstash 8.15.3 Release Notes
- Logstash 8.15.2 Release Notes
- Logstash 8.15.1 Release Notes
- Logstash 8.15.0 Release Notes
- Logstash 8.14.3 Release Notes
- Logstash 8.14.2 Release Notes
- Logstash 8.14.1 Release Notes
- Logstash 8.14.0 Release Notes
- Logstash 8.13.4 Release Notes
- Logstash 8.13.3 Release Notes
- Logstash 8.13.2 Release Notes
- Logstash 8.13.1 Release Notes
- Logstash 8.13.0 Release Notes
- Logstash 8.12.2 Release Notes
- Logstash 8.12.1 Release Notes
- Logstash 8.12.0 Release Notes
- Logstash 8.11.4 Release Notes
- Logstash 8.11.3 Release Notes
- Logstash 8.11.2 Release Notes
- Logstash 8.11.1 Release Notes
- Logstash 8.11.0 Release Notes
- Logstash 8.10.4 Release Notes
- Logstash 8.10.3 Release Notes
- Logstash 8.10.2 Release Notes
- Logstash 8.10.1 Release Notes
- Logstash 8.10.0 Release Notes
- Logstash 8.9.2 Release Notes
- Logstash 8.9.1 Release Notes
- Logstash 8.9.0 Release Notes
- Logstash 8.8.2 Release Notes
- Logstash 8.8.1 Release Notes
- Logstash 8.8.0 Release Notes
- Logstash 8.7.1 Release Notes
- Logstash 8.7.0 Release Notes
- Logstash 8.6.2 Release Notes
- Logstash 8.6.1 Release Notes
- Logstash 8.6.0 Release Notes
- Logstash 8.5.3 Release Notes
- Logstash 8.5.2 Release Notes
- Logstash 8.5.1 Release Notes
- Logstash 8.5.0 Release Notes
- Logstash 8.4.2 Release Notes
- Logstash 8.4.1 Release Notes
- Logstash 8.4.0 Release Notes
- Logstash 8.3.3 Release Notes
- Logstash 8.3.2 Release Notes
- Logstash 8.3.1 Release Notes
- Logstash 8.3.0 Release Notes
- Logstash 8.2.3 Release Notes
- Logstash 8.2.2 Release Notes
- Logstash 8.2.1 Release Notes
- Logstash 8.2.0 Release Notes
- Logstash 8.1.3 Release Notes
- Logstash 8.1.2 Release Notes
- Logstash 8.1.1 Release Notes
- Logstash 8.1.0 Release Notes
- Logstash 8.0.1 Release Notes
- Logstash 8.0.0 Release Notes
- Logstash 8.0.0-rc2 Release Notes
- Logstash 8.0.0-rc1 Release Notes
- Logstash 8.0.0-beta1 Release Notes
- Logstash 8.0.0-alpha2 Release Notes
- Logstash 8.0.0-alpha1 Release Notes
Logstash-to-Logstash: Output to Input
editLogstash-to-Logstash: Output to Input
editThe Logstash output to Logstash input is the default approach for Logstash-to-Logstash communication.
Check out these considerations before you implement Logstash-to-Logstash.
Configuration overview
editTo connect two Logstash instances:
- Configure the downstream (server) Logstash to use Logstash input
- Configure the upstream (client) Logstash to use Logstash output
- Secure the communication between Logstash input and Logstash output
Configure the downstream Logstash to use Logstash input
editConfigure the Logstash input on the downstream (receiving) Logstash to receive connections. The minimum configuration requires this option:
-
port
- To set a custom port. The default is 9800 if none is provided.
input { logstash { port => 9800 } }
Configure the upstream Logstash to use Logstash output
editIn order to obtain the best performance when sending data from one Logstash to another, the data is batched and compressed. As such, the upstream Logstash (the sending Logstash) only needs to be concerned about configuring the receiving endpoint with these options:
-
hosts
- The receiving one or more Logstash host and port pairs. If no port specified, 9800 will be used.
Logstash load balances batched events to all of its configured downstream hosts. Any failures caused by network issues, back-pressure or other conditions, will result in the downstream host being isolated from load balancing for at least 60 seconds.
output { logstash { hosts => ["10.0.0.123", "10.0.1.123:9800"] } }
Secure Logstash to Logstash
editIt is important that you secure the communication between Logstash instances. Use SSL/TLS mutual authentication in order to ensure that the upstream Logstash instance sends encrypted data to a trusted downstream Logstash instance, and vice versa.
-
Create a certificate authority (CA) in order to sign the certificates that you plan to use between Logstash instances. Creating a correct SSL/TLS infrastructure is outside the scope of this document.
We recommend you use the elasticsearch-certutil tool to generate your certificates.
-
Configure the downstream (receiving) Logstash to use SSL. Add these settings to the Logstash input configuration:
-
ssl_enabled
: When set totrue
, it enables Logstash use of SSL/TLS -
ssl_key
: Specifies the key that Logstash uses to authenticate with the client. -
ssl_certificate
: Specifies the certificate that Logstash uses to authenticate with the client. -
ssl_certificate_authorities
: Configures Logstash to trust any certificates signed by the specified CA. -
ssl_client_authentication
: Specifies whether Logstash server verifies the client certificate against the CA.
For example:
input { logstash { ... ssl_enabled => true ssl_key => "server.pkcs8.key" ssl_certificate => "server.crt" ssl_certificate_authorities => "ca.crt" ssl_client_authentication => required } }
-
-
Configure the upstream (sending) Logstash to use SSL. Add these settings to the Logstash output configuration:
-
ssl_key
: Specifies the key the Logstash client uses to authenticate with the Logstash server. -
ssl_certificate
: Specifies the certificate that the Logstash client uses to authenticate to the Logstash server. -
ssl_certificate_authorities
: Configures the Logstash client to trust any certificates signed by the specified CA.
For example:
output { logstash { ... ssl_enabled => true ssl_key => "client.pkcs8.key" ssl_certificate => "client.crt" ssl_certificate_authorities => "ca.crt" } }
-
-
If you would like an additional authentication step, you can also use basic user/password authentication in both Logstash instances:
-
username
: Sets the username to use for authentication. -
password
: Sets the password to use for authentication.
For example, you would need to add the following to both Logstash instances:
... logstash { ... username => "your-user" password => "your-secret" } ...
-
On this page
ElasticON events are back!
Learn about the Elastic Search AI Platform from the experts at our live events.
Register now