Logstash-to-Logstash communication
editLogstash-to-Logstash communication
editLogstash-to-Logstash communication is available if you need to have one Logstash instance communicate with another Logstash instance. Implementing Logstash-to-Logstash communication can add complexity to your environment, but you may need it if the data path crosses network or firewall boundaries. However, we suggest you don’t implement unless it is strictly required.
If you are looking for information on connecting multiple pipelines within one Logstash instance, see Pipeline-to-pipeline communication.
Logstash-to-Logstash communication can be achieved in one of two ways:
Logstash to Logstash considerations
This is the preferred method to implement Logstash-to-Logstash. It replaces Logstash-to-Logstash: HTTP output to HTTP input and has these considerations:
- It relies on HTTP as the communication protocol between the Input and Output.
- It does not provide built-in high availability. You will need to implement your own load balancer in between the Logstash output and the Logstash input.
- If you need a proxy between the Logstash instances, you can use any HTTP proxy.
- No connection information is added to events.
Ready to see more configuration details? See Logstash-to-Logstash: Output to Input.
Lumberjack-Beats considerations
Lumberjack output to Beats input has been our standard approach for Logstash-to-Logstash communication, but our recommended approach is now Logstash-to-Logstash: Output to Input. Before you implement the Lumberjack to Beats configuration, keep these points in mind:
- Lumberjack to Beats provides high availability, but does not provide load balancing. The Lumberjack output plugin allows defining multiple output hosts for high availability, but instead of load-balancing between all output hosts, it falls back to one host on the list in the case of failure.
- If you need a proxy between the Logstash instances, TCP proxy is the only option.
- There’s no explicit way to exert back pressure back to the beats input.
Ready to see more configuration details? See Logstash-to-Logstash: Lumberjack output to Beats input.