Logstash output settings

backoff.init

(string) The number of seconds to wait before trying to reconnect to Logstash after a network error. After waiting backoff.init seconds, Elastic Agent tries to reconnect. If the attempt fails, the backoff timer is increased exponentially up to backoff.max. After a successful connection, the backoff timer is reset.

Default: 1s

backoff.max

(string) The maximum number of seconds to wait before attempting to connect to Elasticsearch after a network error.

Default: 60s

bulk_max_size

(int) The maximum number of events to bulk in a single Logstash request.

Events can be collected into batches. Elastic Agent will split batches larger than bulk_max_size into multiple batches.

Specifying a larger batch size can improve performance by lowering the overhead of sending events. However big batch sizes can also increase processing times, which might result in API errors, killed connections, timed-out publishing requests, and, ultimately, lower throughput.

Set this value to 0 to turn off the splitting of batches. When splitting is turned off, the queue determines the number of events to be contained in a batch.

Default: 2048

compression_level

(int) The gzip compression level. Set this value to 0 to disable compression. The compression level must be in the range of 1 (best speed) to 9 (best compression).

Increasing the compression level reduces network usage but increases CPU usage.

escape_html

(boolean) Configures escaping of HTML in strings. Set to true to enable escaping.

Default: false

index

(string) The index root name to write events to.

loadbalance

If true and multiple Logstash hosts are configured, the output plugin load balances published events onto all Logstash hosts. If false, the output plugin sends all events to one host (determined at random) and switches to another host if the selected one becomes unresponsive.

Default: false

Example:

outputs:
  default:
    type: logstash
    hosts: ["localhost:5044", "localhost:5045"]
    loadbalance: true

max_retries

(int) The number of times to retry publishing an event after a publishing failure. After the specified number of retries, the events are typically dropped.

Set max_retries to a value less than 0 to retry until all events are published.

Default: 3

pipelining

(int) The number of batches to send asynchronously to Logstash while waiting for an ACK from Logstash. The output becomes blocking after the specified number of batches are written. Specify 0 to turn off pipelining.

Default: 2

proxy_use_ local_resolver

(boolean) Determines whether Logstash hostnames are resolved locally when using a proxy. If false and a proxy is used, name resolution occurs on the proxy server.

Default: false

queue.mem.events

The number of events the queue can store. This value should be evenly divisible by queue.mem.flush.min_events to avoid sending partial batches to the output.

Default: 3200 events

queue.mem.flush.min_events

The minimum number of events required for publishing. If this value is set to 0 or 1, events are available to the output immediately. If this value is greater than 1 the output must wait for the queue to accumulate this minimum number of events or for queue.mem.flush.timeout to expire before publishing. When greater than 1 this value also defines the maximum possible batch that can be sent by the output.

Default: 1600 events

queue.mem.flush.timeout

(int) The maximum wait time for queue.mem.flush.min_events to be fulfilled. If set to 0s, events are available to the output immediately.

Default: 10s

slow_start

(boolean) If true, only a subset of events in a batch of events is transferred per transaction. The number of events to be sent increases up to bulk_max_size if no error is encountered. On error, the number of events per transaction is reduced again.

Default: false

timeout

(string) The number of seconds to wait for responses from the Logstash server before timing out.

Default: 30s

ttl

(string) Time to live for a connection to Logstash after which the connection will be reestablished. This setting is useful when Logstash hosts represent load balancers. Because connections to Logstash hosts are sticky, operating behind load balancers can lead to uneven load distribution across instances. Specify a TTL on the connection to achieve equal connection distribution across instances.

Default: 0 (turns off the feature)

worker

(int) The number of workers per configured host publishing events. Example: If you have two hosts and three workers, in total six workers are started (three for each host).

Default: 1