logstash.yml
editlogstash.yml
editYou can set options in the Logstash settings file, logstash.yml, to control Logstash execution. For example,
you can specify pipeline settings, the location of configuration files, logging options, and other settings.
Most of the settings in the logstash.yml file are also available as command-line flags
when you run Logstash. Any flags that you set at the command line override the corresponding settings in the
logstash.yml file.
The logstash.yml file is written in YAML. Its location varies by platform (see
Logstash Directory Layout). You can specify settings in hierarchical form or use flat keys. For example, to use
hierarchical form to set the pipeline batch size and batch delay, you specify:
pipeline:
batch:
size: 125
delay: 50
To express the same values as flat keys, you specify:
pipeline.batch.size: 125 pipeline.batch.delay: 50
The logstash.yml file also supports bash-style interpolation of environment variables and
keystore secrets in setting values.
pipeline:
batch:
size: ${BATCH_SIZE}
delay: ${BATCH_DELAY:50}
node:
name: "node_${LS_NODE_NAME}"
path:
queue: "/tmp/${QUEUE_DIR:queue}"
Note that the ${VAR_NAME:default_value} notation is supported, setting a default batch delay
of 50 and a default path.queue of /tmp/queue in the above example.
Modules may also be specified in the logstash.yml file. The modules definition will have
this format:
modules:
- name: MODULE_NAME1
var.PLUGIN_TYPE1.PLUGIN_NAME1.KEY1: VALUE
var.PLUGIN_TYPE1.PLUGIN_NAME1.KEY2: VALUE
var.PLUGIN_TYPE2.PLUGIN_NAME2.KEY1: VALUE
var.PLUGIN_TYPE3.PLUGIN_NAME3.KEY1: VALUE
- name: MODULE_NAME2
var.PLUGIN_TYPE1.PLUGIN_NAME1.KEY1: VALUE
var.PLUGIN_TYPE1.PLUGIN_NAME1.KEY2: VALUE
If the command-line flag --modules is used, any modules defined in the logstash.yml file will be ignored.
The logstash.yml file includes the following settings.
| Setting | Description | Default value |
|---|---|---|
|
A descriptive name for the node. |
Machine’s hostname |
|
The directory that Logstash and its plugins use for any persistent needs. |
|
|
The ID of the pipeline. |
|
|
The number of workers that will, in parallel, execute the filter and output
stages of the pipeline. This setting uses the
|
Number of the host’s CPU cores |
|
The maximum number of events an individual worker thread will collect from inputs
before attempting to execute its filters and outputs.
Larger batch sizes are generally more efficient, but come at the cost of increased memory
overhead. You may need to increase JVM heap space in the |
|
|
When creating pipeline event batches, how long in milliseconds to wait for each event before dispatching an undersized batch to pipeline workers. |
|
|
When set to |
|
|
(Beta) Load Java plugins in independent classloaders to isolate their dependencies. |
|
|
Set the pipeline event ordering. Valid options are:
|
|
|
Sets the pipeline’s default value for
This option allows the early opt-in (or preemptive opt-out) of ECS compatibility modes in plugins, which is scheduled to be on-by-default in a future major release of Logstash. Values other than |
|
|
The path to the Logstash config for the main pipeline. If you specify a directory or wildcard, config files are read from the directory in alphabetical order. |
Platform-specific. See Logstash Directory Layout. |
|
A string that contains the pipeline configuration to use for the main pipeline. Use the same syntax as the config file. |
N/A |
|
When set to |
|
|
When set to |
|
|
How often in seconds Logstash checks the config files for changes. Note that the unit qualifier ( |
|
|
When set to |
|
|
When set to |
|
|
Provides a way to reference fields that contain field reference special characters This feature is in technical preview and may change in the future. Current options are:
|
|
|
When configured, |
N/A |
|
The internal queuing model to use for event buffering. Specify |
|
|
The directory path where the data files will be stored when persistent queues are enabled ( |
|
|
The size of the page data files used when persistent queues are enabled ( |
64mb |
|
The maximum number of unread events in the queue when persistent queues are enabled ( |
0 (unlimited) |
|
The total capacity of the queue ( |
1024mb (1g) |
|
The maximum number of ACKed events before forcing a checkpoint when persistent queues are enabled ( |
1024 |
|
The maximum number of written events before forcing a checkpoint when persistent queues are enabled ( |
1024 |
|
When enabled, Logstash will retry four times per attempted checkpoint write for any checkpoint writes that fail. Any subsequent errors are not retried. This is a workaround for failed checkpoint writes that have been seen only on Windows platform, filesystems with non-standard behavior such as SANs and is not recommended except in those specific circumstances. ( |
|
|
When enabled, Logstash waits until the persistent queue ( |
|
|
Flag to instruct Logstash to enable the DLQ feature supported by plugins. |
|
|
The maximum size of each dead letter queue. Entries will be dropped if they would increase the size of the dead letter queue beyond this setting. |
|
|
Defines the action to take when the dead_letter_queue.max_bytes setting is reached: |
|
|
The directory path where the data files will be stored for the dead-letter queue. |
|
|
The HTTP API is enabled by default. It can be disabled, but features that rely on it will not work as intended. |
|
|
The API returns the provided string as a part of its response. Setting your environment may help to disambiguate between similarly-named nodes in production vs test environments. |
|
|
The bind address for the HTTP API endpoint.
By default, the Logstash HTTP API binds only to the local loopback interface.
When configured securely ( |
|
|
The bind port for the HTTP API endpoint. |
|
|
Set to |
|
|
The path to a valid JKS or PKCS12 keystore for use in securing the Logstash API.
The keystore must be password-protected, and must contain a single certificate chain and a private key.
This setting is ignored unless |
N/A |
|
The password to the keystore provided with |
N/A |
|
List of allowed SSL/TLS versions to use when establishing a secure connection. The availability of protocols depends on the JVM version. Certain protocols are disabled by default and need to be enabled manually by changing
|
N/A |
|
Set to |
|
|
The username to require for HTTP Basic auth
Ignored unless |
N/A |
|
The password to require for HTTP Basic auth. Ignored unless
|
N/A |
|
Raises either |
|
|
The log level. Valid options are:
|
|
|
The log format. Set to |
|
|
When the log format is |
|
|
The directory where Logstash will write its log to. |
|
|
This a boolean setting to enable separation of logs per pipeline in different log files. If enabled Logstash will create a different log file for each pipeline, using the pipeline.id as name of the file. The destination directory is taken from the `path.log`s setting. When there are many pipelines configured in Logstash, separating each log lines per pipeline could be helpful in case you need to troubleshoot what’s happening in a single pipeline, without interference of the other ones. |
|
|
Where to find custom plugins. You can specify this setting multiple times to include
multiple paths. Plugins are expected to be in a specific directory hierarchy:
|
Platform-specific. See Logstash Directory Layout. |
|
Setting to |
|
|
Determine where to allocate memory buffers, for plugins that leverage them.
Defaults to |
|