Secrets keystore for secure settings

When you configure Logstash, you might need to specify sensitive settings or configuration, such as passwords. Rather than relying on file system permissions to protect these values, you can use the Logstash keystore to securely store secret values for use in configuration settings.

After adding a key and its secret value to the keystore, you can use the key in place of the secret value when you configure sensitive settings.

The syntax for referencing keys is identical to the syntax for environment variables:

${KEY}

Where KEY is the name of the key.

Example

Imagine that the keystore contains a key called ES_PWD with the value yourelasticsearchpassword.

In configuration files, use:

output { elasticsearch {...password => "${ES_PWD}" } } }

In logstash.yml, use:

xpack.management.elasticsearch.password: ${ES_PWD}

Notice that the Logstash keystore differs from the Elasticsearch keystore. Whereas the Elasticsearch keystore lets you store elasticsearch.yml values by name, the Logstash keystore lets you specify arbitrary names that you can reference in the Logstash configuration.

When Logstash parses the settings (logstash.yml) or configuration (/etc/logstash/conf.d/*.conf), it resolves keys from the keystore before resolving environment variables.

You can protect access to the Logstash keystore by storing a password in an environment variable called LOGSTASH_KEYSTORE_PASS. If you create the Logstash keystore after setting this variable, the keystore will be password protected. This means that the environment variable needs to be accessible to the running instance of Logstash. This environment variable must also be correctly set for any users who need to issue keystore commands (add, list, remove, etc.).

Using a keystore password is recommended, but optional. The data will be encrypted even if you do not set a password. However, it is highly recommended to configure the keystore password and grant restrictive permissions to any files that may contain the environment variable value. If you choose not to set a password, then you can skip the rest of this section.

For example:

set +o history
export LOGSTASH_KEYSTORE_PASS=mypassword
set -o history
bin/logstash-keystore create

This setup requires the user running Logstash to have the environment variable LOGSTASH_KEYSTORE_PASS=mypassword defined. If the environment variable is not defined, Logstash cannot access the keystore.

When you run Logstash from an RPM or DEB package installation, the environment variables are sourced from /etc/sysconfig/logstash.

For other distributions, such as Docker or ZIP, see the documentation for your runtime environment (Windows, Docker, etc) to learn how to set the environment variable for the user that runs Logstash. Ensure that the environment variable (and thus the password) is only accessible to that user.

The keystore must be located in the Logstash path.settings directory. This is the same directory that contains the logstash.yml file. When performing any operation against the keystore, it is recommended to set path.settings for the keystore command. For example, to create a keystore on a RPM/DEB installation:

set +o history
export LOGSTASH_KEYSTORE_PASS=mypassword
set -o history
sudo -E /usr/share/logstash/bin/logstash-keystore --path.settings /etc/logstash create

See Logstash Directory Layout for more about the default directory locations.

The create command creates a new keystore or overwrites an existing keystore:

bin/logstash-keystore create

Creates the keystore in the directory defined in the path.settings setting.

To store sensitive values, such as authentication credentials for Elasticsearch, use the add command:

bin/logstash-keystore add ES_USER ES_PWD

When prompted, enter a value for each key.

To list the keys defined in the keystore, use:

bin/logstash-keystore list

To remove keys from the keystore, use:

bin/logstash-keystore remove ES_USER ES_PWD