Encrypting communications in Elasticsearch
editEncrypting communications in Elasticsearch
editElastic Stack security features enable you to encrypt traffic to, from, and within your Elasticsearch cluster. Connections are secured using Transport Layer Security (TLS/SSL).
Clusters that do not have encryption enabled send all data in plain text including passwords. If the Elasticsearch security features are enabled, unless you have a trial license, you must configure SSL/TLS for internode-communication.
To enable encryption, you need to perform the following steps on each node in the cluster:
-
Verify that the
xpack.security.enabled
setting istrue
. For more information, see Security settings. - Generate a private key and X.509 certificate.
-
Configure each node to:
- Required: Enable TLS on the transport layer.
- Recommended: Enable TLS on the HTTP layer.
- If you are using Active Directory user authentication, encrypt communications between Elasticsearch and your Active Directory server.
- If you are using LDAP user authentication, encrypt communications between Elasticsearch and your LDAP server.
For more information about encrypting communications across the Elastic Stack, see Encrypting communications.
Generating node certificates
editTLS requires X.509 certificates to perform encryption and authentication of the application that is being communicated with. In order for the communication between nodes to be truly secure, the certificates must be validated. The recommended approach for validating certificate authenticity in an Elasticsearch cluster is to trust the certificate authority (CA) that signed the certificate. By doing this, as nodes are added to your cluster they just need to use a certificate signed by the same CA and the node is automatically allowed to join the cluster. Additionally, it is recommended that the certificates contain subject alternative names (SAN) that correspond to the node’s IP address and DNS name so that hostname verification can be performed.
The elasticsearch-certutil
command simplifies the process
of generating certificates for the Elastic Stack. It takes care of generating a CA and
signing certificates with the CA. It can be used interactively or in a silent
mode through the use of an input file. It also supports generation of
certificate signing requests (CSR), so that a commercial- or
organization-specific CA can be used to sign the certificates.
If you choose not to use elasticsearch-certutil
, the certificates that
you obtain must allow for both clientAuth
and serverAuth
if the extended key
usage extension is present. The certificates need to be in PEM or PKCS#12
format. Although not required, it is highly recommended that the certificate
contain the DNS names and/or IP addresses of the node so that hostname
verification can be used.
-
Optional: Create a certificate authority for your Elasticsearch cluster.
Use the
elasticsearch-certutil ca
command:./bin/elasticsearch-certutil ca
You can configure the cluster to trust all nodes that have a certificate that has been signed by this CA.
The command outputs a single file with a default name of
elastic-stack-ca.p12
. This file is a PKCS#12 keystore that contains the public certificate for your CA and the private key that is used to sign the certificates for each node.The
elasticsearch-certutil
command also prompts you for a password to protect the file and key. If you plan to add more nodes to your cluster in the future, retain a copy of the file and remember its password. -
Generate a certificate and private key for each node in your cluster.
Use the
elasticsearch-certutil cert
command:./bin/elasticsearch-certutil cert --ca elastic-stack-ca.p12
You are prompted for a password. You can enter a password for your certificate and key, or you can leave the password blank.
The output is a single PKCS#12 keystore that includes the node certificate, node key, and CA certificate.
Secure all output files, which contain the private keys for your instance.
The
elasticsearch-certutil
command generates certificates that have no hostname information in them such as SAN fields. This means that you can use the certificate for every node in your cluster, but you must turn off hostname verification.If you want to use hostname verification within your cluster, run the
elasticsearch-certutil cert
command once for each of your nodes and provide the--name
,--dns
and--ip
options.Alternatively, if you want to use a commercial or organization-specific CA, you can use the
elasticsearch-certutil csr
command to generate certificate signing requests (CSR) for the nodes in your cluster. -
Optional: Generate additional certificates specifically for encrypting HTTP client communications.
Use the
elasticsearch-certutil http
command:./bin/elasticsearch-certutil http
This command guides you through the process of generating the appropriate certificates for use in Elasticsearch and Kibana. If you created a CA for your cluster, you can re-use it by supplying its location when prompted.
The first question that the
elasticsearch-certutil
tool prompts you with is whether you want to generate a Certificate Signing Request (CSR). Answery
if a trusted authority, such as in internal security team or a commercial certificate authority, will sign your certificates. Answern
if you have access to a trusted CA and will sign your own certificates.The output is a
.zip
file containing one directory each for both Elasticsearch and Kibana:/elasticsearch |_ README.txt |_ http.p12 |_ sample-elasticsearch.yml /kibana |_ README.txt |_ elasticsearch-ca.pem |_ sample-kibana.yml
-
Copy the node certificate to the appropriate locations.
-
Create a folder in the configuration directory on each Elasticsearch node to contain your security certificates. For example, create a
certs
folder in the/home/es/config/certs
directory.The Elasticsearch configuration directory varies depending on your Elasticsearch installation.
-
On each node, copy the certificate that you created into the
/certs
directory. This certificate is typically a single.p12
(PKCS#12) file, such aselastic-certificates.p12
. -
If you generated HTTP certificates, copy the
http.p12
file into thecerts
directory as well. - For each additional Elastic product that you want to configure, copy the certificates to the relevant configuration directory.
-
Encrypting communications between nodes in a cluster
editThe transport networking layer is used for internal communication between nodes in a cluster. When security features are enabled, you must use TLS to ensure that communication between the nodes is encrypted.
- Generate node certificates.
-
Enable TLS and specify the information required to access the node’s certificate.
-
If the signed certificate is in PKCS#12 format, add the following information to the
elasticsearch.yml
file on each node:xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.keystore.path: elastic-certificates.p12 xpack.security.transport.ssl.truststore.path: elastic-certificates.p12
If you used the
--dns
or--ip
options with theelasticsearch-certutil cert
command and you want to enable strict hostname checking, set the verification mode tofull
. For a description of these values, see Transport TLS/SSL settings.If you created a separate certificate for each node, then you might need to customize this path on each node. If the filename matches the node name, you can use the
${node.name}.p12
format, for example.The
elasticsearch-certutil
outputs a PKCS#12 keystore which includes the CA certificate as a trusted certificate entry. This allows for the keystore to also be used as a truststore. In this case, the path value should match thekeystore.path
value. Note, however, that this is not the general rule. There are keystores that cannot be used as truststores, only specifically crafted ones can -
If the certificate is in PEM format, add the following information to the
elasticsearch.yml
file on each node:xpack.security.transport.ssl.enabled: true xpack.security.transport.ssl.verification_mode: certificate xpack.security.transport.ssl.key: /home/es/config/node01.key xpack.security.transport.ssl.certificate: /home/es/config/node01.crt xpack.security.transport.ssl.certificate_authorities: [ "/home/es/config/ca.crt" ]
If you used the
--dns
or--ip
options with theelasticsearch-certutil cert
command and you want to enable strict hostname checking, set the verification mode tofull
. For a description of these values, see Transport TLS/SSL settings.The full path to the node key file. This must be a location within the Elasticsearch configuration directory.
The full path to the node certificate. This must be a location within the Elasticsearch configuration directory.
An array of paths to the CA certificates that should be trusted. These paths must be a location within the Elasticsearch configuration directory.
-
-
If you secured the node’s certificate with a password, add the password to your Elasticsearch keystore:
-
If the signed certificate is in PKCS#12 format, use the following commands:
bin/elasticsearch-keystore add xpack.security.transport.ssl.keystore.secure_password bin/elasticsearch-keystore add xpack.security.transport.ssl.truststore.secure_password
-
If the certificate is in PEM format, use the following commands:
bin/elasticsearch-keystore add xpack.security.transport.ssl.secure_key_passphrase
-
-
Restart Elasticsearch.
You must perform a full cluster restart. Nodes which are configured to use TLS cannot communicate with nodes that are using unencrypted networking (and vice-versa). After enabling TLS you must restart all nodes in order to maintain communication across the cluster.
- All TLS-related node settings are considered to be highly sensitive and therefore are not exposed via the nodes info API For more information about any of these settings, see Security settings.
-
Elasticsearch monitors all files such as certificates, keys, keystores, or truststores
that are configured as values of TLS-related node settings. If you update any of
these files (for example, when your hostnames change or your certificates are
due to expire), Elasticsearch reloads them. The files are polled for changes at
a frequency determined by the global Elasticsearch
resource.reload.interval.high
setting, which defaults to 5 seconds.
Encrypting HTTP client communications
editWhen security features are enabled, you can optionally use TLS to ensure that communication between HTTP clients and the cluster is encrypted.
Enabling TLS on the HTTP layer is strongly recommended, but is not required.
In a production environment, some Elasticsearch features such as tokens and API keys will be disabled unless you enable TLS on the HTTP layer.
If you enable TLS on the HTTP layer in Elasticsearch, you might need to make configuration changes in other parts of the Elastic Stack and in any Elasticsearch clients that you use. For example, the following changes are typically necessary:
-
Update all clients, tools, and applications that connect to Elasticsearch to use the
https
protocol in their configuration URL. For example, Kibana, Beats, Logstash, language clients, and custom applications. - Set trusted certificate authorities. This change is typically required only if the Elasticsearch server certificate is issued by a certificate authority that is not automatically trusted by the clients, tools, and applications that connect to Elasticsearch.
If you enable advanced TLS features on Elasticsearch (such as
ssl.client_authentication
), additional
configuration could be necessary, but is not typically required in most Elasticsearch
deployments.
-
If you have not done so already, generate node certificates.
In particular, you need the files that are generated by the following command:
bin/elasticsearch-certutil http
This command generates a zip file that contains certificates and keys for use in Elasticsearch and Kibana. Each folder contains a readme that explains how to use the files.
-
Verify that you’ve copied the output files to the appropriate locations, as specified in the readme files.
For example, copy the
http.p12
file from theelasticsearch
folder into a directory within the Elasticsearch configuration directory on each node. If you chose to generate one certificate per node, copy the appropriatehttp.p12
file to each node. If you want to use Kibana to access this cluster, copy theelasticsearch-ca.pem
file from thekibana
folder into the Kibana configuration directory. -
Enable TLS and specify the information required to access the node’s certificate. For example:
-
Update the
elasticsearch.yml
file on each node with the location of the certificates.If the certificates are in PKCS#12 format:
xpack.security.http.ssl.enabled: true xpack.security.http.ssl.keystore.path: "http.p12"
If you have certificates in PEM format:
xpack.security.http.ssl.enabled: true xpack.security.http.ssl.key: /home/es/config/node1_http.key xpack.security.http.ssl.certificate: /home/es/config/node1_http.crt xpack.security.http.ssl.certificate_authorities: [ "/home/es/config/ca.crt" ]
The full path to the node key file. This must be a location within the Elasticsearch configuration directory.
The full path to the node certificate. This must be a location within the Elasticsearch configuration directory.
An array of paths to the CA certificates that should be trusted. These paths must be a location within the Elasticsearch configuration directory.
-
If you secured the keystore or the private key with a password, add that password to a secure setting in Elasticsearch.
If the certificates are in PKCS#12 format:
bin/elasticsearch-keystore add xpack.security.http.ssl.keystore.secure_password
If the certificates are in PEM format:
bin/elasticsearch-keystore add xpack.security.http.ssl.secure_key_passphrase
-
-
Optional: If you want to use Kibana, follow the instructions in the readme
provided by the
elasticsearch-certutil http
command or see Encrypting communications in Kibana. - Restart Elasticsearch.
- All TLS-related node settings are considered to be highly sensitive and therefore are not exposed via the nodes info API For more information about any of these settings, see Security settings.
-
Elasticsearch monitors all files such as certificates, keys, keystores, or truststores
that are configured as values of TLS-related node settings. If you update any of
these files (for example, when your hostnames change or your certificates are
due to expire), Elasticsearch reloads them. The files are polled for changes at
a frequency determined by the global Elasticsearch
resource.reload.interval.high
setting, which defaults to 5 seconds.
Encrypting communications between Elasticsearch and Active Directory
editTo protect the user credentials that are sent for authentication, it’s highly recommended to encrypt communications between Elasticsearch and your Active Directory server. Connecting via SSL/TLS ensures that the identity of the Active Directory server is authenticated before Elasticsearch transmits the user credentials and the usernames and passwords are encrypted in transit.
Clients and nodes that connect via SSL/TLS to the Active Directory server need to have the Active Directory server’s certificate or the server’s root CA certificate installed in their keystore or truststore.
-
Create the realm configuration for the
xpack.security.authc.realms
namespace in theelasticsearch.yml
file. See Configuring an Active Directory realm. -
Set the
url
attribute in the realm configuration to specify the LDAPS protocol and the secure port number. For example,url: ldaps://ad.example.com:636
. -
Configure each node to trust certificates signed by the certificate authority (CA) that signed your Active Directory server certificates.
The following example demonstrates how to trust a CA certificate (
cacert.pem
), which is located within the configuration directory:xpack: security: authc: realms: active_directory: ad_realm: order: 0 domain_name: ad.example.com url: ldaps://ad.example.com:636 ssl: certificate_authorities: [ "ES_PATH_CONF/cacert.pem" ]
The CA cert must be a PEM encoded certificate.
For more information about these settings, see Active Directory realm settings.
- Restart Elasticsearch.
By default, when you configure Elasticsearch to connect to Active Directory
using SSL/TLS, it attempts to verify the hostname or IP address
specified with the url
attribute in the realm configuration with the
values in the certificate. If the values in the certificate and realm
configuration do not match, Elasticsearch does not allow a connection to the
Active Directory server. This is done to protect against man-in-the-middle
attacks. If necessary, you can disable this behavior by setting the
ssl.verification_mode
property to certificate
.
Encrypting communications between Elasticsearch and LDAP
editTo protect the user credentials that are sent for authentication in an LDAP realm, it’s highly recommended to encrypt communications between Elasticsearch and your LDAP server. Connecting via SSL/TLS ensures that the identity of the LDAP server is authenticated before Elasticsearch transmits the user credentials and the contents of the connection are encrypted. Clients and nodes that connect via TLS to the LDAP server need to have the LDAP server’s certificate or the server’s root CA certificate installed in their keystore or truststore.
For more information, see LDAP user authentication.
-
Configure the realm’s TLS settings on each node to trust certificates signed by the CA that signed your LDAP server certificates. The following example demonstrates how to trust a CA certificate,
cacert.pem
, located within the Elasticsearch configuration directory (ES_PATH_CONF):xpack: security: authc: realms: ldap: ldap1: order: 0 url: "ldaps://ldap.example.com:636" ssl: certificate_authorities: [ "ES_PATH_CONF/cacert.pem" ]
The CA certificate must be a PEM encoded.
You can also specify the individual server certificates rather than the CA certificate, but this is only recommended if you have a single LDAP server or the certificates are self-signed.
-
Set the
url
attribute in the realm configuration to specify the LDAPS protocol and the secure port number. For example,url: ldaps://ldap.example.com:636
. - Restart Elasticsearch.
By default, when you configure Elasticsearch to connect to an LDAP server
using SSL/TLS, it attempts to verify the hostname or IP address
specified with the url
attribute in the realm configuration with the
values in the certificate. If the values in the certificate and realm
configuration do not match, Elasticsearch does not allow a connection to the
LDAP server. This is done to protect against man-in-the-middle attacks. If
necessary, you can disable this behavior by setting the
ssl.verification_mode
property to certificate
.