Encrypt traffic in clusters with a self-managed Fleet Server

edit

Encrypt traffic in clusters with a self-managed Fleet Server

edit

If you’re running a self-managed cluster, configure Transport Layer Security (TLS) to encrypt traffic between Elastic Agents, Fleet Server, and other components in the Elastic Stack.

Our hosted Elasticsearch Service on Elastic Cloud provides secure, encrypted connections out of the box!

Prerequisites
edit

Configure security and generate certificates for the Elastic Stack. For more information about securing the Elastic Stack, refer to Configure security for the Elastic Stack.

Elastic Agents require a PEM-formatted CA certificate to send encrypted data to Elasticsearch. If you followed the steps in Configure security for the Elastic Stack, your certificate will be in a p12 file. To convert it, use openssl:

openssl pkcs12 -in path.p12 -out cert.crt -clcerts -nokeys
openssl pkcs12 -in path.p12 -out private.key -nocerts -nodes

Key passwords are not currently supported.

Generate a custom certificate and private key for Fleet Server
edit

This section describes how to use the certutil tool provided by Elasticsearch, but you can use whatever process you typically use to generate PEM-formatted certificates.

  1. Generate a certificate authority (CA). Skip this step if you want to use an existing CA.

    ./bin/elasticsearch-certutil ca --pem

    This command creates a zip file that contains the CA certificate and key you’ll use to sign the Fleet Server certificate. Extract the zip file:

    Screen capture of a folder called ca that contains two files: ca.crt and ca.key

    Store the files in a secure location.

  2. Use the certificate authority to generate certificates for Fleet Server. For example:

    ./bin/elasticsearch-certutil cert \
      --name fleet-server \
      --ca-cert /path/to/ca/ca.crt \
      --ca-key /path/to/ca/ca.key \
      --dns your.host.name.here \
      --ip 192.0.2.1 \
      --pem

    Where dns and ip specify the name and IP address of the Fleet Server. Run this command for each Fleet Server you plan to deploy.

    This command creates a zip file that includes a .crt and .key file. Extract the zip file:

    Screen capture of a folder called fleet-server that contains two files: fleet-server.crt and fleet-server.key

    Store the files in a secure location. You’ll need these files later to encrypt traffic between Elastic Agents and Fleet Server.

Encrypt traffic between Elastic Agents, Fleet Server, and Elasticsearch
edit

Fleet Server needs a CA certificate to connect securely to Elasticsearch. It also needs to expose a Fleet Server certificate so other Elastic Agents can connect to it securely.

For the steps in this section, imagine you have the following files:

ca.crt

The CA certificate to use to connect to Fleet Server. This is the CA used to generate a certificate and key for Fleet Server.

fleet-server.crt

The certificate you generated for Fleet Server.

fleet-server.key

The private key you generated for Fleet Server.

elasticsearch-ca.crt

The CA certificate to use to connect to Elasticsearch. This is the CA used to generate certs for Elasticsearch (see Prerequisites).

To encrypt traffic between Elastic Agents, Fleet Server, and Elasticsearch:

  1. Configure Fleet settings:

    1. In Kibana go to Management > Fleet.
    2. In the upper-right corner, click Fleet settings and set connection details that apply to all Fleet-managed Elastic Agents:

      Screen capture that shows Fleet Server hosts
    3. In the Fleet Server hosts field, specify the URLs Elastic Agents will use to connect to Fleet Server. For example, https://192.0.2.1:8220, where 192.0.2.1 is the host IP where you will install Fleet Server.
    4. In the Elasticsearch host field, specify the Elasticsearch URLs where Elastic Agents will send data. For example, https://192.0.2.0:9200.

      Use the https protocol for both host settings. DNS-based names are also allowed.

    5. Under Elasticsearch output configuration (YAML), set ssl.certificate_authorities and specify the CA certificate to use to connect to Elasticsearch. You can specify a list of file paths (if the files are available), or embed a certificate directly in the YAML configuration. If you specify file paths, the certificates must be available on the hosts running the Elastic Agents.

      Path example:

      ssl.certificate_authorities: ["/path/to/your/elasticsearch-ca.crt"] 

      The path to the CA certificate on the Elastic Agent host.

      Pasted certificate example:

      ssl:
        certificate_authorities:
        - |
          -----BEGIN CERTIFICATE-----
          MIIDSjCCAjKgAwIBAgIVAKlphSqJclcni3P83gVsirxzuDuwMA0GCSqGSIb3DQEB
          CwUAMDQxMjAwBgNVBAMTKUVsYXN0aWMgQ2VydGlmaWNhdGUgVG9vbCBBdXRvZ2Vu
          ZXJhdGVkIENBMB4XDTIxMDYxNzAxMzIyOVoXDTI0MDYxNjAxMzIyOVowNDEyMDAG
          A1UEAxMpRWxhc3RpYyBDZXJ0aWZpY2F0ZSBUb29sIEF1dG9nZW5lcmF0ZWQgQ0Ew
          ggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDOFgtVri7Msy2iR33nLrVO
          /M/6IyF72kFXup1E67TzetI22avOxNlq+HZTpZoWGV1I4RgxiQeN12FLuxxhd9nm
          rxfZEqpuIjvo6fvU9ifC03WjXg1opgdEb6JqH93RHKw0PYimxhQfFcwrKxFseHUx
          DeUNQgHkMQhDZgIfNgr9H/1X6qSU4h4LemyobKY3HDKY6pGsuBzsF4iOCtIitE9p
          sagiWR21l1gW/lNaEW2ICKhJXbaqbE/pis45/yyPI4Q1Jd1VqZv744ejnZJnpAx9
          mYSE5RqssMeV6Wlmu1xWljOPeerOVIKUfHY38y8GZwk7TNYAMajratG2dj+v9eAV
          AgMBAAGjUzBRMB0GA1UdDgQWBBSCNCjkb66eVsIaa+AouwUsxU4b6zAfBgNVHSME
          GDAWgBSCNCjkb66eVsIaa+AouwUsxU4b6zAPBgNVHRMBAf8EBTADAQH/MA0GCSqG
          SIb3DQEBCwUAA4IBAQBVSbRObxPwYFk0nqF+THQDG/JfpAP/R6g+tagFIBkATLTu
          zeZ6oJggWNSfgcBviTpXc6i1AT3V3iqzq9KZ5rfm9ckeJmjBd9gAcyqaeF/YpWEb
          ZAtbxfgPLI3jK+Sn8S9fI/4djEUl6F/kARpq5ljYHt9BKlBDyL2sHymQcrDC3pTZ
          hEOM4cDbyKHgt/rjcNhPRn/q8g3dDhBdzjlNzaCNH/kmqWpot9AwmhhfPTcf1VRc
          gxdg0CTQvQvuceEvIYYYVGh/cIsIhV2AyiNBzV5jJw5ztQoVyWvdqn3B1YpMP8oK
          +nadUcactH4gbsX+oXRULNC7Cdd9bp2G7sQc+aZm
          -----END CERTIFICATE-----
  2. Install an Elastic Agent as a Fleet Server on the host and configure it to use TLS:

    1. If you don’t already have a Fleet Server service token, click the Agents tab in Fleet and follow the instructions to generate the service token now.

      The in-product installation steps are incomplete. Before running the install command, add the settings shown in the next step.

    2. From the directory where you extracted Fleet Server, run the install command and specify the certificates to use.

      The following command installs Elastic Agent as a service, enrolls it in the Fleet Server policy, and starts the service.

      If you’re using DEB or RPM, or already have the Elastic Agent installed, use the enroll command along with the following options, then start the service as described in Start Elastic Agent.

      sudo ./elastic-agent install -f \
         --url=https://192.0.2.1:8220 \
         --fleet-server-es=https://192.0.2.0:9200 \
         --fleet-server-service-token=AAEBAWVsYXm0aWMvZmxlZXQtc2XydmVyL3Rva2VuLTE2MjM4OTAztDU1OTQ6dllfVW1mYnFTVjJwTC2ZQ0EtVnVZQQ \
         --fleet-server-es-ca=/path/to/elasticsearch-ca.crt \
         --certificate-authorities=/path/to/ca.crt \
         --fleet-server-cert=/path/to/fleet-server.crt \
         --fleet-server-cert-key=/path/to/fleet-server.key

      Where:

      url
      Fleet Server URL.
      fleet-server-es
      Elasticsearch URL
      fleet-server-service-token
      Service token to use to communicate with Elasticsearch.
      fleet-server-es-ca
      CA certificate to use to connect to Elasticsearch.
      certificate-authorities
      CA certificate to use to connect to Fleet Server.
      fleet-server-cert
      Certificate to use for the exposed Fleet Server HTTPS endpoint.
      fleet-server-cert-key
      Private key to use for the exposed Fleet Server HTTPS endpoint.
  3. Install your Elastic Agents and enroll them in Fleet.

    Elastic Agents connecting to a secured Fleet Server need to pass in the CA certificate used by the Fleet Server. The CA certificate used by Elasticsearch is already specified in the agent policy because it’s set under Fleet settings in Kibana. You do not need to pass it on the command line.

    The following command installs Elastic Agent as a service, enrolls it in the agent policy associated with the specified token, and starts the service.

    sudo elastic-agent install -f --url=https://192.0.2.1:8220 \
      --enrollment-token=<string> \
      --certificate-authorities=/path/to/ca.crt

    Where:

    url
    Fleet Server URL to use to enroll the Elastic Agent into Fleet.
    enrollment-token
    The enrollment token for the policy that will be applied to the Elastic Agent.
    certificate-authorities
    CA certificate to use to connect to Fleet Server. This is the CA used to generate a certificate and key for Fleet Server.

    Don’t have an enrollment token? On the Agents tab in Fleet, click Add agent. Under Enroll and start the Elastic Agent, follow the in-product installation steps, making sure that you add the --certificate-authorities option before you run the command.