Manage HTTP certificates on ECK

ECK

ECK offers several options for securing the HTTP endpoints of Elasticsearch and Kibana. By default, the operator generates a dedicated CA per deployment, and issues individual certificates for each instance. Alternatively, you can supply your own certificates or integrate with external solutions like cert-manager.

By default, the operator manages a self-signed certificate with a custom CA for each resource. The CA, the certificate and the private key are each stored in a separate Secret.

> kubectl get secret | grep es-http
hulk-es-http-ca-internal         Opaque                                2      28m
hulk-es-http-certs-internal      Opaque                                2      28m
hulk-es-http-certs-public        Opaque                                1      28m

The public certificate is stored in a secret named <name>-[es|kb|apm|ent|agent]-http-certs-public.

> kubectl get secret hulk-es-http-certs-public -o go-template='{{index .data "tls.crt" | base64decode }}'
-----BEGIN CERTIFICATE-----
MIIDQDCCAiigAwIBAgIQHC4O/RWX15a3/P3upsm3djANBgkqhkiG9w0BAQsFADA6
...
QLYL4zLEby3vRxq65+xofVBJAaM=
-----END CERTIFICATE-----

You can provide your own CA and certificates instead of the self-signed certificate to connect to Elastic Stack applications through HTTPS using a Kubernetes secret.

Check Setup your own certificate to learn how to do that with elasticsearch-certutil tool.

This example illustrates how to create your own self-signed certificate for the quickstart Elasticsearch cluster using the OpenSSL command line utility. Note the subject alternative name (SAN) entry for quickstart-es-http.default.svc.

$ openssl req -x509 -sha256 -nodes -newkey rsa:4096 -days 365 -subj "/CN=quickstart-es-http" -addext "subjectAltName=DNS:quickstart-es-http.default.svc" -keyout tls.key -out tls.crt
$ kubectl create secret generic quickstart-es-cert --from-file=ca.crt=tls.crt --from-file=tls.crt=tls.crt --from-file=tls.key=tls.key

This example illustrates how to issue a self-signed certificate for the quickstart Elasticsearch cluster using a cert-manager self-signed issuer.

---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: selfsigned-issuer
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: quickstart-es-cert
spec:
  isCA: true
  dnsNames:
    - quickstart-es-http
    - quickstart-es-http.default.svc
    - quickstart-es-http.default.svc.cluster.local
  issuerRef:
    kind: Issuer
    name: selfsigned-issuer
  secretName: quickstart-es-cert
  subject:
    organizations:
      - quickstart

Here is how to issue multiple Elasticsearch certificates from a single self-signed CA. This is useful for example for Remote clusters which need to trust each other’s CA, in order to avoid mounting N CAs when a cluster is connected to N other clusters.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: selfsigned-issuer
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: selfsigned-ca
spec:
  isCA: true
  commonName: selfsigned-ca
  secretName: root-ca-secret
  privateKey:
    algorithm: ECDSA
    size: 256
  issuerRef:
    kind: ClusterIssuer
    name: selfsigned-issuer
---
apiVersion: cert-manager.io/v1
kind: Issuer
metadata:
  name: ca-issuer
spec:
  ca:
    secretName: root-ca-secret
---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: quickstart-es-cert
spec:
  isCA: false
  dnsNames:
    - quickstart-es-http
    - quickstart-es-http.default.svc
    - quickstart-es-http.default.svc.cluster.local
  subject:
    organizations:
      - quickstart
  privateKey:
    algorithm: RSA
    encoding: PKCS1
    size: 2048
  issuerRef:
    kind: Issuer
    name: ca-issuer
  secretName: quickstart-es-cert

To use a custom domain with a self-signed certificate:

spec:
  http:
    service:
      spec:
        type: LoadBalancer
    tls:
      selfSignedCertificate:
        subjectAltNames:
        - ip: 160.46.176.15
        - dns: hulk.example.com

You can bring your own certificate to configure TLS to ensure that communication between HTTP clients and the Elastic Stack application is encrypted.

Create a Kubernetes secret with:

• ca.crt: CA certificate (optional if tls.crt was issued by a well-known CA).
• tls.crt: The certificate.
• tls.key: The private key to the first certificate in the certificate chain.

kubectl create secret generic my-cert --from-file=ca.crt --from-file=tls.crt --from-file=tls.key

Alternatively you can also bring your own CA certificate including a private key and let ECK issue certificates with it. Any certificate SANs you have configured as described in Reserve static IP and custom domain will also be respected when issuing certificates with this CA certificate.

Create a Kubernetes secret with:

• ca.crt: CA certificate.
• ca.key: The private key to the CA certificate.

kubectl create secret generic my-cert --from-file=ca.crt --from-file=ca.key

In both cases, you have to reference the secret name in the http.tls.certificate section of the resource manifest.

spec:
  http:
    tls:
      certificate:
        secretName: my-cert

By default, ECK creates a ClusterIP Service and associates it with the Kibana deployment.

If you need to expose Kibana externally or customize the service settings, ECK provides flexible options, including support for load balancers, custom DNS/IP SANs, and user-provided certificates.

If you want to expose Kibana externally with a load balancer, it is recommended to include a custom DNS name or IP in the self-generated certificate.

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: kibana-sample
spec:
  version: 8.16.1
  count: 1
  elasticsearchRef:
    name: "elasticsearch-sample"
  http:
    service:
      spec:
        type: LoadBalancer
    tls:
      selfSignedCertificate:
        subjectAltNames:
        - ip: 1.2.3.4
        - dns: kibana.example.com

1. default is ClusterIP

If you want to use your own certificate, the required configuration is identical to Elasticsearch. Refer to setup your own Elasticsearch certificate for more information.

You can explicitly disable the generation of the self-signed certificate and hence disable TLS for Kibana, APM Server, and the HTTP layer of Elasticsearch.

To disable TLS, add the following setting to the appropriate resource:

spec:
  http:
    tls:
      selfSignedCertificate:
        disabled: true