Skip to main content
Login
Start free trialContact Sales

ECK in production environment

By

Fram Souza

eck-blog-thumb.png

This article will guide you on how to spin up an ECK environment ready for production which contains,

  • dedicated stack monitoring,
  • fleet-server & elastic-agent (with kubernetes integration),
  • elasticsearch autoscaling,
  • nodeAffinity & nodeSelector,
  • SAML with auth0,
  • hot, warm, cold, frozen architecture,
  • heartbeat monitor SSL certificate expiration,
  • external-dns (with CloudFlare integration),
  • cert-manager (with let's encrypt integration),
  • ingress controller (using Cloudflare to register the endpoints),
  • Optional: esrally to validate autoscaling and run benchmark against es cluster

Remember you must have basic or enterprise license to run ECK.

Demo

To see the demo and files, please access https://github.com/framsouza/eck-ready-for-production

cert-manager

cert-manager adds certificates and certificate issuers as resource types in Kubernetes, and simplifies the process of obtaining, renewing and using those certificates. It's very command Kubernetes administrator use cert-manager to handle certificate, and on this example we are going to use cert-manager with let's encrypt to access Kibana. Remember that, TLS certificates for the Elasticsearch transport layer that are used for internal communications between Elasticsearch nodes are managed by ECK and cannot be changed.

ingress

Ingress controller is specialized load balancer for Kubernetes, which accepts traffic from outside the Kubernetes cluster and balances it to pods.

external-dns

It's a addon that configures public DNS servers about exposed Kubernetes services, on this examples we are integrating external-dns with Cloudflare. For each Ingress/Service resource you us, a DNS entry will created on Cloudflare with the respective IP address, on external-dns logs you should be able to see the following:

level=info msg="Using inCluster-config based on serviceaccount-token"
level=info msg="Created Kubernetes client https://10.76.0.1:443"
level=info msg="Changing record." action=CREATE record=kibana.framsouza.co ttl=1 type=A zone=4cd4c7c1cb8f7bf3a7482749654ae6fb
level=info msg="Changing record." action=CREATE record=monitoring.framsouza.co ttl=1 type=TXT zone=4cd4c7c1cb8f7bf3a7482749654ae6fb

How-to setup

Make sure to respect the commands execution order:

  1. Create GKE cluster with Kubernetes type hot, warm, cold, frozen for each dedicated node pool, make sure you will have enough resource to run the pods in the nodes.
  2. Create a cluster role mapping that gives you permission to install ECK operator
    • kubectl create clusterrolebinding cluster-admin-binding --cluster-role=cluster-admin --user=<USERNAME>
  3. Install ECK operator
    • helm repo add elastic https://helm.elastic.co && helm repo update && helm install elastic-operator elastic/eck-operator -n elastic-system --create-namespace
  4. Create dedicated storage class by applying files storageclass-hot.yaml and storageclass-warm.yaml
  5. Download your license and apply it via secret (or apply the license.yaml)
    • kubectl create secret generic eck-license --from-file <LICENSE-PATH> -n elastic-system && kubectl label secret eck-license "license.k8s.elastic.co/scope"=operator -n elastic-system
  6. Create the monitoring cluster (it will create a ns call monitoring) by applying monitoring-es.yaml
  7. Create elasticsearch resource, elasticsearch.yaml
  8. Create kibana resource, kibana.yaml
  9. Create fleet resource, fleet.yaml
  10. Create heartbeat, heartbeat.yaml
  11. Install external-dns
    • kubectl apply -f external-dns.yml
  12. Install cert-manager
    • kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.7.1/cert-manager.yaml
  13. Install ingress-nginx ,
    • helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace
  14. Create ingress resource, ingress.yaml
  15. Create cluster issuer apply, clusterissuer.yaml
  16. Create let's encrypt certificate, certificate.yaml

Accessing

For this example, I am using a domain call framsouza.co and as I am using external-dns, the DNS entry will be automatically added to Cloudflare, https://kibana.framsouza.co and https://monitoring.framsouza.co. You can check the connection is safe and we are using a valid certificate by let's encrypt.

If you want to login using SAML, make sure to adjust the saml session on elasticsearch.yml according to your environment.

Autoscaling validation

(Optional)

To confirm if autoscaling is working as expected, you can use esrally to test that and run benchmark against your cluster. Make sure to have params-file.json with the following content:

{

    "number_of_replicas": 1,

    "number_of_shards": 2

}

then you just need to run and wait a couple of hours until the test is finished.

docker run -v /tmp/params-file.json:/tmp/params-file.json elastic/rally race --track=http_logs --target-hosts=${IP}:9200 --pipeline=benchmark-only --client-options="timeout:60,use_ssl:true,verify_certs:false,basic_auth_user:'elastic',basic_auth_password:'${PASSWORD}'"  --track-params=/tmp/params-file.json

Have a look at esrally-result.txt.

Keep learning about ECK:

Sign up for Elastic Cloud free trial

Spin up a fully loaded deployment on the cloud provider you choose. As the company behind Elasticsearch, we bring our features and support to your Elastic clusters in the cloud.

Start free trial