Elastic Observability: Monitoring Kubernetes clusters

Get more familiar with Elastic Observability as well as an overview on how to ingest, view, and analyze customer logs from your applications using Elastic Cloud.

Get started with a 14-day trial. Once you go to cloud.elastic.co and create an account, follow this video to learn how to launch your first Elastic stack in any one of our 50+ supported regions globally.

Once your deployment is complete, under the Observability tab, select Monitor Kubernetes clusters and follow the guidance provided. Be sure to check out this blog for more details on managing your Kubernetes cluster.

If this is your first time using Elastic you will need to add an Elastic agent to the Kubernetes cluster you want to monitor and configure the Kubernetes integration.

After you select Monitor Kubernetes clusters, click Start and you'll be prompted to add the Kubernetes integration.

When you select the Add Kubernetes, you'll be prompted with 3 simple steps for ingesting your Kubernetes data.

First, you'll add an Elastic agent with the Kubernetes integration. While the integration is preset to collect all Kubernetes data, you can select only what you want to collect (i.e., pod metrics, container logs, etc.).

For a high level overview of what the Kubernetes integration using the Elastic Agentcan do for you, check out this blog.

In addition to the standard Kubernetes components, the Elastic Agent integration for Kubernetes also supports collection from kube-state-metrics (ksm), a widely popular addition for monitoring Kubernetes clusters; ksm does not come as default with most Kubernetes distributions and you will need to install it separately.

For more information on how to run a Elastic agent on Kubernetes, follow along in the documentation.

Kubernetes logs can be searched and analyzed in Elastic’s Discover capability. To leverage Discover, navigate to the left menu and select Discover under Analytics.

Here we see how Kubernetes logs can be searched, reviews, and analyzed:

Additionally we can also look at metrics in discover:

Additionally, there are a variety of Kubernetes out-of-the-box dashboards that allow you to analyze Kubernetes metrics within Kibana. Simply navigate to Dashboards located under Analytics on the left and search Kubernetes.

If we look at the Kubernetes Overview, we can see metrics for the entire Kubernetes Cluster. All the nodes, pods, and CPU and memory usage.

Kibana allows you to analyze logs with interactive dashboards to derive insights, automate workflows, find anomalies and trends, and more. Be sure to explore the tutorial library below and get more familiar with Kibana so that you can customize or create your own dashboards.

When you select Dashboards located under Analytics you can also Create dashboard and customize your new out-of-the-box dashboard to your needs.

If you get stuck, check out this guided tour for getting your data in.

To learn more about Kibana and creating dashboards and using Discover.

Set up alerts

On the left navigation under Observability, next you'll want to select Alerts and Create rules. This will allow you to get notifications when various events happen (i.e., latency is anomalous, metric aggregation exceeds threshold, etc.) via email, Jira, Slack, and more.

Leverage machine learning to uncover insights

Prior to beginning this section make sure you deploy your node towards machine learning.

Lastly, navigate to Machine Learning located under Analytics in the menu. On the left menu on the Machine Learning Overview page, under Anomaly Detection select Jobs to create a machine learning job. By setting up machine learning jobs, for example, rather than having an alert when a specific percentage (i.e., 50%, 80% etc.) of memory usage has occurred, you will know when the usage is just not normal.

• Explore observability demo gallery
• Get started with collecting and analyzing your logs
• Get started with monitoring your application performance (APM/tracking)
• Get started with monitoring your hosts
• Get started with optimizing workloads with universal profiling
• Get started with creating a synthetic monitor
• Managing your Kubernetes cluster with Elastic Observability