Kubernetes
Collect logs and metrics from Kubernetes clusters with Elastic Agent.
Version | 1.66.4 (View all) |
Compatible Kibana version(s) | 8.15.0 or higher |
Supported Serverless project types | Security Observability |
Subscription level | Basic |
Level of support | Elastic |
This integration is used to collect logs and metrics from Kubernetes clusters. Time series index mode enabled for metrics data streams.
This integration requires kube-state-metrics, which is not included with Kubernetes by default. For dashboards to properly populate, the kube-state-metrics service must be deployed to your Kubernetes cluster |
As one of the main pieces provided for Kubernetes monitoring, this integration is capable of fetching metrics from several components:
Some of the previous components are running on each of the Kubernetes nodes (like kubelet
or proxy
) while others provide
a single cluster-wide endpoint. This is important to determine the optimal configuration and running strategy
for the different datasets included in the integration.
Kubernetes endpoints and metricsets
Kubernetes module is a bit complex as its internal datasets require access to a wide variety of endpoints.
This section highlights and introduces some groups of datasets with similar endpoint access needs.
For more details on the datasets see configuration example
and the datasets
sections below.
node / system / pod / container / module / volume
The datasets container
, node
, pod
, system
and volume
require access to the kubelet endpoint
in each of
the Kubernetes nodes, hence it's recommended to include them as part
of an Agent DaemonSet
or standalone Agents running on the hosts.
Depending on the version and configuration of Kubernetes nodes, kubelet
might provide a read only http port (typically 10255),
which is used in some configuration examples. But in general, and lately, this endpoint requires SSL (https
) access
(to port 10250 by default) and token based authentication.
state_* and event
State_* datasets are enabled by default.
All datasets with the state_
prefix require hosts
field pointing to kube-state-metrics
service within the cluster. As the service provides cluster-wide metrics, there's no need to fetch them per node,
hence the recommendation is to run these datasets as part of an Agent Deployment
with one only replica.
Generally kube-state-metrics
runs a Deployment
and is accessible via a service called kube-state-metrics
on
kube-system
namespace, which will be the service to use in our configuration.
apiserver
The apiserver dataset requires access to the Kubernetes API, which should be easily available in all Kubernetes
environments. Depending on the Kubernetes configuration, the API access might require SSL (https
) and token
based authentication.
proxy
The proxy dataset requires access to the proxy endpoint in each of Kubernetes nodes, hence it's recommended
to configure it as a part of an Agent DaemonSet
.
scheduler and controllermanager
These datasets require access to the Kubernetes controller-manager
and scheduler
endpoints. By default, these pods
run only on master nodes, and they are not exposed via a Service, but there are different strategies
available for its configuration:
- Create
Kubernetes Services
to makekube-controller-manager
andkube-scheduler
available and configure the datasets to point to these services as part of anAgent Deployment
. - Run these datasets as part an
Agent Daemonset
(with HostNetwork setting) with anodeSelector
to only run on Master nodes.
These datasets are not enabled by default.
Note: In some "As a Service" Kubernetes implementations, like GKE
, the master nodes or even the pods running on
the masters won't be visible. In these cases it won't be possible to use scheduler
and controllermanager
metricsets.
container-logs
The container-logs dataset requires access to the log files in each Kubernetes node where the container logs are stored.
This defaults to /var/log/containers/*${kubernetes.container.id}.log
.
Routing
The container-logs data stream allows routing logs to a different dataset or namespace using pod annotations.
For example, suppose you are running Nginx on your Kubernetes cluster, and you want to drive the Nginx container logs into a dedicated dataset or namespace. By annotating the pod with elastic.co/namespace: nginx
, the integration will send all the container logs to the nginx
namespace.
To learn more about routing container-logs, see https://docs.elastic.co/integrations/kubernetes/container-logs.
Preserve original event
The agent can be configured to set the tag preserve_original_event
on container-logs using pod annotation.
For example, suppose you are routing your Nginx container logs into a dedicated dataset or namespace as described above to make use of the Nginx fleet integration. Enabling preserve_original_event on the Nginx integration will have no effect since the logs were shipped via kubernetes integration and not Nginx. As well, you may not want to have all original events from all Nginx pods preserved as well.
By annotating the pod with elastic.co/preserve_original_event: 'true'
, the integration will add the tag preserve_original_event
as it would be done by the nginx
integration otherwise.
audit-logs
The audit-logs dataset requires access to the log files on each Kubernetes node where the audit logs are stored.
This defaults to /var/log/kubernetes/kube-apiserver-audit.log
.
Compatibility
The Kubernetes package is tested with Kubernetes [1.28.x - 1.31.x] versions
Dashboard
Kubernetes integration is shipped including default dashboards for apiserver
, controllermanager
, overview
, proxy
and scheduler
.
If you are using HA for those components, be aware that when gathering data from all instances the dashboard will usually show the average of the metrics. For those scenarios filtering by hosts or service address is possible.
Cluster selector in overview
dashboard helps in distinguishing and filtering metrics collected from multiple clusters. If you want to focus on a subset of the Kubernetes clusters for monitoring a specific scenario, this cluster selector could be a handy tool. Note that this selector gets populated from the orchestrator.cluster.name
field that may not always be available. This field gets its value from sources like kube_config
, kubeadm-config
configMap, and Google Cloud's meta API for GKE. If the sources mentioned above don't provide this value, metricbeat will not report it. However, you can always use processors to set this field and utilize it in the cluster overview
dashboard.
Changelog
Version | Details | Kibana version(s) |
---|---|---|
1.66.4 | Bug fix View pull request | 8.15.0 or higher |
1.66.3 | Enhancement View pull request | 8.15.0 or higher |
1.66.2 | Bug fix View pull request | 8.15.0 or higher |
1.66.1 | Bug fix View pull request | 8.15.0 or higher |
1.66.0 | Enhancement View pull request | 8.15.0 or higher |
1.65.0 | Enhancement View pull request | 8.15.0 or higher |
1.64.0 | Enhancement View pull request | 8.15.0 or higher |
1.63.1 | Bug fix View pull request | 8.15.0 or higher |
1.63.0 | Enhancement View pull request | 8.15.0 or higher |
1.62.1 | Bug fix View pull request | 8.14.0 or higher |
1.62.0 | Enhancement View pull request | 8.14.0 or higher |
1.61.1 | Bug fix View pull request | 8.14.0 or higher |
1.61.0 | Enhancement View pull request | 8.14.0 or higher |
1.60.0 | Bug fix View pull request | 8.14.0 or higher |
1.59.0 | Enhancement View pull request | 8.14.0 or higher |
1.58.0 | Enhancement View pull request | 8.12.0 or higher |
1.57.0 | Enhancement View pull request | 8.12.0 or higher |
1.56.0 | Enhancement View pull request | 8.12.0 or higher |
1.55.1 | Enhancement View pull request | 8.11.0 or higher |
1.55.0 | Enhancement View pull request | 8.11.0 or higher |
1.54.0 | Enhancement View pull request | 8.11.0 or higher |
1.53.0 | Enhancement View pull request | 8.11.0 or higher |
1.52.0 | Enhancement View pull request | 8.11.0 or higher |
1.51.0 | Enhancement View pull request | 8.10.2 or higher |
1.50.0 | Enhancement View pull request | 8.10.2 or higher |
1.49.0 | Enhancement View pull request | 8.10.2 or higher |
1.48.0 | Enhancement View pull request | 8.10.2 or higher |
1.47.0 | Enhancement View pull request | 8.10.2 or higher |
1.46.0 | Enhancement View pull request | 8.10.1 or higher |
1.45.0 | Enhancement View pull request | 8.10.0 or higher |
1.44.0 | Enhancement View pull request | 8.10.0 or higher |
1.43.1 | Enhancement View pull request | 8.8.0 or higher |
1.43.0 | Enhancement View pull request | 8.8.0 or higher |
1.42.0 | Enhancement View pull request | 8.8.0 or higher |
1.41.0 | Enhancement View pull request | 8.8.0 or higher |
1.40.0 | Bug fix View pull request | 8.8.0 or higher |
1.40.0-beta.2 | Bug fix View pull request | — |
1.40.0-beta.1 | Bug fix View pull request | — |
1.40.0-beta | Enhancement View pull request | — |
1.39.0 | Enhancement View pull request | 8.6.1 or higher |
1.38.1 | Enhancement View pull request | 8.6.1 or higher |
1.38.0 | Enhancement View pull request | 8.6.1 or higher |
1.37.0 | Enhancement View pull request | 8.6.1 or higher |
1.36.0 | Enhancement View pull request | 8.6.1 or higher |
1.35.0 | Enhancement View pull request | 8.6.1 or higher |
1.34.1 | Enhancement View pull request | 8.6.1 or higher |
1.34.0 | Enhancement View pull request | 8.6.1 or higher |
1.33.0 | Enhancement View pull request | 8.6.1 or higher |
1.32.2 | Enhancement View pull request | 8.6.1 or higher |
1.32.1 | Enhancement View pull request | 8.6.1 or higher |
1.32.0 | Enhancement View pull request | 8.6.1 or higher |
1.31.2 | Enhancement View pull request | 8.6.1 or higher |
1.31.1 | Enhancement View pull request | 8.6.1 or higher |
1.31.0 | Enhancement View pull request | 8.6.0 or higher |
1.30.0 | Enhancement View pull request | 8.6.0 or higher |
1.29.2 | Bug fix View pull request | 8.5.0 or higher |
1.29.1 | Bug fix View pull request | 8.5.0 or higher |
1.29.0 | Bug fix View pull request | 8.5.0 or higher |
1.28.2 | Bug fix View pull request | 8.5.0 or higher |
1.28.1 | Enhancement View pull request | 8.5.0 or higher |
1.28.0 | Enhancement View pull request | 8.5.0 or higher |
1.27.1 | Enhancement View pull request | 8.5.0 or higher |
1.27.0 | Enhancement View pull request | 8.5.0 or higher |
1.26.0 | Enhancement View pull request | 8.4.0 or higher |
1.25.0 | Enhancement View pull request | 8.4.0 or higher |
1.24.0 | Enhancement View pull request | 8.4.0 or higher |
1.23.1 | Enhancement View pull request | 8.4.0 or higher |
1.23.0 | Enhancement View pull request | 8.4.0 or higher |
1.22.1 | Enhancement View pull request | 8.4.0 or higher |
1.22.0 | Enhancement View pull request | 8.4.0 or higher |
1.21.2 | Bug fix View pull request | 8.3.0 or higher |
1.21.1 | Enhancement View pull request | 8.3.0 or higher |
1.21.0 | Enhancement View pull request | 8.3.0 or higher |
1.20.0 | Enhancement View pull request | 8.2.0 or higher |
1.19.1 | Enhancement View pull request | 8.2.0 or higher |
1.19.0 | Enhancement View pull request | 8.2.0 or higher |
1.18.1 | Enhancement View pull request | 8.2.0 or higher |
1.18.0 | Enhancement View pull request | 8.2.0 or higher |
1.17.3 | Bug fix View pull request | 7.16.0 or higher |
1.17.2 | Bug fix View pull request | 7.16.0 or higher |
1.17.1 | Enhancement View pull request | — |
1.17.0 | Enhancement View pull request | — |
1.16.0 | Enhancement View pull request | — |
1.15.0 | Enhancement View pull request | — |
1.14.3 | Bug fix View pull request | — |
1.14.2 | Bug fix View pull request | — |
1.14.1 | Bug fix View pull request | — |
1.14.0 | Enhancement View pull request | — |
1.13.0 | Enhancement View pull request | — |
1.12.0 | Enhancement View pull request | — |
1.11.0 | Enhancement View pull request | — |
1.10.0 | Enhancement View pull request | — |
1.9.0 | Enhancement View pull request | 7.16.0 or higher |
1.8.1 | Bug fix View pull request | 7.16.0 or higher |
1.8.0 | Enhancement View pull request | 7.16.0 or higher |
1.7.0 | Enhancement View pull request | 7.16.0 or higher |
1.6.0 | Enhancement View pull request | 7.16.0 or higher |
1.5.0 | Enhancement View pull request | 7.16.0 or higher |
1.4.2 | Enhancement View pull request | — |
1.4.1 | Enhancement View pull request | 8.0.0 or higher |
1.4.0 | Enhancement View pull request | — |
1.3.3 | Bug fix View pull request | — |
1.3.2 | Enhancement View pull request | — |
1.3.1 | Enhancement View pull request | — |
1.3.0 | Enhancement View pull request | — |
1.2.1 | Bug fix View pull request | — |
1.2.0 | Enhancement View pull request | — |
1.1.1 | Bug fix View pull request | — |
1.1.0 | Enhancement View pull request | 7.15.0 or higher |
1.0.0 | Enhancement View pull request | — |
0.14.1 | Enhancement View pull request | — |
0.14.0 | Enhancement View pull request | — |
0.13.0 | Enhancement View pull request | — |
0.12.2 | Bug fix View pull request | — |
0.12.1 | Bug fix View pull request | — |
0.12.0 | Enhancement View pull request | — |
0.11.1 | Enhancement View pull request | — |
0.11.0 | Enhancement View pull request | — |
0.10.0 | Enhancement View pull request | — |
0.9.1 | Bug fix View pull request | — |
0.9.0 | Enhancement View pull request | — |
0.8.0 | Enhancement View pull request | — |
0.7.0 | Enhancement View pull request | — |
0.6.0 | Enhancement View pull request | — |
0.5.3 | Enhancement View pull request | — |
0.5.2 | Bug fix View pull request | — |
0.5.1 | Bug fix View pull request | — |
0.5.0 | Enhancement View pull request | — |
0.4.5 | Enhancement View pull request | — |
0.4.4 | Enhancement View pull request | — |
0.4.3 | Bug fix View pull request | — |
0.4.2 | Bug fix View pull request | — |
0.4.1 | Enhancement View pull request | — |
0.1.0 | Enhancement View pull request | — |