Kubernetes Container Logs
Collect container related logs from Kubernetes clusters with Elastic Agent.
Version | 1.66.4 (View all) |
Compatible Kibana version(s) | 8.15.0 or higher |
Supported Serverless project types | Security Observability |
Subscription level | Basic |
container-logs integration collects and parses logs of Kubernetes containers.
It requires access to the log files in each Kubernetes node where the container logs are stored.
This defaults to /var/log/containers/*${kubernetes.container.id}.log
.
By default only container parser is enabled. Additional log parsers can be added as an advanced options configuration.
Rerouting and preserve original event based on pod annotations
You can customize the routing of container logs events and sending them to different datasets and namespaces,
as well as enable preserve_original_event
based on using pods' annotations.
Customization can happen at:
- pod definition time, e.g., using a deployment.
- pod runtime, annotating pods using
kubectl
.
Set at pod definition time
Here is an example of an Nginx deployment where we set both elastic.co/dataset
and elastic.co/namespace
annotations to route the container logs to specific datasets and namespace, respectively.
# nginx-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deployment
spec:
replicas: 1
selector:
matchLabels:
app: nginx
template:
metadata:
annotations:
elastic.co/dataset: kubernetes.container_logs.nginx
elastic.co/namespace: nginx
elastic.co/preserve_original_event: "true"
labels:
app: nginx
app.kubernetes.io/name: myservice
app.kubernetes.io/version: v0.1.2
app.kubernetes.io/instance: myservice-abcxzy
spec:
containers:
- name: nginx-container
image: nginx:latest
ports:
- containerPort: 80
Set at runtime
Suppose you want to change the container logs routing and enable preserve_original_event
on a running container.
In that case, you can annotate the pod using kubectl
, and the integration will apply it immediately sending all the following documents to the new destination:
Here is an example where we route the container logs for a pod running the Elastic Agent to the kubernetes.container_logs.agents
dataset:
kubectl annotate pods elastic-agent-managed-daemonset-6p22g elastic.co/dataset=kubernetes.container_logs.agents
Here's a similar example to change the namespace on a pod running Nginx:
kubectl annotate pods elastic-agent-managed-daemonset-6p22g elastic.co/namespace=nginx
Here is an example to enable preserve_original_event
on a pod running Nginx:
kubectl annotate pods elastic-agent-managed-daemonset-6p22g elastic.co/preserve_original_event=true
You can restore the standard settings by removing the annotations:
kubectl annotate pods elastic-agent-managed-daemonset-6p22g elastic.co/dataset-
kubectl annotate pods elastic-agent-managed-daemonset-6p22g elastic.co/namespace-
kubectl annotate pods elastic-agent-managed-daemonset-6p22g elastic.co/preserve_original_event-
Annotations Reference
Here are the annotations available to customize routing:
Label | Description |
---|---|
elastic.co/dataset | Defines the target data stream's dataset for this pod. |
elastic.co/namespace | Defines the target data stream's namespace for this pod. |
elastic.co/preserve_original_event | Enables 'preserve_original_event' for this pod. Use string 'true' (case-insensitive) to enable |
Changelog
Version | Details | Kibana version(s) |
---|---|---|
1.66.4 | Bug fix View pull request | 8.15.0 or higher |
1.66.3 | Enhancement View pull request | 8.15.0 or higher |
1.66.2 | Bug fix View pull request | 8.15.0 or higher |
1.66.1 | Bug fix View pull request | 8.15.0 or higher |
1.66.0 | Enhancement View pull request | 8.15.0 or higher |
1.65.0 | Enhancement View pull request | 8.15.0 or higher |
1.64.0 | Enhancement View pull request | 8.15.0 or higher |
1.63.1 | Bug fix View pull request | 8.15.0 or higher |
1.63.0 | Enhancement View pull request | 8.15.0 or higher |
1.62.1 | Bug fix View pull request | 8.14.0 or higher |
1.62.0 | Enhancement View pull request | 8.14.0 or higher |
1.61.1 | Bug fix View pull request | 8.14.0 or higher |
1.61.0 | Enhancement View pull request | 8.14.0 or higher |
1.60.0 | Bug fix View pull request | 8.14.0 or higher |
1.59.0 | Enhancement View pull request | 8.14.0 or higher |
1.58.0 | Enhancement View pull request | 8.12.0 or higher |
1.57.0 | Enhancement View pull request | 8.12.0 or higher |
1.56.0 | Enhancement View pull request | 8.12.0 or higher |
1.55.1 | Enhancement View pull request | 8.11.0 or higher |
1.55.0 | Enhancement View pull request | 8.11.0 or higher |
1.54.0 | Enhancement View pull request | 8.11.0 or higher |
1.53.0 | Enhancement View pull request | 8.11.0 or higher |
1.52.0 | Enhancement View pull request | 8.11.0 or higher |
1.51.0 | Enhancement View pull request | 8.10.2 or higher |
1.50.0 | Enhancement View pull request | 8.10.2 or higher |
1.49.0 | Enhancement View pull request | 8.10.2 or higher |
1.48.0 | Enhancement View pull request | 8.10.2 or higher |
1.47.0 | Enhancement View pull request | 8.10.2 or higher |
1.46.0 | Enhancement View pull request | 8.10.1 or higher |
1.45.0 | Enhancement View pull request | 8.10.0 or higher |
1.44.0 | Enhancement View pull request | 8.10.0 or higher |
1.43.1 | Enhancement View pull request | 8.8.0 or higher |
1.43.0 | Enhancement View pull request | 8.8.0 or higher |
1.42.0 | Enhancement View pull request | 8.8.0 or higher |
1.41.0 | Enhancement View pull request | 8.8.0 or higher |
1.40.0 | Bug fix View pull request | 8.8.0 or higher |
1.40.0-beta.2 | Bug fix View pull request | — |
1.40.0-beta.1 | Bug fix View pull request | — |
1.40.0-beta | Enhancement View pull request | — |
1.39.0 | Enhancement View pull request | 8.6.1 or higher |
1.38.1 | Enhancement View pull request | 8.6.1 or higher |
1.38.0 | Enhancement View pull request | 8.6.1 or higher |
1.37.0 | Enhancement View pull request | 8.6.1 or higher |
1.36.0 | Enhancement View pull request | 8.6.1 or higher |
1.35.0 | Enhancement View pull request | 8.6.1 or higher |
1.34.1 | Enhancement View pull request | 8.6.1 or higher |
1.34.0 | Enhancement View pull request | 8.6.1 or higher |
1.33.0 | Enhancement View pull request | 8.6.1 or higher |
1.32.2 | Enhancement View pull request | 8.6.1 or higher |
1.32.1 | Enhancement View pull request | 8.6.1 or higher |
1.32.0 | Enhancement View pull request | 8.6.1 or higher |
1.31.2 | Enhancement View pull request | 8.6.1 or higher |
1.31.1 | Enhancement View pull request | 8.6.1 or higher |
1.31.0 | Enhancement View pull request | 8.6.0 or higher |
1.30.0 | Enhancement View pull request | 8.6.0 or higher |
1.29.2 | Bug fix View pull request | 8.5.0 or higher |
1.29.1 | Bug fix View pull request | 8.5.0 or higher |
1.29.0 | Bug fix View pull request | 8.5.0 or higher |
1.28.2 | Bug fix View pull request | 8.5.0 or higher |
1.28.1 | Enhancement View pull request | 8.5.0 or higher |
1.28.0 | Enhancement View pull request | 8.5.0 or higher |
1.27.1 | Enhancement View pull request | 8.5.0 or higher |
1.27.0 | Enhancement View pull request | 8.5.0 or higher |
1.26.0 | Enhancement View pull request | 8.4.0 or higher |
1.25.0 | Enhancement View pull request | 8.4.0 or higher |
1.24.0 | Enhancement View pull request | 8.4.0 or higher |
1.23.1 | Enhancement View pull request | 8.4.0 or higher |
1.23.0 | Enhancement View pull request | 8.4.0 or higher |
1.22.1 | Enhancement View pull request | 8.4.0 or higher |
1.22.0 | Enhancement View pull request | 8.4.0 or higher |
1.21.2 | Bug fix View pull request | 8.3.0 or higher |
1.21.1 | Enhancement View pull request | 8.3.0 or higher |
1.21.0 | Enhancement View pull request | 8.3.0 or higher |
1.20.0 | Enhancement View pull request | 8.2.0 or higher |
1.19.1 | Enhancement View pull request | 8.2.0 or higher |
1.19.0 | Enhancement View pull request | 8.2.0 or higher |
1.18.1 | Enhancement View pull request | 8.2.0 or higher |
1.18.0 | Enhancement View pull request | 8.2.0 or higher |
1.17.3 | Bug fix View pull request | 7.16.0 or higher |
1.17.2 | Bug fix View pull request | 7.16.0 or higher |
1.17.1 | Enhancement View pull request | — |
1.17.0 | Enhancement View pull request | — |
1.16.0 | Enhancement View pull request | — |
1.15.0 | Enhancement View pull request | — |
1.14.3 | Bug fix View pull request | — |
1.14.2 | Bug fix View pull request | — |
1.14.1 | Bug fix View pull request | — |
1.14.0 | Enhancement View pull request | — |
1.13.0 | Enhancement View pull request | — |
1.12.0 | Enhancement View pull request | — |
1.11.0 | Enhancement View pull request | — |
1.10.0 | Enhancement View pull request | — |
1.9.0 | Enhancement View pull request | 7.16.0 or higher |
1.8.1 | Bug fix View pull request | 7.16.0 or higher |
1.8.0 | Enhancement View pull request | 7.16.0 or higher |
1.7.0 | Enhancement View pull request | 7.16.0 or higher |
1.6.0 | Enhancement View pull request | 7.16.0 or higher |
1.5.0 | Enhancement View pull request | 7.16.0 or higher |
1.4.2 | Enhancement View pull request | — |
1.4.1 | Enhancement View pull request | 8.0.0 or higher |
1.4.0 | Enhancement View pull request | — |
1.3.3 | Bug fix View pull request | — |
1.3.2 | Enhancement View pull request | — |
1.3.1 | Enhancement View pull request | — |
1.3.0 | Enhancement View pull request | — |
1.2.1 | Bug fix View pull request | — |
1.2.0 | Enhancement View pull request | — |
1.1.1 | Bug fix View pull request | — |
1.1.0 | Enhancement View pull request | 7.15.0 or higher |
1.0.0 | Enhancement View pull request | — |
0.14.1 | Enhancement View pull request | — |
0.14.0 | Enhancement View pull request | — |
0.13.0 | Enhancement View pull request | — |
0.12.2 | Bug fix View pull request | — |
0.12.1 | Bug fix View pull request | — |
0.12.0 | Enhancement View pull request | — |
0.11.1 | Enhancement View pull request | — |
0.11.0 | Enhancement View pull request | — |
0.10.0 | Enhancement View pull request | — |
0.9.1 | Bug fix View pull request | — |
0.9.0 | Enhancement View pull request | — |
0.8.0 | Enhancement View pull request | — |
0.7.0 | Enhancement View pull request | — |
0.6.0 | Enhancement View pull request | — |
0.5.3 | Enhancement View pull request | — |
0.5.2 | Bug fix View pull request | — |
0.5.1 | Bug fix View pull request | — |
0.5.0 | Enhancement View pull request | — |
0.4.5 | Enhancement View pull request | — |
0.4.4 | Enhancement View pull request | — |
0.4.3 | Bug fix View pull request | — |
0.4.2 | Bug fix View pull request | — |
0.4.1 | Enhancement View pull request | — |
0.1.0 | Enhancement View pull request | — |