Configuration Examples

Below you can find manifests that address a number of common use cases and can be your starting point in exploring Beats deployed with ECK. These manifests are self-contained and work out-of-the-box on any non-secured Kubernetes cluster. They all contain three-node Elasticsearch cluster and single Kibana instance. All Beat configurations set up Kibana dashboards if they are available for a given Beat and all required RBAC resources.

The examples in this section are purely descriptive and should not be considered to be production-ready. Some of these examples use the node.store.allow_mmap: false setting which has performance implications and should be tuned for production workloads, as described in Virtual memory.

Metricbeat for Kubernetes monitoring

edit

kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.9/config/recipes/beats/metricbeat_hosts.yaml

Deploys Metricbeat as a DaemonSet that monitors the usage of the following resources:

Host: CPU, memory, network, filesystem.
Kubernetes: Nodes, Pods, Containers, Volumes.

Filebeat with autodiscover

edit

kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.9/config/recipes/beats/filebeat_autodiscover.yaml

Deploys Filebeat as a DaemonSet with the autodiscover feature enabled. It collects logs from Pods in every namespace and loads them to the connected Elasticsearch cluster.

Filebeat with autodiscover for metadata

edit

kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.9/config/recipes/beats/filebeat_autodiscover_by_metadata.yaml

Deploys Filebeat as a DaemonSet with the autodiscover feature enabled. Logs from Pods that match the following criteria are shipped to the connected Elasticsearch cluster:

Pod is in log-namespace namespace
Pod has log-label: "true" label

Filebeat without autodiscover

edit

kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.9/config/recipes/beats/filebeat_no_autodiscover.yaml

Deploys Filebeat as a DaemonSet with the autodiscover feature disabled. Uses the entire logs directory on the host as the input source. This configuration does not require any RBAC resources as no Kubernetes APIs are used.

Elasticsearch and Kibana Stack Monitoring

edit

kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.9/config/recipes/beats/stack_monitoring.yaml

Deploys Metricbeat configured for Elasticsearch and Kibana Stack Monitoring and Filebeat using autodiscover. Deploys one monitored Elasticsearch cluster and one monitoring Elasticsearch cluster. You can access the Stack Monitoring app in the monitoring cluster’s Kibana.

In this example, TLS verification is disabled when Metricbeat communicates with the monitored cluster, which is not secure and should not be used in production. To solve this, use custom certificates and configure Metricbeat to verify them.

Heartbeat monitoring Elasticsearch and Kibana health

edit

kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.9/config/recipes/beats/heartbeat_es_kb_health.yaml

Deploys Heartbeat as a single Pod deployment that monitors the health of Elasticsearch and Kibana by TCP probing their Service endpoints. Heartbeat expects that Elasticsearch and Kibana are deployed in the default namespace.

Auditbeat

edit

kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.9/config/recipes/beats/auditbeat_hosts.yaml

Deploys Auditbeat as a DaemonSet that checks file integrity and audits file operations on the host system.

Journalbeat

edit

kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.9/config/recipes/beats/journalbeat_hosts.yaml

Deploys Journalbeat as a DaemonSet that ships data from systemd journals.

Packetbeat monitoring DNS and HTTP traffic

edit

kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.9/config/recipes/beats/packetbeat_dns_http.yaml

Deploys Packetbeat as a DaemonSet that monitors DNS on port 53 and HTTP(S) traffic on ports 80, 8000, 8080 and 9200.

OpenShift monitoring

edit

kubectl apply -f https://raw.githubusercontent.com/elastic/cloud-on-k8s/1.9/config/recipes/beats/openshift_monitoring.yaml

Deploys Metricbeat as a DaemonSet that monitors the host resource usage (CPU, memory, network, filesystem), OpenShift resources (Nodes, Pods, Containers, Volumes), API Server and Filebeat using autodiscover. Deploys an Elasticsearch cluster and Kibana to centralize data collection.

« Configuration Troubleshooting »