Stack Monitoring

You can enable Stack Monitoring on Elasticsearch, Kibana and Beats to collect and ship their metrics and logs to a dedicated monitoring cluster.

To enable Stack Monitoring, simply reference the monitoring Elasticsearch cluster in the spec.monitoring section of their specification.

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: monitored-sample
  namespace: production
spec:
  version: 8.16.0
  monitoring:
    metrics:
      elasticsearchRefs:
      - name: monitoring
        namespace: observability 
    logs:
      elasticsearchRefs:
      - name: monitoring
        namespace: observability 
  nodeSets:
  - name: default
    count: 1
    config:
      node.store.allow_mmap: false
---
apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: monitored-sample
  namespace: production
spec:
  version: 8.16.0
  elasticsearchRef:
    name: monitored-sample
    namespace: production 
  monitoring:
    metrics:
      elasticsearchRefs:
      - name: monitoring
        namespace: observability 
    logs:
      elasticsearchRefs:
      - name: monitoring
        namespace: observability 
  count: 1
---
apiVersion: beat.k8s.elastic.co/v1beta1
kind: Beat
metadata:
  name: monitored-sample
spec:
  type: filebeat
  version: 8.16.0
  monitoring:
    metrics:
      elasticsearchRefs:
      - name: monitoring
        namespace: observability 
    logs:
      elasticsearchRefs:
      - name: monitoring
        namespace: observability

	The use of `namespace` is optional if the monitoring Elasticsearch cluster and the monitored Elasticsearch cluster are running in the same namespace.
	The use of `namespace` is optional if the Elasticsearch cluster and the Kibana instance are running in the same namespace.
	The use of `namespace` is optional if the Elasticsearch cluster and the Beats instance are running in the same namespace.

You can configure an Elasticsearch cluster to monitor itself.

If Stack Monitoring is configured for a Beat, but the corresponding Elasticsearch cluster is not monitored, the Kibana Stack Monitoring page will not show the Beats data.

If Logs Stack Monitoring is configured for a Beat, and custom container arguments (podTemplate.spec.containers[].args) include -e, which enables logging to stderr and disables log file output, this argument will be removed from the Pod to allow the Filebeat sidecar to consume the Beat’s log files.

The monitoring cluster must be managed by ECK in the same Kubernetes cluster as the monitored one.

You can send metrics and logs to two different Elasticsearch monitoring clusters.

You can also enable Stack Monitoring on Elasticsearch only or on Kibana only. In the latter case, Kibana will not be available on the Stack Monitoring Kibana page (check View monitoring data in Kibana).

« Webhook namespace selectors Connect to an external monitoring Elasticsearch cluster »