Prometheus requirements

edit

The previous options requires the following settings within Prometheus to function properly:

RBAC settings for scraping the metrics

edit

Configure the RBAC settings for the Prometheus instance to access the metrics endpoint similar to the following: (These typically will be set automatically when using the Prometheus operator)

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: prometheus
rules:
- nonResourceURLs:
  - /metrics
  verbs:
  - get

Optional Prometheus operator Helm settings to allow reading PodMonitor and ServiceMonitor across namespaces

edit
  • If using the Prometheus operator and your Prometheus instance is not in the same namespace as the ECK operator you will need the Prometheus operator configured with the following Helm values:
prometheus:
  prometheusSpec:
    podMonitorNamespaceSelector: {}
    podMonitorSelectorNilUsesHelmValues: false
    serviceMonitorNamespaceSelector: {}
    serviceMonitorSelectorNilUsesHelmValues: false

Optional settings to allow full TLS verification when using a custom TLS certificate

edit

If you are using a custom TLS certificate and you need to set insecureSkipVerify to false you will need to do the following:

  • Create a Kubernetes secret within the Prometheus namespace that contains the Certificate Authority in PEM format.

The easiest way to create the CA secret within the Prometheus namespace is to use the kubectl create secret generic command. For example:

kubectl create secret generic eck-metrics-tls-ca -n monitoring --from-file=ca.crt=/path/to/ca.pem
  • Ensure that the CA secret is mounted within the Prometheus Pod.

This will vary between Prometheus installations, but if using the Prometheus operator you can set the spec.secrets field of the Prometheus custom resource to the name of the previously created Kubernetes Secret. See the ECK Helm chart values file for more information.