Network policies

Network policies allow you to isolate pods by restricting incoming and outgoing network connections to a trusted set of sources and destinations. This section describes how to use network policies to isolate the ECK operator and the Elastic Stack applications to a set of namespaces to implement a form of soft multi-tenancy. Soft multi-tenancy is a term used to describe a scenario where a group of trusted users (different teams within an organization, for example) share a single resource such as a Kubernetes cluster. Note that network policies alone are not sufficient for security. You should complement them with strict RBAC policies, resource quotas, node taints, and other available security mechanisms to ensure that tenants cannot access, modify, or disrupt resources belonging to each other.

There are several efforts to support multi-tenancy on Kubernetes, including the official working group for multi-tenancy and community extensions such as loft and kiosk, that can make configuration and management easier. You may still need to employ network policies such the ones described below to have fine-grained control over Elastic Stack applications deployed by your tenants.

The following sections assume that the operator is installed in the elastic-system namespace with the namespaces configuration set to team-a,team-b. Each namespace is expected to be labelled as follows:

kubectl label namespace elastic-system eck.k8s.elastic.co/operator-name=elastic-operator
kubectl label namespace team-a eck.k8s.elastic.co/tenant=team-a
kubectl label namespace team-b eck.k8s.elastic.co/tenant=team-b

Isolating the operator

edit

The minimal set of permissions required are as follows:

Egress (outgoing)	TCP port 443 of the Kubernetes API server. UDP port 53 for DNS lookup. TCP port 9200 of Elasticsearch nodes on managed namespace.
Ingress (incoming)	TCP port 9443 for webhook requests from the Kubernetes API server.

Assuming that the Kubernetes API server IP address is 10.0.0.1, the following network policy implements the rules above.

Run kubectl cluster-info | grep master to obtain the API server IP address for your cluster.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: elastic-operator
  namespace: elastic-system
spec:
  egress:
  - ports:
    - port: 53
      protocol: UDP
  - ports:
    - port: 443
      protocol: TCP
    to:
    - ipBlock:
        cidr: 10.0.0.1/32
  - ports:
    - port: 9200
      protocol: TCP
    to:
    - namespaceSelector:
        matchExpressions:
        - key: eck.k8s.elastic.co/tenant
          operator: In
          values:
          - team-a
          - team-b
      podSelector:
        matchLabels:
          common.k8s.elastic.co/type: elasticsearch
  ingress:
  - from:
    - ipBlock:
        cidr: 10.0.0.1/32
    ports:
    - port: 9443
      protocol: TCP
  podSelector:
    matchLabels:
      control-plane: elastic-operator

Isolating Elasticsearch

edit

Egress (outgoing)	TCP port 9300 to other Elasticsearch nodes in the namespace (transport port). UDP port 53 for DNS lookup.
Ingress (incoming)	TCP port 9200 from the operator and other pods in the namespace.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: eck-elasticsearch
  namespace: team-a
spec:
  egress:
  - ports:
    - port: 9300
      protocol: TCP
    to:
    - namespaceSelector:
        matchLabels:
          eck.k8s.elastic.co/tenant: team-a
      podSelector:
        matchLabels:
          common.k8s.elastic.co/type: elasticsearch
  - ports:
    - port: 53
      protocol: UDP
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          eck.k8s.elastic.co/operator-name: elastic-operator
      podSelector:
        matchLabels:
          control-plane: elastic-operator
    - namespaceSelector:
        matchLabels:
          eck.k8s.elastic.co/tenant: team-a
    # [Optional] Allow ingress controller pods from the ingress-nginx namespace.
    #- namespaceSelector:
    #    matchLabels:
    #      name: ingress-nginx
    ports:
    - port: 9200
      protocol: TCP
  - from:
    - namespaceSelector:
        matchLabels:
          eck.k8s.elastic.co/tenant: team-a
      podSelector:
        matchLabels:
          common.k8s.elastic.co/type: elasticsearch
    ports:
    - port: 9300
      protocol: TCP
  podSelector:
    matchLabels:
      common.k8s.elastic.co/type: elasticsearch

Isolating Kibana

edit

Egress (outgoing)	TCP port 9200 to Elasticsearch nodes in the namespace. UDP port 53 for DNS lookup.
Ingress (incoming)	TCP port 5601 from other pods in the namespace.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: eck-kibana
  namespace: team-a
spec:
  egress:
  - ports:
    - port: 9200
      protocol: TCP
    to:
    - namespaceSelector:
        matchLabels:
          eck.k8s.elastic.co/tenant: team-a
      podSelector:
        matchLabels:
          common.k8s.elastic.co/type: elasticsearch
          # [Optional] Restrict to a single Elasticsearch cluster named hulk.
          # elasticsearch.k8s.elastic.co/cluster-name=hulk
  - ports:
    - port: 53
      protocol: UDP
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          eck.k8s.elastic.co/tenant: team-a
    # [Optional] Allow ingress controller pods from the ingress-nginx namespace.
    #- namespaceSelector:
    #    matchLabels:
    #      name: ingress-nginx
    ports:
    - port: 5601
      protocol: TCP
  podSelector:
    matchLabels:
      common.k8s.elastic.co/type: kibana

Isolating APM Server

edit

Egress (outgoing)	TCP port 9200 to Elasticsearch nodes in the namespace. TCP port 5601 to Kibana instances in the namespace. UDP port 53 for DNS lookup.
Ingress (incoming)	TCP port 8200 from other pods in the namespace.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: eck-apm-server
  namespace: team-a
spec:
  egress:
  - ports:
    - port: 9200
      protocol: TCP
    to:
    - namespaceSelector:
        matchLabels:
          eck.k8s.elastic.co/tenant: team-a
      podSelector:
        matchLabels:
          common.k8s.elastic.co/type: elasticsearch
  - ports:
    - port: 5601
      protocol: TCP
    to:
    - namespaceSelector:
        matchLabels:
          eck.k8s.elastic.co/tenant: team-a
      podSelector:
        matchLabels:
          common.k8s.elastic.co/type: kibana
  - ports:
    - port: 53
      protocol: UDP
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          eck.k8s.elastic.co/tenant: team-a
    # [Optional] Allow ingress controller pods from the ingress-nginx namespace.
    #- namespaceSelector:
    #    matchLabels:
    #      name: ingress-nginx
    ports:
    - port: 8200
      protocol: TCP
  podSelector:
    matchLabels:
      common.k8s.elastic.co/type: apm-server

Isolating Enterprise Search

edit

Egress (outgoing)	TCP port 9200 to Elasticsearch nodes in the namespace. UDP port 53 for DNS lookup.
Ingress (incoming)	TCP port 3002 from other pods in the namespace.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: eck-enterprise-search
  namespace: team-a
spec:
  egress:
  - ports:
    - port: 9200
      protocol: TCP
    to:
    - namespaceSelector:
        matchLabels:
          eck.k8s.elastic.co/tenant: team-a
      podSelector:
        matchLabels:
          common.k8s.elastic.co/type: elasticsearch
  - ports:
    - port: 53
      protocol: UDP
  ingress:
  - from:
    - namespaceSelector:
        matchLabels:
          eck.k8s.elastic.co/tenant: team-a
    # [Optional] Allow ingress controller pods from the ingress-nginx namespace.
    #- namespaceSelector:
    #    matchLabels:
    #      name: ingress-nginx
    ports:
    - port: 3002
      protocol: TCP
  podSelector:
    matchLabels:
      common.k8s.elastic.co/type: enterprise-search

Isolating Beats

edit

Some Beats may require additional access rules than what is listed here. For example, Heartbeat will require a rule to allow access to the endpoint it is monitoring.

Egress (outgoing)

TCP port 9200 to Elasticsearch nodes in the namespace.
TCP port 5601 to Kibana instances in the namespace.
UDP port 53 for DNS lookup.

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: eck-beats
  namespace: team-a
spec:
  egress:
  - ports:
    - port: 9200
      protocol: TCP
    to:
    - namespaceSelector:
        matchLabels:
          eck.k8s.elastic.co/tenant: team-a
      podSelector:
        matchLabels:
          common.k8s.elastic.co/type: elasticsearch
  - ports:
    - port: 5601
      protocol: TCP
    to:
    - namespaceSelector:
        matchLabels:
          eck.k8s.elastic.co/tenant: team-a
      podSelector:
        matchLabels:
          common.k8s.elastic.co/type: kibana
  - ports:
    - port: 53
      protocol: UDP
  podSelector:
    matchLabels:
      common.k8s.elastic.co/type: beat

« Traffic Splitting Reference »