Get up and running

edit

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

To get up and running with infrastructure monitoring, you need:

You might also need to modify UI settings in Kibana to change default behaviors, such as the index pattern used to query the data, and the timestamp field used for sorting. For more information, see Infrastructure UI Settings and Logs UI Settings.

Install Beats shippers

edit

To populate the Infrastructure UI with metrics and log data, you need to install and configure the following shippers:

To learn how to install and configure the shippers, do one of the following:

  • Follow the instructions in the Add Data section of the Kibana home page. Click Add log data or Add metrics, and follow the links for the types of data you want to collect.

    add data
  • If your data source isn’t in the list, or you want to install Beats the old fashioned way:

For either approach, you need to enable modules in Filebeat and Metricbeat to populate the Infrastructure UI with data.

Which modules and configuration options do I enable?

edit

To populate the Hosts view and add logs, enable:

To populate the Docker view and add logs, enable:

To populate the Kubernetes view and add logs, enable:

Which fields are used for the metrics on the Infrastructue home page?

edit

The metrics listed below are provided by the Beats Shippers. Each system type requires their corresponding identity field to be in the same event document:

  • Hosts require host.name
  • Docker containers require container.id
  • Kibernetes pods require kibernetes.pod.uid
Host Metrics
edit
CPU Usage
Average of system.cpu.user.pct added to the average of system.cpu.system.pct divided by system.cpu.cores
Memory Usage
Average of system.memory.actual.used.pct
Load
Average of system.load.5
Inbound Traffic
Derivative of the max of system.netowrk.in.bytes scaled to a 1 second rate
Outbound Traffic
Derivative of the max of system.netowrk.out.bytes scaled to a 1 second rate
Log Rate
Derivative of the cumulative sum of the document count scaled to a 1 second rate. This metric relies on the same indices as the logs.
Docker Container Metrics
edit
CPU Usage
Average of docker.cpu.total.pct
Memory Usage
Average of docker.memory.usage.pct
Inbound Traffic
Derivative of the max of docker.network.in.bytes scaled to a 1 second rate
Outbound Traffic
Derivative of the max of docker.network.out.bytes scaled to a 1 second rate
Kubernetes Pod Metrics
edit
CPU Usage
Average of kubernetes.pod.cpu.usage.node.pct
Memory Usage
Average of kubernetes.pod.memory.usage.node.pct
Inbound Traffic
Derivative of the max of kubernetes.pod.network.rx.bytes scaled to a 1 second rate
Outbound Traffic
Derivative of the max of kubernetes.pod.network.tx.bytes scaled to a 1 second rate

More about container monitoring

edit

If you’re monitoring containers, you can use autodiscover to automatically apply configuration changes in response to changes in your containers. To learn how, see:

Known Workarounds

edit
Running Metricbeat ⇐6.5 with Kibana 6.6+
For Kubernetes, you will need to change the "Pod ID" field in the "Configure Source" panel (xpack.infra.sources.default.fields.pod in config/kibana.yml for Kibana 6.6) to kubernetes.pod.name. There is a caveat for this workaround; if you have two pods with the same name, only one will be visible in the UI.