- Heartbeat Reference: other versions:
- Heartbeat overview
- Quick start: installation and configuration
- Set up and run
- Configure
- Monitors
- Task scheduler
- General settings
- Project paths
- Output
- Kerberos
- SSL
- Index lifecycle management (ILM)
- Elasticsearch index template
- Processors
- Define processors
- add_cloud_metadata
- add_cloudfoundry_metadata
- add_docker_metadata
- add_fields
- add_host_metadata
- add_id
- add_kubernetes_metadata
- add_labels
- add_locale
- add_network_direction
- add_nomad_metadata
- add_observer_metadata
- add_process_metadata
- add_tags
- append
- community_id
- convert
- copy_fields
- decode_base64_field
- decode_duration
- decode_json_fields
- decode_xml
- decode_xml_wineventlog
- decompress_gzip_field
- detect_mime_type
- dissect
- dns
- drop_event
- drop_fields
- extract_array
- fingerprint
- include_fields
- move_fields
- rate_limit
- registered_domain
- rename
- replace
- script
- syslog
- translate_ldap_attribute
- translate_sid
- truncate_fields
- urldecode
- Autodiscover
- Internal queue
- Logging
- HTTP endpoint
- Regular expression support
- Instrumentation
- Feature flags
- heartbeat.reference.yml
- How to guides
- Exported fields
- Beat fields
- Synthetics browser metrics fields
- Cloud provider metadata fields
- Common heartbeat monitor fields
- Docker fields
- ECS fields
- Host fields
- HTTP monitor fields
- ICMP fields
- Jolokia Discovery autodiscover provider fields
- Kubernetes fields
- Process fields
- Host lookup fields
- APM Service fields
- SOCKS5 proxy fields
- Monitor state fields
- Monitor summary fields
- Synthetics types fields
- TCP layer fields
- TLS encryption layer fields
- Monitor
- Secure
- Troubleshoot
- Get help
- Debug
- Understand logged metrics
- Common problems
- Heartbeat uses too much bandwidth
- Error loading config file
- Found unexpected or unknown characters
- Logstash connection doesn’t work
- Publishing to Logstash fails with "connection reset by peer" message
- @metadata is missing in Logstash
- Not sure whether to use Logstash or Beats
- SSL client fails to connect to Logstash
- Monitoring UI shows fewer Beats than expected
- High RSS memory usage due to MADV settings
- Contribute to Beats
Add Docker metadata
editAdd Docker metadata
editThe add_docker_metadata
processor annotates each event with relevant metadata
from Docker containers. At startup it detects a docker environment and caches the metadata.
The events are annotated with Docker metadata, only if a valid configuration
is detected and the processor is able to reach Docker API.
Each event is annotated with:
- Container ID
- Name
- Image
- Labels
When running Heartbeat in a container, you need to provide access to
Docker’s unix socket in order for the add_docker_metadata
processor to work.
You can do this by mounting the socket inside the container. For example:
docker run -v /var/run/docker.sock:/var/run/docker.sock ...
To avoid privilege issues, you may also need to add --user=root
to the
docker run
flags. Because the user must be part of the docker group in order
to access /var/run/docker.sock
, root access is required if Heartbeat is
running as non-root inside the container.
If Docker daemon is restarted the mounted socket will become invalid and metadata will stop working, in these situations there are two options:
- Restart Heartbeat every time Docker is restarted
-
Mount the entire
/var/run
directory (instead of just the socket)
processors: - add_docker_metadata: host: "unix:///var/run/docker.sock" #match_fields: ["system.process.cgroup.id"] #match_pids: ["process.pid", "process.parent.pid"] #match_source: true #match_source_index: 4 #match_short_id: true #cleanup_timeout: 60 #labels.dedot: false # To connect to Docker over TLS you must specify a client and CA certificate. #ssl: # certificate_authority: "/etc/pki/root/ca.pem" # certificate: "/etc/pki/client/cert.pem" # key: "/etc/pki/client/cert.key"
It has the following settings:
-
host
-
(Optional) Docker socket (UNIX or TCP socket). It uses
unix:///var/run/docker.sock
by default. -
ssl
- (Optional) SSL configuration to use when connecting to the Docker socket.
-
match_fields
- (Optional) A list of fields to match a container ID, at least one of them should hold a container ID to get the event enriched.
-
match_pids
-
(Optional) A list of fields that contain process IDs. If the
process is running in Docker then the event will be enriched. The default value
is
["process.pid", "process.parent.pid"]
. -
match_source
-
(Optional) Match container ID from a log path present in the
log.file.path
field. Enabled by default. -
match_short_id
-
(Optional) Match container short ID from a log path present
in the
log.file.path
field. Disabled by default. This allows to match directories names that have the first 12 characters of the container ID. For example,/var/log/containers/b7e3460e2b21/*.log
. -
match_source_index
-
(Optional) Index in the source path split by
/
to look for container ID. It defaults to 4 to match/var/lib/docker/containers/<container_id>/*.log
-
cleanup_timeout
- (Optional) Time of inactivity to consider we can clean and forget metadata for a container, 60s by default.
-
labels.dedot
-
(Optional) Default to be false. If set to true, replace dots in
labels with
_
.