- Heartbeat Reference: other versions:
- Heartbeat overview
- Quick start: installation and configuration
- Set up and run
- Configure
- Monitors
- Task scheduler
- General settings
- Project paths
- Output
- Kerberos
- SSL
- Index lifecycle management (ILM)
- Elasticsearch index template
- Processors
- Define processors
- add_cloud_metadata
- add_cloudfoundry_metadata
- add_docker_metadata
- add_fields
- add_host_metadata
- add_id
- add_kubernetes_metadata
- add_labels
- add_locale
- add_network_direction
- add_nomad_metadata
- add_observer_metadata
- add_process_metadata
- add_tags
- append
- community_id
- convert
- copy_fields
- decode_base64_field
- decode_duration
- decode_json_fields
- decode_xml
- decode_xml_wineventlog
- decompress_gzip_field
- detect_mime_type
- dissect
- dns
- drop_event
- drop_fields
- extract_array
- fingerprint
- include_fields
- move_fields
- rate_limit
- registered_domain
- rename
- replace
- script
- syslog
- translate_ldap_attribute
- translate_sid
- truncate_fields
- urldecode
- Autodiscover
- Internal queue
- Logging
- HTTP endpoint
- Regular expression support
- Instrumentation
- Feature flags
- heartbeat.reference.yml
- How to guides
- Exported fields
- Beat fields
- Synthetics browser metrics fields
- Cloud provider metadata fields
- Common heartbeat monitor fields
- Docker fields
- ECS fields
- Host fields
- HTTP monitor fields
- ICMP fields
- Jolokia Discovery autodiscover provider fields
- Kubernetes fields
- Process fields
- Host lookup fields
- APM Service fields
- SOCKS5 proxy fields
- Monitor state fields
- Monitor summary fields
- Synthetics types fields
- TCP layer fields
- TLS encryption layer fields
- Monitor
- Secure
- Troubleshoot
- Get help
- Debug
- Understand logged metrics
- Common problems
- Heartbeat uses too much bandwidth
- Error loading config file
- Found unexpected or unknown characters
- Logstash connection doesn’t work
- Publishing to Logstash fails with "connection reset by peer" message
- @metadata is missing in Logstash
- Not sure whether to use Logstash or Beats
- SSL client fails to connect to Logstash
- Monitoring UI shows fewer Beats than expected
- High RSS memory usage due to MADV settings
- Contribute to Beats
Hints based autodiscover
editHints based autodiscover
editHeartbeat supports autodiscover based on hints from the provider. The hints system looks for
hints in Kubernetes Pod annotations or Docker labels that have the prefix co.elastic.monitor
. As soon as
the container starts, Heartbeat will check if it contains any hints and launch the proper config for
it. Hints tell Heartbeat how to get logs for the given container. By default monitors will be created
for the container that exposes the port being requested to be monitored. You can use hints to modify this behavior. This is the full
list of supported hints:
co.elastic.monitor/type
editDefine the monitor type to use. Ex: http, tcp, icmp
co.elastic.monitor/hosts
editThe URIs to monitor. Example:
co.elastic.monitor/type: icmp co.elastic.monitor/hosts: ${data.host}
This would ensure that each host has an ICMP monitor enabled on it.
co.elastic.monitor/schedule
editDefine the schedule on which the monitor should be executed.
co.elastic.monitor/schedule: "@every 5s"
co.elastic.monitor/processors
editDefine a processor to be added to the Heartbeat monitor configuration. See Processors for the list of supported processors.
In order to provide ordering of the processor definition, numbers can be provided. If not, the hints builder will do arbitrary ordering:
co.elastic.monitor/processors.1.drop_fields.fields: "field1, field2" co.elastic.monitor/processors.drop_fields.fields: "field3"
In the above sample the processor definition tagged with 1
would be executed first.
When hints are used along with templates, then hints will be evaluated only in case there is no template’s condition that resolves to true. For example:
heartbeat.autodiscover: - type: docker hints.enabled: true templates: - condition: contains: docker.container.image: redis config: - type: tcp hosts: ["${data.host}:${data.port}"] schedule: "@every 1s" timeout: 1s
In this example first the condition docker.container.image: redis
is evaluated
and if not matched the hints will be processed and if there is again no valid config
the hints.default_config
will be used.
Kubernetes
editKubernetes autodiscover provider supports hints in Pod annotations. To enable it just set hints.enabled
:
heartbeat.autodiscover: providers: - type: kubernetes hints.enabled: true
You can annotate Kubernetes Pods with useful info to spin up Heartbeat monitors:
annotations: co.elastic.monitor/type: icmp co.elastic.monitor/hosts: ${data.host} co.elastic.monitor/schedule: "@every 5s"
Multiple containers
editWhen a pod has multiple containers, the settings are shared unless you add the container name in the
hint. For example, these hints configure the container exposing port 8080 to do a HTTP check and have the sidecar
container to have an TCP check.
annotations: co.elastic.monitor/type: http co.elastic.monitor/hosts: ${data.host}:8080/healthz co.elastic.monitor/schedule: "@every 5s" co.elastic.monitor.sidecar/type: tcp co.elastic.monitor.sidecar/hosts: ${data.host}:8081 co.elastic.monitor.sidecar/schedule: "@every 5s"
Multiple sets of hints
editWhen a container needs multiple monitors to be defined on it, sets of annotations can be provided with numeric prefixes. Annotations without numeric prefixes would default into a single monitor configuration.
annotations: co.elastic.monitor/type: http co.elastic.monitor/hosts: ${data.host}:8080/healthz co.elastic.monitor/schedule: "@every 5s" co.elastic.monitor/1.type: tcp co.elastic.monitor/1.hosts: ${data.host}:8080 co.elastic.monitor/1.schedule: "@every 5s"
Docker
editDocker autodiscover provider supports hints in labels. To enable it just set hints.enabled
:
heartbeat.autodiscover: providers: - type: docker hints.enabled: true
You can label Docker containers with useful info to spin up Heartbeat monitors similar to the Kubernetes example:
LABEL co.elastic.monitor/1.type=tcp co.elastic.monitor/1.hosts='${data.host}:6379' co.elastic.monitor/1.schedule='@every 10s' LABEL co.elastic.monitor/2.type=icmp co.elastic.monitor/2.hosts='${data.host}' co.elastic.monitor/2.schedule='@every 10s'
On this page
ElasticON events are back!
Learn about the Elastic Search AI Platform from the experts at our live events.
Register now