Beats version 6.4.0

Known issue

Due to a packaging mistake, the modules.d configuration directory is installed in the wrong path in the Metricbeat DEB and RPM packages. This issue results in an empty list when you run metricbeat modules list and failures when you try to enable or disable modules. To work around this issue, run the following command:

sudo cp -r /usr/share/metricbeat/modules.d /etc/metricbeat/

This issue affects all new installations on DEB and RPM. Upgrades will run, but use old configurations defined in the modules.d directory from the previous installation.

The issue will be fixed in the 6.4.1 release.

Breaking changes

edit

Affecting all Beats

Set default kafka version to 1.0.0 in kafka output. Older versions are still supported by configuring the version setting. Minimally supported version is 0.11 (older versions might work, but are untested). 7025

Heartbeat

Rename http.response.status to http.response.status_code to align with ECS. 7274
Remove type field as not needed. 7307

Metricbeat

Fixed typo in values for state_container status.phase, from terminate to terminated. 6916
RabbitMQ management plugin path is now configured at the module level instead of having to do it in each of the metricsets. New management_path_prefix option should be used now 7074
RabbitMQ node metricset only collects metrics of the instance it connects to, node.collect: cluster can be used to collect all nodes as before. 6556 6971
Change http/server metricset to put events by default under http.server and prefix config options with server.. 7100
Disable dedotting in docker module configuration. This will change the out-of-the-box behaviour, but not the one of already configured instances. 7485
Fix typo in etcd/self metricset fields from *.bandwithrate to *.bandwidthrate. 7456
Changed the definition of the system.cpu.total.pct and system.cpu.total.norm.cou fields to exclude the IOWait time. 7691

Bugfixes

edit

Affecting all Beats

Error out on invalid Autodiscover template conditions settings. 7200
Allow to override the ignore_above option when defining new field with the type keyword. 7238
Fix a panic on the Dissect processor when we have data remaining after the last delimiter. 7449
When we fail to build a Kubernetes' indexer or matcher we produce a warning but we don’t add them to the execution. 7466
Fix default value for logging.files.keepfiles. It was being set to 0 and now it’s set to the documented value of 7. 7494
Retain compatibility with older Docker server versions. 7542
Fix errors unpacking configs modified via CLI by ignoring -E key=value pairs with missing value. 7599

Auditbeat

Allow auditbeat setup to run without requiring elevated privileges for the audit client. 7111
Fix goroutine leak that occurred when the auditd module was stopped. 7163

Filebeat

Fix a data race between stopping and starting of the harvesters. 6879
Fix an issue when parsing ISO8601 dates with timezone definition 7367
Fix Grok pattern of MongoDB module. 7568
Fix registry duplicates and log resending on upgrade. 7634

Metricbeat

Fix Windows service metricset when using a 32-bit binary on a 64-bit OS. 7294
Do not report Metricbeat container host as hostname in Kubernetes deployment. 7199
Ensure metadata updates don’t replace existing pod metrics. 7573
Fix kubernetes pct fields reporting. 7677
Add support for new kube_node_status_condition in Kubernetes state_node. 7699

Added

edit

Affecting all Beats

Add dissect processor. 6925
Add IP-addresses and MAC-addresses to add_host_metadata. 6878
Added a seccomp (secure computing) filter on Linux that whitelists the necessary system calls used by each Beat. 5213
Ship fields.yml as part of the binary 4834
Added options to dev-tools/cmd/dashboards/export_dashboard.go: -indexPattern to include index-pattern in output, -quiet to be quiet. 7101
Add Indexer indexing by pod uid. Enable pod uid metadata gathering in add_kubernetes_metadata. Extended Matcher log_path matching to support volume mounts 7072
Add default_fields to Elasticsearch template when connecting to Elasticsearch >= 7.0. 7015
Add support for loading a template.json file directly instead of using fields.yml. 7039
Add support for keyword multifields in field.yml. 7131
Add experimental Jolokia Discovery autodiscover provider. 7141
Add owner object info to Kubernetes metadata. 7231
Add Beat export dashboard command. 7239
Add support for docker autodiscover to monitor containers on host network 6708
Add ability to define input configuration as stringified JSON for autodiscover. 7372
Add processor definition support for hints builder 7386
Add support to disable html escaping in outputs. 7445
Refactor error handing in schema.Apply(). 7335
Add additional types to Kubernetes metadata 7457
Add module state reporting for Beats Monitoring. 7075
Release the rename processor as GA. 7656
Add support for Openstack Nova in add_cloud_metadata processor. 7663
Add support to set Beats services to automatic-delayed start on Windows. 8711

Auditbeat

Added XXH64 hash option for file integrity checks. 7311
Added the show auditd-rules and show auditd-status commands to show kernel rules and status. 7114
Add Kubernetes specs for auditbeat file integrity monitoring 7642

Filebeat

Add Kibana module with log fileset. 7052
Support MySQL 5.7.19 by mysql/slowlog 6969
Correctly join partial log lines when using docker input. 6967
Add support for TLS with client authentication to the TCP input 7056
Converted part of pipeline from treafik/access metricSet to dissect to improve efficiency. 7209
Add GC fileset to the Elasticsearch module. 7305
Add Audit log fileset to the Elasticsearch module. 7365
Add Slow log fileset to the Elasticsearch module. 7473
Add deprecation fileset to the Elasticsearch module. 7474
Add convert_timezone option to Kafka module to convert dates to UTC. 7546 7578
Add patterns for kafka 1.1 logs. 7608
Move debug messages in tcp input source 7712

Metricbeat

Add experimental Elasticsearch index metricset. 6881
Add dashboards and visualizations for haproxy metrics. 6934
Add Jolokia agent in proxy mode. 6475
Add message rates to the RabbitMQ queue metricset 6442 6606
Add exchanges metricset to the RabbitMQ module 6442 6607
Add Elasticsearch index_summary metricset. 6918
Add shard metricset to Elasticsearch module. 7006
Add apiserver metricset to Kubernetes module. 7059
Add maxmemory to redis info metricset. 7127
Set guest as default user in RabbitMQ module. 7107
Add postgresql statement metricset. 7048 7060
Update state_container metricset to support latest kube-state-metrics version. 7216
Add TLS support to MongoDB module. 7401
Added Traefik module with health metricset. 7413
Add Elasticsearch ml_job metricsets. 7196
Add support for bearer token files to HTTP helper. 7527
Add Elasticsearch index recovery metricset. 7225
Add locks, global_locks, oplatencies and process fields to status metricset of MongoDB module. 7613
Run Kafka integration tests on version 1.1.0 7616
Release raid and socket metricset from system module as GA. 7658
Release elasticsearch module and all its metricsets as beta. 7662
Release munin and traefik module as beta. 7660
Add envoyproxy module. 7569
Release prometheus collector metricset as GA. 7660
Add Elasticsearch cluster_stats metricset. 7638
Added basepath setting for HTTP-based metricsets 7700
Add couchdb module. 9406

Packetbeat

The process monitor now reports the command-line for all processes, under Linux and Windows. 7135
Updated the TLS protocol parser with new cipher suites added to TLS 1.3. 7455
Flows are enriched with process information using the process monitor. 7507
Added UDP support to process monitor. 7571

Deprecated

edit

Metricbeat

Kubernetes state_container cpu.limit.nanocores and cpu.request.nanocores have been deprecated in favor of cpu.*.cores. 6916

« Beats version 6.4.1 Beats version 6.3.1 »