IMPORTANT: No additional bug fixes or documentation updates
will be released for this version. For the latest information, see the
current release documentation.
Beats version 7.7.0
editBeats version 7.7.0
editBreaking changes
editAffecting all Beats
- Environment variables can no longer reference other environment variables or objects. 15937
-
Change
aws_elb
autodiscovery provider field name fromelb_listener.*
toaws.elb.*
. 16219 16402 -
Remove support for using
add_docker_metadata
andadd_kubernetes_metadata
processors from thescript
processor. They can still be used as normal processors in the configuration. 16349 16514
Bugfixes
editAffecting all Beats
- Fix Kubernetes autodiscovery provider to correctly handle pod states and avoid missing event data. 17223
-
Fix
add_cloud_metadata
processor to better support modifying sub-fields with other processors. 13808 - Fix panic in the Logstash output when trying to send events to closed connection. 15568
- Fix logging target settings being ignored when Beats are started via systemd or docker. 12024 15442
- Fix issue where default go logger is not discarded when either * or stdout is selected. 10251 15708
-
Remove superfluous use of
number_of_routing_shards
setting from the default template. 16038 - Automatically convert index names to lowercase. 16081
- Fix loading processor annotation hints, allowing the value to be a full configuration section. 16348
-
Add
ssl.ca_sha256
to the list of supported TLS options. This option allows you to check that a specific certificate is used as part of the verified chain. 15717 -
Fix
NewContainerMetadataEnricher
to use default config for kubernetes module. No longer requires the user to havelabels.dedot: true
in the configuration as it is now properly the default. 16857 -
Improve logging messages for the
add_kubernetes_metadata
processor. 16866 - Fail to start if httpprof is used and it cannot be initialized. 17028
- Fix concurrency issues in convert processor when used in the global context. 17032
-
Fix bug with
monitoring.cluster_uuid
setting not always being exposed via GET /state Beats API. 16732 17420 -
Fix building on FreeBSD by removing build flags from
add_cloudfoundry_metadata
processor. 17486
Filebeat
- Fix mapping error when zeek weird logs do not contain IP addresses. 15906
- Fix merging of fileset inputs to replace paths and append processors. 16450
-
Fix Elasticsearch
_id
field set by S3 and Google Pub/Sub inputs. 17026 - Fix various Cisco FTD parsing issues. 16863 16889
- Fix default index pattern in IBM MQ Filebeat dashboard. 17146
- Fix a mapping exception when ingesting Logstash plain logs (7.4+) with pipeline ids containing non alphanumeric chars. 17242 17243
- Fix MySQL slowlog module causing "regular expression has redundant nested repeat operator" warning in Elasticsearch. 17086 17156
-
Fix
elasticsearch.audit
data ingest pipeline to be more forgiving with date formats found in Elasticsearch audit logs. 17406 - Fix decoding errors caused by trailing spaces in CEF messages. 17253
- Fix activemq module causing "regular expression has redundant nested repeat operator" warning in Elasticsearch. 17428
Metricbeat
-
Change
lookup_fields
setting frommetricset.host
toservice.address
. 15883 -
Make
logstash-xpack
module once again have parity with internally-collected Logstash monitoring data. 16198 -
Improve metrics collection in the
system/service
metricset on older linux distributions. 16902 - Use max in k8s apiserver dashboard aggregations. 17018
-
Check if CCR feature is available on Elasticsearch cluster before attempting to call CCR APIs from
elasticsearch/ccr
metricset. 16511 17073 - Use max in k8s overview dashboard aggregations. 17015
- Fix Disk Used and Disk Usage visualizations in the Metricbeat System dashboards. 12435 17272
- Fix missing Accept header for Prometheus and OpenMetrics module. 16870 17291
- Combine cloudwatch aggregated metrics into single event. 17345
- Fix how we filter services by name in system/service. 17400
-
Fix problem where
cloudwatch
metricset was not collecting tags correctly. 17419 17424 - Check if cpuOptions field is nil in DescribeInstances output in ec2 metricset. 17418
-
Fix
aws.s3.bucket.name
terms_field in s3 overview dashboard. 17542 - Fix Unix socket path in memcached module. 17512
- Fix vsphere VM dashboard host aggregation visualizations. 17555
Added
editAffecting all Beats
-
Include network information by default when using the
add_host_metadata
oradd_observer_metadata
processor. 15347 16077 -
Add
aws_ec2
provider for autodiscovery. 12518 14823 - Add support for multiple passwords in redis output. 16058 16206
- Add support for Histogram type in fields.yml. 16570
- Windows .exe files now have embedded file version info. 15232
-
Remove experimental flag from
setup.template.append_fields
. 16576 -
Add
add_cloudfoundry_metadata
processor to annotate events with Cloud Foundry application data. 16621 -
Add
translate_sid
processor on Windows for converting Windows security identifier (SID) values to names. 7451 16013 - Add support for Kubernetes provider to recognize namespace level defaults. 16321
-
Add ability to enrich the
container.id
with the process id by using theadd_process_metadata
processor. 15947 - Update RPM packages contained in Beat Docker images. 17035
- Add Kerberos support to Kafka input and output. 16781
Auditbeat
- Add examples to the kubernetes manifests to show how to configure the auditd module and use processors to enrich events with metadata.
- In the kubernetes manifests, mount the data directory from the host, so data persist between executions in the same node. 17429
- Log to stderr when using kubernetes manifests. 174443
- Fix memory leak on when we miss socket close kprobe events. 17500
Filebeat
- Add ECS tls fields to the smtp, rdp, and ssl filesets in the zeek module, and the s3access and elb filesets in the aws module. 15757 15936
-
Add Nginx
ingress_controller
fileset. 16197 - Add ECS tls and categorization fields to apache module. 16032 16121
- Add MQTT input. 15602 16204
- Improve ECS categorization, container, and process field mappings in auditd module. 16153 16280
- Add ECS categorization fields to activemq module. 16151 16201
- Improve ECS field mappings in aws module. 16154 16307
- Improve ECS categorization field mappings in googlecloud module. 16030 16500
-
Add
cloudwatch
andec2
filesets to aws module. 13716 16579 - Improve ECS categorization field mappings in kibana module. 16168 16652
-
Add
cloudfoundry
input to send events from Cloud Foundry. 16586 - Improve ECS field mappings in haproxy module. 16162 16529
- Allow users to override pipeline ID in fileset input config. 9531 16561
- Improve ECS categorization field mappings in logstash module. 16169 16668
- Improve ECS categorization field mappings in iis module. 16165 16618
-
Improve the
decode_cef
processor by reducing the number of memory allocations. 16587 - Improve ECS categorization field mapping in kafka module. 16167 16645
- Improve ECS categorization field mapping in icinga module. 16164 16533
- Improve ECS categorization field mappings in ibmmq module. 16163 16532
- Add custom string mapping to CEF module to support Forcepoint NGFW. 14663 15910
- Add ECS fields to CEF module. 16157 16338
- Improve ECS categorization and host field mappings in elasticsearch module. 16160 16469
- Improve ECS categorization field mappings in suricata module. 16181 16843
- Release ActiveMQ module as GA. 17047 17049
- Improve ECS categorization field mappings in iptables module. 16166 16637
- Add pattern for Cisco ASA / FTD Message 734001. 16212 16612
-
Add
o365audit
input type for consuming events from Office 365 Management Activity API. 16196 16244 - Add custom string mapping to CEF module to support Check Point devices. 16041 16907
-
Add
o365
module for ingesting Office 365 management activity API events. 16196 16386 - Add Okta module. 16362
- Improve AWS cloudtrail field mappings. 16086 16110 17155
-
Make the
azure-eventhub
input GA. 15671 17313 -
Add
access_key_id
,secret_access_key
, andsession_token
to the aws module config. 17456
Heartbeat
- Allow a list of status codes for HTTP checks. 15587
Journalbeat
-
Improve parsing of
syslog.pid
in Journalbeat to strip the username when present. 16116
Metricbeat
- Add lambda metricset in aws module. 15260
- Add DynamoDB AWS light module. 15097
- Add IBM MQ light-weight module. 15301
- Add mixer metricset for Istio Metricbeat module. 15696
- Add mesh metricset for Istio Metricbeat module. 15535
- Add pilot metricset for Istio Metricbeat module. 15761
- Add galley metricset for Istio Metricbeat module. 15857
-
Add
key/value
mode for SQL module. 15770 15845 - Add support for Unix socket in Memcached module. 13685 15822
-
Make the
system/cpu
metricset collect normalized CPU metrics by default. 15618 15729 - Add kubernetes storage class support via kube-state-metrics. 16145
-
Add
up
metric to prometheus metrics collected from host. 15948 - Add citadel metricset for Istio Metricbeat module. 15990
- Add support for processors in light modules. 14740 15923
- Add ability to collect AuroraDB metrics in rds metricset. 14142 16004
- Reuse connections in SQL module. 16001
-
Improve the
logstash
module (whenxpack.enabled
is set totrue
) to use the overridecluster_uuid
returned by Logstash APIs. 15772 15795 - Add region parameter in googlecloud module. 15780 16203
-
Add
database_account
azure metricset. 15758 - Add support for Dropwizard metrics 4.1. 16332
- Add support for NATS 2.1. 16317
- Add azure container metricset in order to monitor containers. 15751 16421
-
Improve the
haproxy
module to support metrics exposed via HTTPS. 14579 16333 - Add filtering option for prometheus collector. 16420
-
Add metricsets based on Ceph Manager Daemon to the
ceph
module. 7723 16254 - Add Load Balancing metricset to GCP. 15559
-
Release
statsd
module as GA. 16447 14280 - Add collecting tags and tags_filter for rds metricset in aws module. 16605 16358
- Add OpenMetrics module. 16596
-
Add
redisenterprise
module. 16482 15269 -
Add
cloudfoundry
module to send events from Cloud Foundry. 16671 - Add system/users metricset as beta. 16569
- Align fields to ECS and add more tests for the azure module. 16024 16754
- Add additional cgroup fields to docker/diskio. 16638
- Add overview dashboard for googlecloud compute metricset. 16534 16819
- Add Prometheus remote write endpoint. 16609
- Release STAN module as GA. 16980
- Add query metricset for prometheus module. 17104
- Release ActiveMQ module as GA. 17047 17049
- Add support for CouchDB v2. 16352 16455
- Add dashboards for the azure container metricsets. 17194
-
Separate the
vpc
metricset into three smaller metricsets:vpn
,transitgateway
, andnatgateway
. 16892 - Use Elasticsearch histogram type to store Prometheus histograms. 17061
- Allow to rate Prometheus counters when scraping them. 17061
- Release the Oracle module as GA. 14279 16833
- Add Storage metricsets to GCP module. 15598
- Release the vsphere module as GA. 15798 17119
- Add PubSub metricset to Google Cloud Platform module. 15536
-
Add dashboard for
redisenterprise
module. 16752 - Add dashboard for VSphere host cluster and virtual machine. 14135
- Add test for documented fields check for metricsets without a http input. 17315 17334
- Release the azure module as GA. 17319
- In the kubernetes manifests, mount the data directory from the host, so data persist between executions in the same node. 17429
Packetbeat
Winlogbeat