Beats version 7.16.0

edit

Beats version 7.16.0

edit

View commits

Breaking changes

edit

Affecting all Beats

Load index templates v2 (composable index templates) by default when talking to ES 7.16 or ES 8.x. Please note that you cannot load templates into Elasticsearch 7.8 or older with this default. To load templates to these ES version, set setup.template.type back to legacy. 28538
Previously, RE2 and thus Golang had a bug where (|a)* matched more characters than (|a)+. To stay consistent with PCRE, the bug was fixed. Configurations that rely on the old, buggy behaviour has to be adjusted. See more about Golang bug: https://github.com/golang/go/issues/46123 27543
Remove Journalbeat. Use journald input of Filebeat instead. 29131

Heartbeat

Change behavior in case of duplicate monitor IDs in configs to be last monitor wins. 29041

Metricbeat

Align fields to Beats naming conventions in GCP module. 27231 27974

Functionbeat

Support for Google Cloud Functions have been removed, as it has been in Beta for a long time and been broken for a few releases. Please use other tools provided by Elastic to fetch data from GCP (e.g. Filebeat).

Bugfixes

edit

Affecting all Beats

Fix discovery of Nomad allocations with multiple events during startup. 28700
Fix the wrong beat name on monitoring and state endpoint 27755
Skip configuration checks in autodiscover for configurations that are already running 29048
Fix decode_json_processor to always respect add_error_key 29107
Fix add_labels flattening of array values. 29211
Skip add_kubernetes_metadata processor when Kubernetes metadata are already present 27689

Auditbeat

Fix handling of root and relative paths 24430 28354
Fix handling of long file names on Windows. 25334 28517
System/socket dataset: Fix uninstallation of return kprobes. 28608 28609
Fix auditbeat tracing struct decoding. 28580

Filebeat

Update indentation for azure filebeat configuration. 26604
Tolerate faults when Windows Event Log session is interrupted 27947 28191
Add support for username in Cisco ASA security negotiation logs 26975
Relax time parsing and capture group and session type in Cisco ASA module 24710 28325
Correctly track bytes read when max_bytes is exceeded. 28317 28352
Fix parsing of apache log levels including numbers. 28717
Upgrade azure-eventhub SDK reference, contains potential checkpoint fixes. 28919
Revert usageDetails api version to 2019-01-01. 28995
Fix in aws-s3 input regarding provider discovery through endpoint 28963
Fix threatintel.misp filters configuration. 27970
Fix opening files on Windows in filestream so open files can be deleted. 29113 29180

Heartbeat

Fix broken seccomp filtering and improve security via setcap and setuid when running as root on linux in containers. 27878
Log browser zip_url download failures as warn instead of as info. 28440
Properly locate base stream in fleet configs. 28455
Stop logging params values. 28774
Remove accidentally included cups library in Docker images. pull
Fix broken monitors with newer versions of image relying on dup3. https://github.com/elastic/beats/pull/28938[pull

Metricbeat

beat module respects basepath config option. 28162
Fix list_docker.go 28374
Fix RDS metadata in Cloudwatch metricset. 29106
Errors should be thrown as errors. Metricsets inside metricbeat will now throw errors as the error log level. 27804

Winlogbeat

Tolerate faults when Windows Event Log session is interrupted 27947 28191
Add ECS 1.9 new users fields 26509
Don’t split hyphenated tokens 28483
Correctly handle AccessMask if it is an integer or list of masks. 29016

Added

edit

Affecting all Beats

Allow non-padded base64 data to be decoded by decode_base64_field 27311, 27021
The Kafka support library Sarama has been updated to 1.29.1. 27717
Kafka is now supported up to version 2.8.0. 27720
Add Huawei Cloud provider to add_cloud_metadata. 27607
Add default seccomp policy for linux arm64. 27955
Add cluster level add_kubernetes_metadata support for centralized enrichment 24621
Update cloud.google.com/go library. 28229
Add additional metadata to the root HTTP endpoint. 28265
Upgrade k8s.io/client-go library. 28228
Update ECS to 1.12.0. 27770
Fields mapped as match_only_text will automatically fallback to a text mapping when using Elasticsearch versions that do not support match_only_text. 27770
Do not load ML jobs to Elasticsearch 8.x from new Beats 7.x releases. 27771
Update kubernetes scheduler and controllermanager endpoints in elastic-agent-standalone-kubernetes.yaml with secure ports 28675
Add default seccomp policy for Linux arm64. 27955
Add http.pprof.enabled option to libbeat to allow http/pprof endpoints on the socket that libbeat creates for metrics. 21965
Enable IMDSv2 support for add_cloud_metadata processor on AWS. 22101 28285

Filebeat

Add timezone config option to the decode_cef processor. 27232 27727
Add timezone config option to the syslog input. 27727
Added support for parsing syslog dates containing a leading 0 (e.g. Sep 01) rather than a space. 27775
Add base64 Encode functionality to httpjson input. 27681
Add join and sprintf functions to httpjson input. 27735
Improve memory usage of line reader of log and filestream input. 27782
Add ignore_empty_value flag to httpjson split processor. 27880
Add support for passing a prefix on S3 bucket list mode for AWS-S3 input 28252 27965
Update Cisco ASA/FTD ingest pipeline grok/dissect patterns for multiple message IDs. 26869 26879
Add write access to url.value from request.transforms in httpjson input. 27937
Add Base64 encoded HMAC and UUID template functions to httpjson input 27873
Release checkpoint module as GA. 27814
Make aws-cloudwatch input GA. 28161
Move processing to ingest node for AWS vpcflow fileset. 28168
Release zoom module as GA. 28106
Add support for secondary object attribute handling in ThreatIntel MISP module 28124
Azure signinlogs - Add support for ManagedIdentitySignInLogs, NonInteractiveUserSignInLogs, and ServicePrincipalSignInLogs. 23653
Add base64Decode and base64DecodeNoPad functions to httpsjon templates. 28385
Add early_limit config option for Rate-Limiting httpjson. Default rate-limiting for Okta will start when remaining is 1. 28513
Add latency config option for aws-cloudwatch input. 28509
Added proxy support to threatintel/malwarebazaar. 28533
Sophos UTM: Support logs containing hostname in Syslog header. 28638
Moving Oracle Filebeat module to GA. 28754
Add support in aws-s3 input for S3 notification from SNS to SQS. 28800
Add support in aws-s3 input for custom script parsing of S3 notifications. 28946
Improve error handling in aws-s3 input for malformed S3 notifications. 28828 28946
filestream and log inputs accept null (\u0000) as line terminator. 28998

Heartbeat

Support JSON expressions / validation of JSON arrays. 28073
Experimental run once mode. 25972
Add keyword multi-field mapping for synthetics.step.name. 28452

Metricbeat

Enable journald input type in Filebeat. 7955 27351
Added a new beta enterprisesearch module for Elastic Enterprise Search 27549
Register additional name for storage metricset in the azure module. 28447
Update reference to gosigar pacakge for filesystem windows fix. 28909
Override Host() on statsd MetricSet 29103
Add Linux pressure metricset 27355
Add User-Agent header to HTTP requests. 18160 27509

Functionbeat

Add support for AWS Kinesis record deaggregation 28241

Winlogbeat

Add support for event language selection from config file 19818

Deprecated

edit

Affecting all Beats

Deprecate setup.template.type. In the future Beats will load data streams instead of regular indices.

Filebeat

Deprecate log input in favour of filestream input. 28623

« Beats version 7.16.1 Beats version 7.15.2 »