Beats version 7.6.0

Affecting all Beats

Remove version information from default ILM policy for improved upgrade experience on custom policies. 14745
Running setup cmd respects setup.ilm.overwrite setting for improved support of custom policies. 14741
Cleanup the x-pack licenser code to use the new license endpoint and the new format. Replaces the url /_xpack/license with /_license. 15091
The document id fields has been renamed from @metadata.id to @metadata._id 15859
Two Beat instances with the same data path cannot be run concurrently. 14069

Filebeat

Journalbeat

Metricbeat

Packetbeat

TLS: Fields have been changed to adapt to ECS. 15497
TLS: The behavior of send_certificates and include_raw_certificates options has changed. 15497

Affecting all Beats

Fix spooling to disk blocking infinitely if the lock file can not be acquired. 15338
Fix metricbeat test output with an ipv6 ES host in the output.hosts. 15368
Fix convert processor conversion of string to integer with leading zeros. 15513 15557
Fix existing agent.*, ecs.version, and host.name fields getting overwritten by Beats if they are already present in the original event. 14407
Fix issue where TLS settings would be ignored when a forward proxy was in use. $15516
Beats no longer attempts to load dashboards if they are unavailable. 15802

Auditbeat

Filebeat

Fix a problem in Filebeat input httpjson where interval is not used as time.Duration. 14728
Fix SSL config in input.yml for Filebeat httpjson input in the MISP module. 14767
Check content-type when creating new reader in s3 input. 15252 15225
Fix session reset detection and a crash in Netflow input. 14904
Handle errors in handleS3Objects function and add more debug messages for s3 input. 15545
netflow: Allow for options templates without scope fields. 15449
netflow: Fix bytes/packets counters on some devices (NSEL and Netstream). 15449
netflow: Fix compatibility with some Cisco devices by changing the field class_id from short to long. 15449
Fix dashboard for Cisco ASA Firewall. 15420 15553
Fix s3 input hanging with GetObjectRequest API call by adding context_timeout config. 15502 15590
Add shared_credential_file to cloudtrail config. 15652 15656
Fix typos in zeek notice fileset config file. 15764 15765
Prevent Elasticsearch from spewing log warnings about redundant wildcards when setting up ingest pipelines for the elasticsearch module. 15840 15900
Improve elasticsearch/audit fileset to handle timestamps correctly. 15942

Heartbeat

Fix excessive memory usage introduced in 7.5 due to over-allocating memory for HTTP checks. 15639

Metricbeat

Fix regular expression to detect instance name in perfmon metricset. 14273 14666
Fix docker.container.size fields values 14979 15224
Make kibana module more resilient to Kibana unavailability. 15258 15270
Fix panic exception with some unicode strings in perfmon metricset. 15264
Make logstash module more resilient to Logstash unavailability. 15276 15306
Add username/password in Metricbeat autodiscover hints 15349
Add dedot for tags in ec2 metricset and cloudwatch metricset. 15843 15844
Use RFC3339 format for timestamps collected using the SQL module. 15847
Add dedot for cloudwatch metric name. 15916 15917
Fixed issue logstash-xpack module suddenly ceasing to monitor Logstash. 15974 16044

Affecting all Beats

Add a friendly log message when a request to docker has exceeded the deadline. 15336
GA the script processor. 14325
Add fingerprint processor. 11173 14205
Add support for API keys in Elasticsearch outputs. 14324
Add consumer_lag in Kafka consumergroup metricset 14822
Make use of consumer_lag in Kafka dashboard 14863
Refactor kubernetes autodiscover to enable different resource based discovery 14738
Add add_id processor. 14524
Enable TLS 1.3 in all beats. 12973
Spooling to disk creates a lockfile on each platform. 15338
Enable DEP (Data Execution Protection) for Windows packages. 15149
Users can now specify monitoring.cloud.* to override monitoring.elasticsearch.* settings. 14399 15254
Add support to kubernetes autodiscovery to add additional metadata from other source to events. 14875
Update to ECS 1.4.0. 14844
Add document_id setting to decode_json_fields processor. 15859

Filebeat

Add new fileset googlecloud/audit for ingesting Google Cloud Audit logs. 15200
Add dashboards to the CEF module (ported from the Logstash ArcSight module). 14342
Add expand_event_list_from_field support in s3 input for reading json format AWS logs. 15357 15370
Add azure-eventhub input which will use the azure eventhub go sdk. 14092 14882
Expose more metrics of harvesters (e.g. read_offset, start_time). 13395
Include log.source.address for unparseable syslog messages. 13268 15453
Release aws elb fileset as GA. 15426 15380
Integrate the azure-eventhub with filebeat azure module (replace the kafka input). 15480
Release aws s3access fileset to GA. 15431 15430
Add cloudtrail fileset to AWS module. 14657 15227
New fileset googlecloud/firewall for ingesting Google Cloud Firewall logs. 14553
google-pubsub input: ACK pub/sub message when acknowledged by publisher. 13346 14715
Remove Beta label from google-pubsub input. 13346 14715
Add dashboard for AWS ELB fileset. 15804
Set event.outcome field based on googlecloud audit log output. 15731
Add dashboard for AWS vpcflow fileset. 16007

Heartbeat

Metricbeat

Expand data for the system/memory metricset 15492
Add azure storage metricset in order to retrieve metric values for storage accounts. 14548 15342
Add cost warnings for the azure module. 15356
Release elb module as GA. 15485
Add a system/network_summary metricset 15196
Allow Metricbeat’s beat module to read monitoring information over a named pipe or unix domain socket. 14558
Enable script processor. 14711
Add STAN dashboard 15654

Functionbeat