Beats version 7.6.0

edit

View commits

Breaking changes

edit

Affecting all Beats

  • Remove version information from default ILM policy for improved upgrade experience on custom policies. 14745
  • Running setup cmd respects setup.ilm.overwrite setting for improved support of custom policies. 14741
  • Cleanup the x-pack licenser code to use the new license endpoint and the new format. Replaces the url /_xpack/license with /_license. 15091
  • The document id fields has been renamed from @metadata.id to @metadata._id 15859
  • Two Beat instances with the same data path cannot be run concurrently. 14069

Filebeat

  • CEF extensions are now mapped to the data types defined in the CEF guide. 14342

Journalbeat

  • Remove broken dashboard. 15288

Metricbeat

  • Update cloudwatch metricset mapping for both metrics and dimensions. 15245

Packetbeat

  • TLS: Fields have been changed to adapt to ECS. 15497
  • TLS: The behavior of send_certificates and include_raw_certificates options has changed. 15497

Bugfixes

edit

Affecting all Beats

  • Fix spooling to disk blocking infinitely if the lock file can not be acquired. 15338
  • Fix metricbeat test output with an ipv6 ES host in the output.hosts. 15368
  • Fix convert processor conversion of string to integer with leading zeros. 15513 15557
  • Fix existing agent.*, ecs.version, and host.name fields getting overwritten by Beats if they are already present in the original event. 14407
  • Fix issue where TLS settings would be ignored when a forward proxy was in use. $15516
  • Beats no longer attempts to load dashboards if they are unavailable. 15802

Auditbeat

  • system/socket: Fix compatibility issue with kernel 5.x. 15771

Filebeat

  • Fix a problem in Filebeat input httpjson where interval is not used as time.Duration. 14728
  • Fix SSL config in input.yml for Filebeat httpjson input in the MISP module. 14767
  • Check content-type when creating new reader in s3 input. 15252 15225
  • Fix session reset detection and a crash in Netflow input. 14904
  • Handle errors in handleS3Objects function and add more debug messages for s3 input. 15545
  • netflow: Allow for options templates without scope fields. 15449
  • netflow: Fix bytes/packets counters on some devices (NSEL and Netstream). 15449
  • netflow: Fix compatibility with some Cisco devices by changing the field class_id from short to long. 15449
  • Fix dashboard for Cisco ASA Firewall. 15420 15553
  • Fix s3 input hanging with GetObjectRequest API call by adding context_timeout config. 15502 15590
  • Add shared_credential_file to cloudtrail config. 15652 15656
  • Fix typos in zeek notice fileset config file. 15764 15765
  • Prevent Elasticsearch from spewing log warnings about redundant wildcards when setting up ingest pipelines for the elasticsearch module. 15840 15900
  • Improve elasticsearch/audit fileset to handle timestamps correctly. 15942

Heartbeat

  • Fix excessive memory usage introduced in 7.5 due to over-allocating memory for HTTP checks. 15639

Metricbeat

  • Fix regular expression to detect instance name in perfmon metricset. 14273 14666
  • Fix docker.container.size fields values 14979 15224
  • Make kibana module more resilient to Kibana unavailability. 15258 15270
  • Fix panic exception with some unicode strings in perfmon metricset. 15264
  • Make logstash module more resilient to Logstash unavailability. 15276 15306
  • Add username/password in Metricbeat autodiscover hints 15349
  • Add dedot for tags in ec2 metricset and cloudwatch metricset. 15843 15844
  • Use RFC3339 format for timestamps collected using the SQL module. 15847
  • Add dedot for cloudwatch metric name. 15916 15917
  • Fixed issue logstash-xpack module suddenly ceasing to monitor Logstash. 15974 16044

Added

edit

Affecting all Beats

  • Add a friendly log message when a request to docker has exceeded the deadline. 15336
  • GA the script processor. 14325
  • Add fingerprint processor. 11173 14205
  • Add support for API keys in Elasticsearch outputs. 14324
  • Add consumer_lag in Kafka consumergroup metricset 14822
  • Make use of consumer_lag in Kafka dashboard 14863
  • Refactor kubernetes autodiscover to enable different resource based discovery 14738
  • Add add_id processor. 14524
  • Enable TLS 1.3 in all beats. 12973
  • Spooling to disk creates a lockfile on each platform. 15338
  • Enable DEP (Data Execution Protection) for Windows packages. 15149
  • Users can now specify monitoring.cloud.* to override monitoring.elasticsearch.* settings. 14399 15254
  • Add support to kubernetes autodiscovery to add additional metadata from other source to events. 14875
  • Update to ECS 1.4.0. 14844
  • Add document_id setting to decode_json_fields processor. 15859

Filebeat

  • Add new fileset googlecloud/audit for ingesting Google Cloud Audit logs. 15200
  • Add dashboards to the CEF module (ported from the Logstash ArcSight module). 14342
  • Add expand_event_list_from_field support in s3 input for reading json format AWS logs. 15357 15370
  • Add azure-eventhub input which will use the azure eventhub go sdk. 14092 14882
  • Expose more metrics of harvesters (e.g. read_offset, start_time). 13395
  • Include log.source.address for unparseable syslog messages. 13268 15453
  • Release aws elb fileset as GA. 15426 15380
  • Integrate the azure-eventhub with filebeat azure module (replace the kafka input). 15480
  • Release aws s3access fileset to GA. 15431 15430
  • Add cloudtrail fileset to AWS module. 14657 15227
  • New fileset googlecloud/firewall for ingesting Google Cloud Firewall logs. 14553
  • google-pubsub input: ACK pub/sub message when acknowledged by publisher. 13346 14715
  • Remove Beta label from google-pubsub input. 13346 14715
  • Add dashboard for AWS ELB fileset. 15804
  • Set event.outcome field based on googlecloud audit log output. 15731
  • Add dashboard for AWS vpcflow fileset. 16007

Heartbeat

Metricbeat

  • Expand data for the system/memory metricset 15492
  • Add azure storage metricset in order to retrieve metric values for storage accounts. 14548 15342
  • Add cost warnings for the azure module. 15356
  • Release elb module as GA. 15485
  • Add a system/network_summary metricset 15196
  • Allow Metricbeat’s beat module to read monitoring information over a named pipe or unix domain socket. 14558
  • Enable script processor. 14711
  • Add STAN dashboard 15654

Functionbeat

  • Add monitoring info about triggered functions. 14876
  • Add Google Cloud Platform support. 13598