Beats version 6.5.0

edit

View commits

Bugfixes

edit

Affecting all Beats

  • Fixed add_host_metadata not initializing correctly on Windows. 7715
  • Fixed missing file unlock in spool file on Windows, so file can be reopened and locked. 7859
  • Fix spool file opening/creation failing due to file locking on Windows. 7859
  • Fix size of maximum mmaped read area in spool file on Windows. 7859
  • Fix potential data loss on OS X in spool file by using fcntl with F_FULLFSYNC. 7859
  • Improve fsync on linux, by assuming the kernel resets error flags of failed writes. 7859
  • Remove unix-like permission checks on Windows, so files can be opened. 7849
  • Replace index patterns in TSVB visualizations. 7929
  • Deregister pipeline loader callback when inputsRunner is stopped. 7893[7893]
  • Add backoff support to x-pack monitoring outputs. 7966
  • Removed execute permissions systemd unit file. 7873
  • Fix a race condition with the add_host_metadata and the event serialization. 8223 8653
  • Enforce that data used by k8s or docker doesn’t use any reference. 8240
  • Switch to different UUID lib due to to non-random generated UUIDs. 8485
  • Fix race condition when publishing monitoring data. 8646
  • Fix bug in loading dashboards from zip file. 8051
  • Fix in-cluster kubernetes configuration on IPv6. 8754
  • The export config subcommand should not display real value for field reference. 8769
  • Start autodiscover consumers before producers. 7926
  • The setup command will not fail if no dashboard is available to import. 8977
  • Fix central management configurations reload when a configuration is removed in Kibana. 9010

Auditbeat

  • Fixed a crash in the file_integrity module under Linux. 7753
  • Fixed the RPM by designating the config file as configuration data in the RPM spec. 8075
  • Fixed a concurrent map write panic in the auditd module. 8158
  • Fixed a data race in the file_integrity module. 8009
  • Fixed a deadlock in the file_integrity module. 8027

Filebeat

  • Fix date format in Mongodb Ingest pipeline. 7974
  • Fixed a docker input error due to the offset update bug in partial log join.https://github.com/elastic/beats/pull/8177[8177]
  • Update CRI format to support partial/full tags. 8265
  • Fix some errors happening when stopping syslog input. 8347
  • Fix RFC3339 timezone and nanoseconds parsing with the syslog input. 8346
  • Mark the TCP and UDP input as GA. 8125
  • Support multiline logs in logstash/log fileset of Filebeat. 8562
  • Support different timestamp format in postgresql module. 9494 9650

Heartbeat

  • Fixed bug where HTTP responses with larger bodies would incorrectly report connection errors. 8660

Metricbeat

  • Fix golang.heap.gc.cpu_fraction type from long to float in Golang module. 7789
  • Fixed the RPM by designating the modules.d config files as configuration data in the RPM spec. 8075
  • Fixed the location of the modules.d dir in Deb and RPM packages. 8104
  • Add docker diskio stats on Windows. 6815 8126
  • Fix incorrect type conversion of average response time in Haproxy dashboards 8404
  • Added io disk read and write times to system module 8473 8508
  • Avoid mapping issues in kubernetes module. 8487
  • Recover metrics for old apache versions removed by mistake on #6450. 7871
  • Fix dropwizard module parsing of metric names. 8365 8385
  • Fix issue that would prevent kafka module to find a proper broker when port is not set 8613
  • Fix range colors in multiple visualizations. 8633 8634
  • Fix incorrect header parsing on http metricbeat module 8564 8585
  • Fixed a panic when the kvm module cannot establish a connection to libvirtd. 7792.

Packetbeat

  • Fixed a seccomp related error where the fcntl64 syscall was not permitted on 32-bit Linux and the sniffer failed to start. 7839
  • Added missing cmdline and client_cmdline fields to index template. 8258

Added

edit

Affecting all Beats

  • Added time-based log rotation. 8349
  • Add backoff on error support to redis output. 7781
  • Allow for cloud-id to specify a custom port. This makes cloud-id work in ECE contexts. 7887
  • Add support to grow or shrink an existing spool file between restarts. 7859
  • Add debug check to logp.Logger 7965
  • Make kubernetes autodiscover ignore events with empty container IDs 7971
  • Implement CheckConfig in RunnerFactory to make autodiscover check configs 7961
  • Add DNS processor with support for performing reverse lookups on IP addresses. 7770
  • Support for Kafka 2.0.0 in kafka output 8399
  • Add setting setup.kibana.space.id to support Kibana Spaces 7942
  • Better tracking of number of open file descriptors. 7986
  • Report number of open file handles on Windows. 8329
  • Added the add_process_metadata processor to enrich events with process information. 6789
  • Add Beats Central Management 8559
  • Report configured queue type. 8091
  • Enable host and cloud metadata processors by default. 8596
  • Autodiscovery no longer requires that the condition field be set. If left unset all configs will be matched. 9029
  • Add DeDot method in add_docker_metadata processor in libbeat. 9350 9505

Filebeat

  • Add tag "truncated" to "log.flags" if incoming line is longer than configured limit. 7991
  • Add haproxy module. 8014
  • Add tag "multiline" to "log.flags" if event consists of multiple lines. 7997
  • Release docker input as GA. 8328
  • Keep unparsed user agent information in user_agent.original. 7832
  • Added default and TCP parsing formats to HAproxy module 8311 8637
  • Add Suricata IDS/IDP/NSM module. 8153 8693
  • Support for Kafka 2.0.0 8853

Heartbeat

  • Heartbeat is marked as GA.
  • Add automatic config file reloading. 8023
  • Added autodiscovery support 8415
  • Added support for extra TLS/x509 metadata. 7944
  • Added stats and state metrics for number of monitors and endpoints started. 8621
  • Add last monitor status to dashboard table. Further break out monitors in dashboard table by monitor.ip. 9022

Journalbeat

  • Add journalbeat. 8703

Metricbeat

  • Add replstatus metricset to MongoDB module 7604
  • Add experimental socket summary metricset to system module 6782
  • Move common kafka fields (broker, topic and partition.id) to the module level to facilitate events correlation 7767
  • Add fields for memory fragmentation, memory allocator stats, copy on write, master-slave status, and active defragmentation to info metricset of Redis module. 7695
  • Increase ignore_above for system.process.cmdline to 2048. 8100
  • Add support to renamed fields planned for redis 5.0. 8167
  • Allow TCP helper to support delimiters and graphite module to accept multiple metrics in a single payload. 8278
  • Added died PID state to process_system metricset on system module 8275
  • Add metrics metricset to MongoDB module. 7611
  • Added ccr metricset to Elasticsearch module. 8335
  • Support for Kafka 2.0.0 8399
  • Added support for query params in configuration 8286 8292
  • Add container image for docker metricsets. 8214 8438
  • Precalculate composed id fields for kafka dashboards. 8504
  • Add support for full status page output for php-fpm module as a separate metricset called process. 8394
  • Add Kafka dashboard. 8457
  • Release Kafka module as GA. 8854

Packetbeat

  • Added DHCP protocol support. 7647

Functionbeat

  • Initial version of Functionbeat. 8678

Deprecated

edit

Heartbeat

  • watch.poll_file is now deprecated and superceded by automatic config file reloading.

Metricbeat

  • Redis info replication.master_offset has been deprecated in favor of replication.master.offset.https://github.com/elastic/beats/pull/7695[7695]
  • Redis info clients fields longest_output_list and biggest_input_buf have been renamed to max_output_buffer and max_input_buffer based on the names they will have in Redis 5.0, both fields will coexist during a time with the same value 8167.
  • Move common kafka fields (broker, topic and partition.id) to the module level 7767.