Logstash 7.0.0 Release Notes
editLogstash 7.0.0 Release Notes
editThe list combines release notes from the 7.0.0-alpha1, -alpha2, -beta1, -rc1 and -rc2 releases.
Logstash core
edit- BUGFIX: Correctly count total queued items across multiple pipelines #10564
- BUGFIX: Fix issue setting enable_metric ⇒ false #10538
- BUGFIX: Prevent concurrent convergence of pipeline actions #10537
- Monitoring: Change internal document type to push "_doc" instead of "doc" #10533
- BUGFIX: Allow explicitly-specified Java codecs #10520
- Central management typeless API #10421
- Improve docs about using Filebeat modules with Logstash #10438
- Bump JRuby to 9.2.6.0 #10425
- BUGFIX: Remove exclusive lock for Ruby pipeline initialization #10462
- Update Java dependencies #10340
- Remove pipeline output workers setting #10358
- Cleanup Ruby gems dependencies #10171
- Ensure compatibility of module data with ES and Kibana 7.0 #10356
- Rename x-pack monitoring and management config option .url and .ca to .hosts and .certificate_authority #10380
- BUGFIX: building of deb and rpm artifacts #10396
- Make Java execution the default #8649
- Field-reference parsing is now strict by default #9543
- Improvements to core Javaification
- BUGFIX: Support for Byte, Short and Date type conversions as seen in the rabbitmq input plugin #9984
Plugins
editHere are the plugin changes.
Codec plugins
edit-
logstash-codec-cef
-
Removed obsolete
sev
anddeprecated_v1_fields
fields - Fixed minor doc inconsistencies (added reverse_mapping to options table, moved it to alpha order in option descriptions, fixed typo) #60
-
Removed obsolete
-
logstash-codec-es_bulk
- Add documentation about use with http input
-
logstash-codec-netflow
- Fix sub-second timestamp math
- BREAKING: Added support for RFC6759 decoding of application_id. This is a breaking change to the way application_id is decoded. The format changes from e.g. 0:40567 to 0..12356..40567
- Fixed IPFIX options template parsing for Juniper MX240 JunOS 15.1
- Fixed incorrect parsing of zero-filled Netflow 9 packets from Palo Alto
- Added support for Netflow v9 devices with VarString fields (H3C Netstream)
- Reduced complexity of creating, persisting, loading an retrieving template caches
- Fixed issue where TTL in template registry was not being respected
- Added Cisco ACI to list of known working Netflow v9 exporters
- Added support for IXIA Packet Broker IPFIX
- Fixed issue with Procera float fields
Filter plugins
edit-
logstash-filter-aggregate
-
new feature: add ability to dynamically define a custom
timeout
orinactivity_timeout
incode
block (fix issues #91 and #92) -
new feature: add meta informations available in
code
block throughmap_meta
variable - new feature: add Logstash metrics, specific to aggregate plugin: aggregate_maps, pushed_events, task_timeouts, code_errors, timeout_code_errors
-
new feature: validate at startup that
map_action
option equals to create, update or create_or_update
-
new feature: add ability to dynamically define a custom
-
logstash-filter-clone
- Make clones a required option
- Added a warning when clones is empty since that results in a no-op #14
-
logstash-filter-de_dot
- fix failure of fieldnames with boolean value "false"
-
logstash-filter-dns
-
logstash-filter-elasticsearch
- Add support for extracting hits total from Elasticsearch 7.x responses
- Added connection check during register to avoid failures during processing
- Changed Elasticsearch Client transport to use Manticore
- Changed amount of logging details during connection failure
-
logstash-filter-fingerprint
- Fixed concurrent SHA fingerprinting by making the instances thread local
-
logstash-filter-geoip
- Removed obsolete lru_cache_size field
-
NEW: logstash-filter-http
- Beta version of HTTP filter plugin based on @lucashenning’s REST filter.
- Fixed minor documentation issues #9
- Minor documentation fixes
-
logstash-filter-jdbc_static
- Added info to documentation to emphasize significance of table order 36
-
logstash-filter-jdbc_streaming
- Swap out mysql for postgresql for testing #11
-
logstash-filter-json
- Updated documentation with some clarifications and fixes
-
logstash-filter-kv
-
Added a timeout enforcer which prevents inputs that are pathological against the generated parser from blocking
the pipeline. By default, timeout is a generous 30s, but can be configured or disabled entirely with the new
timeout_millis
andtag_on_timeout
directives #79 -
Made error-handling configurable with
tag_on_failure
directive.
-
Added a timeout enforcer which prevents inputs that are pathological against the generated parser from blocking
the pipeline. By default, timeout is a generous 30s, but can be configured or disabled entirely with the new
-
NEW: logstash-filter-memcached
- Updated to 1.0.0
-
The plugin common options (e.g.,
add_field
,add_tag
, etc.) are now correctly only invoked when the plugin successfully gets one or more values from, or sets one or more values to memcached (#4) - Fix links to argument types in documentation (#3)
-
logstash-filter-metrics
- Fixed two minor typos in documentation
-
logstash-filter-mutate
- Added ability to directly convert from integer and float to boolean #127
-
logstash-filter-split
- Fixed numeric values, optimized @target verification, cleanups and specs #36
-
logstash-filter-xml
Input plugins
edit-
logstash-input-azure_event_hubs
- Updated Azure event hub library dependencies #27
-
logstash-input-beats
- Removed obsolete setting congestion_threshold and target_field_for_codec
-
Changed default value of
add_hostname
to false - Loosen jar-dependencies manager gem dependency to allow plugin to work with JRubies that include a later version
- Updated jar dependencies to reflect newer releases
-
logstash-input-elasticsearch
-
Added managed slice scrolling with
slices
option
-
Added managed slice scrolling with
-
logstash-input-file
- Fixed problem in Windows where some paths would fail to return an identifier ("inode"). Make path into a C style String before encoding to UTF-16LE. #232
- Fixed issue where logs were being spammed with needless error messages #224
- Fixed problem in tail and read modes where the read loop could get stuck if an IO error occurs in the loop. The file appears to be being read but it is not, suspected with file truncation schemes. #205
- Fixed problem in rotation handling where the target file being rotated was subjected to the start_position setting when it must always start from the beginning. #214
-
logstash-input-gelf
- Fixed shutdown handling, robustness in socket closing and restarting, json parsing, code DRYing and cleanups #62
-
logstash-input-http
- Added configurable response code option #103
- Added explanation about operation order of codec and additional_codecs #104
- Added configurable response code option #103
- Added explanation about operation order of codec and additional_codecs #104
- Loosen jar-dependencies manager gem dependency to allow plugin to work with JRubies that include a later version.
- Changed jar dependencies to reflect newer versions
-
logstash-input-http_poller
-
logstash-input-kafka
-
logstash-input-snmp
-
logstash-input-sqs
- Added support for multiple events inside same message from SQS #48
-
logstash-input-tcp
-
Removed obsolete
data_timeout
andssl_cacert
options - Fixed race condition where data would be accepted before queue was configured
- Support multiple certificates per file #140
-
Removed obsolete
Output plugins
edit-
logstash-output-elasticsearch
- Remove support for parent child (still support join data type) since we don’t support multiple document types any more
-
Removed obsolete
flush_size
andidle_flush_time
- Added auto setting for ILM with default of auto #838
- Fixed sniffing support for 7.x #827
- Fixed issue with escaping index names which was causing writing aliases for ILM to fail #831
- Adds support for Index Lifecycle Management for Elasticsearch 6.6.0 and above, running with at least a Basic License(Beta) #805
- Fixed support for Elasticsearch 7.x #812
- Tweaked logging statements to reduce verbosity
- Fixed numerous issues relating to builds on Travis #799
-
logstash-output-file
- Removed JRuby check when using FIFOs #75
-
logstash-output-http
- Relax dependency on http_client mixin since current major works on both
-
Fixed handling of empty
retryable_codes
#99
-
logstash-output-kafka
-
logstash-output-pagerduty
- Update development dependency webmock to latest version to prevent conflicts in logstash core’s dependency matrix.
-
logstash-output-redis
-
Removed obsolete fields
queue
andname
- Changed major version of redis library dependency to 4.x
-
Removed obsolete fields
-
logstash-output-s3
-
logstash-output-sqs
-
Removed obsolete fields
batch
andbatch_timeout
- Removed workaround to JRuby bug #3645
-
Removed obsolete fields
-
logstash-output-tcp
-
Removed obsolete field
message_format
- Removed requirement to have a certificate/key pair when enabling ssl
-
Removed obsolete field
-
logstash-mixin-http_client
- Removed obsolete ssl_certificate_verify option