- Logstash Reference: other versions:
- Logstash Introduction
- Getting Started with Logstash
- How Logstash Works
- Setting Up and Running Logstash
- Logstash Directory Layout
- Logstash Configuration Files
- logstash.yml
- Secrets keystore for secure settings
- Running Logstash from the Command Line
- Running Logstash as a Service on Debian or RPM
- Running Logstash on Docker
- Configuring Logstash for Docker
- Running Logstash on Kubernetes
- Running Logstash on Windows
- Logging
- Shutting Down Logstash
- Upgrading Logstash
- Creating a Logstash pipeline
- Secure your connection
- Advanced Logstash Configurations
- Logstash-to-Logstash communication
- Managing Logstash
- Using Logstash with Elastic Integrations
- Working with Logstash Modules
- Working with Filebeat Modules
- Working with Winlogbeat Modules
- Queues and data resiliency
- Transforming Data
- Deploying and Scaling Logstash
- Managing GeoIP Databases
- Performance tuning
- Monitoring Logstash with Elastic Agent
- Monitoring Logstash (legacy)
- Monitoring Logstash with APIs
- Working with plugins
- Integration plugins
- Input plugins
- azure_event_hubs
- beats
- cloudwatch
- couchdb_changes
- dead_letter_queue
- elastic_agent
- elastic_serverless_forwarder
- elasticsearch
- exec
- file
- ganglia
- gelf
- generator
- github
- google_cloud_storage
- google_pubsub
- graphite
- heartbeat
- http
- http_poller
- imap
- irc
- java_generator
- java_stdin
- jdbc
- jms
- jmx
- kafka
- kinesis
- logstash
- log4j
- lumberjack
- meetup
- pipe
- puppet_facter
- rabbitmq
- redis
- relp
- rss
- s3
- s3-sns-sqs
- salesforce
- snmp
- snmptrap
- sqlite
- sqs
- stdin
- stomp
- syslog
- tcp
- udp
- unix
- varnishlog
- websocket
- wmi
- xmpp
- Output plugins
- boundary
- circonus
- cloudwatch
- csv
- datadog
- datadog_metrics
- dynatrace
- elastic_app_search
- elastic_workplace_search
- elasticsearch
- exec
- file
- ganglia
- gelf
- google_bigquery
- google_cloud_storage
- google_pubsub
- graphite
- graphtastic
- http
- influxdb
- irc
- java_stdout
- juggernaut
- kafka
- librato
- logstash
- loggly
- lumberjack
- metriccatcher
- mongodb
- nagios
- nagios_nsca
- opentsdb
- pagerduty
- pipe
- rabbitmq
- redis
- redmine
- riak
- riemann
- s3
- sink
- sns
- solr_http
- sqs
- statsd
- stdout
- stomp
- syslog
- tcp
- timber
- udp
- webhdfs
- websocket
- xmpp
- zabbix
- Filter plugins
- age
- aggregate
- alter
- bytes
- cidr
- cipher
- clone
- csv
- date
- de_dot
- dissect
- dns
- drop
- elapsed
- elastic_integration
- elasticsearch
- environment
- extractnumbers
- fingerprint
- geoip
- grok
- http
- i18n
- java_uuid
- jdbc_static
- jdbc_streaming
- json
- json_encode
- kv
- memcached
- metricize
- metrics
- mutate
- prune
- range
- ruby
- sleep
- split
- syslog_pri
- threats_classifier
- throttle
- tld
- translate
- truncate
- urldecode
- useragent
- uuid
- wurfl_device_detection
- xml
- Codec plugins
- Tips and best practices
- Troubleshooting
- Contributing to Logstash
- How to write a Logstash input plugin
- How to write a Logstash codec plugin
- How to write a Logstash filter plugin
- How to write a Logstash output plugin
- Logstash Plugins Community Maintainer Guide
- Document your plugin
- Publish your plugin to RubyGems.org
- List your plugin
- Contributing a patch to a Logstash plugin
- Extending Logstash core
- Contributing a Java Plugin
- Breaking changes
- Release Notes
- Logstash 8.17.1 Release Notes
- Logstash 8.17.0 Release Notes
- Logstash 8.16.2 Release Notes
- Logstash 8.16.1 Release Notes
- Logstash 8.16.0 Release Notes
- Logstash 8.15.5 Release Notes
- Logstash 8.15.4 Release Notes
- Logstash 8.15.3 Release Notes
- Logstash 8.15.2 Release Notes
- Logstash 8.15.1 Release Notes
- Logstash 8.15.0 Release Notes
- Logstash 8.14.3 Release Notes
- Logstash 8.14.2 Release Notes
- Logstash 8.14.1 Release Notes
- Logstash 8.14.0 Release Notes
- Logstash 8.13.4 Release Notes
- Logstash 8.13.3 Release Notes
- Logstash 8.13.2 Release Notes
- Logstash 8.13.1 Release Notes
- Logstash 8.13.0 Release Notes
- Logstash 8.12.2 Release Notes
- Logstash 8.12.1 Release Notes
- Logstash 8.12.0 Release Notes
- Logstash 8.11.4 Release Notes
- Logstash 8.11.3 Release Notes
- Logstash 8.11.2 Release Notes
- Logstash 8.11.1 Release Notes
- Logstash 8.11.0 Release Notes
- Logstash 8.10.4 Release Notes
- Logstash 8.10.3 Release Notes
- Logstash 8.10.2 Release Notes
- Logstash 8.10.1 Release Notes
- Logstash 8.10.0 Release Notes
- Logstash 8.9.2 Release Notes
- Logstash 8.9.1 Release Notes
- Logstash 8.9.0 Release Notes
- Logstash 8.8.2 Release Notes
- Logstash 8.8.1 Release Notes
- Logstash 8.8.0 Release Notes
- Logstash 8.7.1 Release Notes
- Logstash 8.7.0 Release Notes
- Logstash 8.6.2 Release Notes
- Logstash 8.6.1 Release Notes
- Logstash 8.6.0 Release Notes
- Logstash 8.5.3 Release Notes
- Logstash 8.5.2 Release Notes
- Logstash 8.5.1 Release Notes
- Logstash 8.5.0 Release Notes
- Logstash 8.4.2 Release Notes
- Logstash 8.4.1 Release Notes
- Logstash 8.4.0 Release Notes
- Logstash 8.3.3 Release Notes
- Logstash 8.3.2 Release Notes
- Logstash 8.3.1 Release Notes
- Logstash 8.3.0 Release Notes
- Logstash 8.2.3 Release Notes
- Logstash 8.2.2 Release Notes
- Logstash 8.2.1 Release Notes
- Logstash 8.2.0 Release Notes
- Logstash 8.1.3 Release Notes
- Logstash 8.1.2 Release Notes
- Logstash 8.1.1 Release Notes
- Logstash 8.1.0 Release Notes
- Logstash 8.0.1 Release Notes
- Logstash 8.0.0 Release Notes
- Logstash 8.0.0-rc2 Release Notes
- Logstash 8.0.0-rc1 Release Notes
- Logstash 8.0.0-beta1 Release Notes
- Logstash 8.0.0-alpha2 Release Notes
- Logstash 8.0.0-alpha1 Release Notes
Event API
editEvent API
editThis section is targeted for plugin developers and users of Logstash’s Ruby filter. Below we document recent changes (starting with version 5.0) in the way users have been accessing Logstash’s event based data in custom plugins and in the Ruby filter. Note that Accessing event data and fields data flow in Logstash’s config files — using Field references — is not affected by this change, and will continue to use existing syntax.
Event Object
editEvent is the main object that encapsulates data flow internally in Logstash and provides an API for the plugin developers to interact with the event’s content. Typically, this API is used in plugins and in a Ruby filter to retrieve data and use it for transformations. Event object contains the original data sent to Logstash and any additional fields created during Logstash’s filter stages.
In 5.0, we’ve re-implemented the Event class and its supporting classes in pure Java. Since Event is a critical component in data processing, a rewrite in Java improves performance and provides efficient serialization when storing data on disk. For the most part, this change aims at keeping backward compatibility and is transparent to the users. To this extent we’ve updated and published most of the plugins in Logstash’s ecosystem to adhere to the new API changes. However, if you are maintaining a custom plugin, or have a Ruby filter, this change will affect you. The aim of this guide is to describe the new API and provide examples to migrate to the new changes.
Event API
editPrior to version 5.0, developers could access and manipulate event data by directly using Ruby hash syntax. For
example, event[field] = foo
. While this is powerful, our goal is to abstract the internal implementation details
and provide well-defined getter and setter APIs.
Get API
The getter is a read-only access of field-based data in an Event.
Syntax: event.get(field)
Returns: Value for this field or nil if the field does not exist. Returned values could be a string, numeric or timestamp scalar value.
field
is a structured field sent to Logstash or created after the transformation process. field
can also
be a nested field reference such as [field][bar]
.
Examples:
event.get("foo" ) # => "baz" event.get("[foo]") # => "zab" event.get("[foo][bar]") # => 1 event.get("[foo][bar]") # => 1.0 event.get("[foo][bar]") # => [1, 2, 3] event.get("[foo][bar]") # => {"a" => 1, "b" => 2} event.get("[foo][bar]") # => {"a" => 1, "b" => 2, "c" => [1, 2]}
Accessing @metadata
event.get("[@metadata][foo]") # => "baz"
Set API
This API can be used to mutate data in an Event.
Syntax: event.set(field, value)
Returns: The current Event after the mutation, which can be used for chainable calls.
Examples:
event.set("foo", "baz") event.set("[foo]", "zab") event.set("[foo][bar]", 1) event.set("[foo][bar]", 1.0) event.set("[foo][bar]", [1, 2, 3]) event.set("[foo][bar]", {"a" => 1, "b" => 2}) event.set("[foo][bar]", {"a" => 1, "b" => 2, "c" => [1, 2]}) event.set("[@metadata][foo]", "baz")
Mutating a collection after setting it in the Event has an undefined behaviour and is not allowed.
h = {"a" => 1, "b" => 2, "c" => [1, 2]} event.set("[foo][bar]", h) h["c"] = [3, 4] event.get("[foo][bar][c]") # => undefined Suggested way of mutating collections: h = {"a" => 1, "b" => 2, "c" => [1, 2]} event.set("[foo][bar]", h) h["c"] = [3, 4] event.set("[foo][bar]", h) # Alternatively, event.set("[foo][bar][c]", [3, 4])
Ruby Filter
editThe Ruby Filter can be used to execute any ruby code and manipulate event data using the API described above. For example, using the new API:
filter { ruby { code => 'event.set("lowercase_field", event.get("message").downcase)' } }
This filter will lowercase the message
field, and set it to a new field called lowercase_field
On this page
ElasticON events are back!
Learn about the Elastic Search AI Platform from the experts at our live events.
Register now