- Packetbeat Reference: other versions:
- Packetbeat overview
- Quick start: installation and configuration
- Set up and run
- Upgrade Packetbeat
- Configure
- Traffic sniffing
- Network flows
- Protocols
- Processes
- General settings
- Project paths
- Output
- Kerberos
- SSL
- Index lifecycle management (ILM)
- Elasticsearch index template
- Kibana endpoint
- Kibana dashboards
- Processors
- Define processors
- add_cloud_metadata
- add_cloudfoundry_metadata
- add_docker_metadata
- add_fields
- add_host_metadata
- add_id
- add_kubernetes_metadata
- add_labels
- add_locale
- add_network_direction
- add_nomad_metadata
- add_observer_metadata
- add_process_metadata
- add_tags
- append
- community_id
- convert
- copy_fields
- decode_base64_field
- decode_duration
- decode_json_fields
- decode_xml
- decode_xml_wineventlog
- decompress_gzip_field
- detect_mime_type
- dissect
- dns
- drop_event
- drop_fields
- extract_array
- fingerprint
- include_fields
- move_fields
- rate_limit
- registered_domain
- rename
- replace
- syslog
- translate_ldap_attribute
- translate_sid
- truncate_fields
- urldecode
- Internal queue
- Logging
- HTTP endpoint
- Instrumentation
- Feature flags
- packetbeat.reference.yml
- How to guides
- Exported fields
- AMQP fields
- Beat fields
- Cassandra fields
- Cloud provider metadata fields
- Common fields
- DHCPv4 fields
- DNS fields
- Docker fields
- ECS fields
- Flow Event fields
- Host fields
- HTTP fields
- ICMP fields
- Jolokia Discovery autodiscover provider fields
- Kubernetes fields
- Memcache fields
- MongoDb fields
- MySQL fields
- NFS fields
- PostgreSQL fields
- Process fields
- Raw fields
- Redis fields
- SIP fields
- Thrift-RPC fields
- Detailed TLS fields
- Transaction Event fields
- Measurements (Transactions) fields
- Monitor
- Secure
- Visualize Packetbeat data in Kibana
- Troubleshoot
- Get help
- Debug
- Understand logged metrics
- Record a trace
- Common problems
- Dashboard in Kibana is breaking up data fields incorrectly
- Packetbeat doesn’t see any packets when using mirror ports
- Packetbeat can’t capture traffic from Windows loopback interface
- Packetbeat is missing long running transactions
- Packetbeat isn’t capturing MySQL performance data
- Packetbeat uses too much bandwidth
- Error loading config file
- Found unexpected or unknown characters
- Logstash connection doesn’t work
- Publishing to Logstash fails with "connection reset by peer" message
- @metadata is missing in Logstash
- Not sure whether to use Logstash or Beats
- SSL client fails to connect to Logstash
- Monitoring UI shows fewer Beats than expected
- Dashboard could not locate the index-pattern
- High RSS memory usage due to MADV settings
- Fields show up as nested JSON in Kibana
- Contribute to Beats
Filter and enhance data with processors
editFilter and enhance data with processors
editYou can define processors in your configuration to process events before they are sent to the configured output. The libbeat library provides processors for:
- reducing the number of exported fields
- enhancing events with additional metadata
- performing additional processing and decoding
Each processor receives an event, applies a defined action to the event, and returns the event. If you define a list of processors, they are executed in the order they are defined in the Packetbeat configuration file.
event -> processor 1 -> event1 -> processor 2 -> event2 ...
It’s recommended to do all drop and renaming of existing fields as the last step in a processor configuration. This is because dropping or renaming fields can remove data necessary for the next processor in the chain, for example dropping the source.ip
field would remove one of the fields necessary for the community_id
processor to function. If it’s necessary to remove, rename or overwrite an existing event field, please make sure it’s done by a corresponding processor (drop_fields
, rename
or add_fields
) placed at the end of the processor list defined in the input configuration.
For example, the following configuration includes a subset of the Packetbeat DNS fields so that only the requests and their response codes are reported:
processors: - include_fields: fields: - client.bytes - server.bytes - client.ip - server.ip - dns.question.name - dns.question.etld_plus_one - dns.response_code
The filtered event would look something like this:
{ "@timestamp": "2019-01-19T03:41:11.798Z", "client": { "bytes": 28, "ip": "10.100.6.82" }, "server": { "bytes": 271, "ip": "10.100.4.1" }, "dns": { "question": { "name": "www.elastic.co", "etld_plus_one": "elastic.co" }, "response_code": "NOERROR" }, "type": "dns" }
If you would like to drop all the successful transactions, you can use the following configuration:
processors: - drop_event: when: equals: http.response.status_code: 200
If you don’t want to export raw data for the successful transactions:
processors: - drop_fields: when: equals: http.response.status_code: 200 fields: ["request", "response"]