Filtering and Enhancing the Exported Data

When your use case requires only a subset of the data exported by Metricbeat or you need to add metadata, you can use Metricbeat config options to filter the data, or you can define processors.

Each module accepts a list of filters in its configuration. These filters are applied to the data generated by the module. These filters are applied to the module data prior to the addition of the common beat fields like beat.hostname and type so they can only be used to filter fields from the module.

The following example reduces the exported fields of the Redis module to include only the redis.info.memory fields.

metricbeat.modules:
- module: redis
  metricsets: ["info"]
  period: 1s
  hosts: ["127.0.0.1:6379"]
  enabled: true
  filters:
    - include_fields:
       fields: ['memory']

You can define processors in your configuration to process events before they are sent to the configured output. The libbeat library provides processors for reducing the number of exported fields, and processors for enhancing events with additional metadata. Each processor receives an event, applies a defined action to the event, and returns the event. If you define a list of processors, they are executed in the order they are defined in the configuration file.

event -> processor 1 -> event1 -> processor 2 -> event2 ...

The processors are defined in the Metricbeat configuration file.

For example, the following filters configuration reduces the exported fields by dropping the beat.name and beat.hostname fields under beat, from all documents.

processors:
 - drop_fields:
    fields: ['beat']

See Processors Configuration for more information.