Reroute processor
editReroute processor
editThis functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
The reroute
processor allows to route a document to another target index or data stream.
It has two main modes:
When setting the destination
option, the target is explicitly specified and the dataset
and namespace
options can’t be set.
When the destination
option is not set, this processor is in a data stream mode.
Note that in this mode, the reroute
processor can only be used on data streams that follow the data stream naming scheme.
Trying to use this processor on a data stream with a non-compliant name will raise an exception.
The name of a data stream consists of three parts: <type>-<dataset>-<namespace>
.
See the data stream naming scheme documentation for more details.
This processor can use both static values or reference fields from the document to determine the dataset
and namespace
components of the new target.
See Table 38, “Reroute options” for more details.
It’s not possible to change the type
of the data stream with the reroute
processor.
After a reroute
processor has been executed, all the other processors of the current pipeline are skipped, including the final pipeline.
If the current pipeline is executed in the context of a Pipeline, the calling pipeline will be skipped, too.
This means that at most one reroute
processor is ever executed within a pipeline,
allowing to define mutually exclusive routing conditions,
similar to a if, else-if, else-if, … condition.
The reroute processor ensures that the data_stream.<type|dataset|namespace>
fields are set according to the new target.
If the document contains a event.dataset
value, it will be updated to reflect the same value as data_stream.dataset
.
Note that the client needs to have permissions to the final target. Otherwise, the document will be rejected with a security exception which looks like this:
{"type":"security_exception","reason":"action [indices:admin/auto_create] is unauthorized for API key id [8-dt9H8BqGblnY2uSI--] of user [elastic/fleet-server] on indices [logs-foo-default], this action is granted by the index privileges [auto_configure,create_index,manage,all]"}
Table 38. Reroute options
Name | Required | Default | Description |
---|---|---|---|
|
no |
- |
A static value for the target. Can’t be set when the |
|
no |
|
Field references or a static value for the dataset part of the data stream name. In addition to the criteria for index names, cannot contain Supports field references with a mustache-like syntax (denoted as |
|
no |
|
Field references or a static value for the namespace part of the data stream name. See the criteria for index names for allowed characters. Must be no longer than 100 characters. Supports field references with a mustache-like syntax (denoted as |
|
no |
- |
Description of the processor. Useful for describing the purpose of the processor or its configuration. |
|
no |
- |
Conditionally execute the processor. See Conditionally run a processor. |
|
no |
|
Ignore failures for the processor. See Handling pipeline failures. |
|
no |
- |
Handle failures for the processor. See Handling pipeline failures. |
|
no |
- |
Identifier for the processor. Useful for debugging and metrics. |
The if
option can be used to define the condition in which the document should be rerouted to a new target.
{ "reroute": { "tag": "nginx", "if" : "ctx?.log?.file?.path?.contains('nginx')", "dataset": "nginx" } }
The dataset and namespace options can contain either a single value or a list of values that are used as a fallback.
If a field reference evaluates to null
, is not present in the document, the next value or field reference is used.
If a field reference evaluates to a non-String
value, the processor fails.
In the following example, the processor would first try to resolve the value for the service.name
field to determine the value for dataset
.
If that field resolves to null
, is missing, or is a non-string value, it would try the next element in the list.
In this case, this is the static value "generic
".
The namespace
option is configured with just a single static value.
{ "reroute": { "dataset": [ "{{service.name}}", "generic" ], "namespace": "default" } }