Processors
editProcessors
editAll processors are defined in the following way within a pipeline definition:
{ "PROCESSOR_NAME" : { ... processor configuration options ... } }
Each processor defines its own configuration parameters, but all processors have
the ability to declare tag
, on_failure
and if
fields. These fields are optional.
A tag
is simply a string identifier of the specific instantiation of a certain
processor in a pipeline. The tag
field does not affect the processor’s behavior,
but is very useful for bookkeeping and tracing errors to specific processors.
The if
field must contain a script that returns a boolean value. If the script evaluates to true
then the processor will be executed for the given document otherwise it will be skipped.
The if
field takes an object with the script fields defined in script-options
and accesses a read only version of the document via the same ctx
variable used by scripts in the
Script.
{ "set": { "if": "ctx.foo == 'someValue'", "field": "found", "value": true } }
See Conditional Execution in Pipelines to learn more about the if
field and conditional execution.
See Handling Failures in Pipelines to learn more about the on_failure
field and error handling in pipelines.
The node info API will provide a per node list of what processors are available.
Custom processors must be installed on all nodes. The put pipeline API will fail if a processor specified in a pipeline
doesn’t exist on all nodes. If you rely on custom processor plugins make sure to mark these plugins as mandatory by adding
plugin.mandatory
setting to the config/elasticsearch.yml
file, for example:
plugin.mandatory: ingest-attachment
A node will not start if this plugin is not available.
The node stats API can be used to fetch ingest usage statistics, globally and on a per pipeline basis. Useful to find out which pipelines are used the most or spent the most time on preprocessing.
Ingest Processor Plugins
editAdditional ingest processors can be implemented and installed as Elasticsearch plugins. See Ingest plugins for information about the available ingest plugins.