Simulate pipeline API
editSimulate pipeline API
editExecutes an ingest pipeline against a set of provided documents.
POST /_ingest/pipeline/my-pipeline-id/_simulate { "docs": [ { "_index": "index", "_id": "id", "_source": { "foo": "bar" } }, { "_index": "index", "_id": "id", "_source": { "foo": "rab" } } ] }
Request
editPOST /_ingest/pipeline/<pipeline>/_simulate
GET /_ingest/pipeline/<pipeline>/_simulate
POST /_ingest/pipeline/_simulate
GET /_ingest/pipeline/_simulate
Prerequisites
edit-
If the Elasticsearch security features are enabled, you must have the
read_pipeline
,manage_pipeline
,manage_ingest_pipelines
, ormanage
cluster privilege to use this API.
Description
editThe simulate pipeline API executes a specific pipeline against a set of documents provided in the body of the request.
You can either specify an existing pipeline to execute against the provided documents or supply a pipeline definition in the body of the request.
Path parameters
edit-
<pipeline>
-
(Required*, string)
Pipeline to test. If you don’t specify a
pipeline
in the request body, this parameter is required.
Query parameters
edit-
verbose
-
(Optional, Boolean)
If
true
, the response includes output data for each processor in the executed pipeline.
Request body
edit-
pipeline
-
(Required*, object) Pipeline to test. If you don’t specify the
<pipeline>
request path parameter, this parameter is required. If you specify both this and the request path parameter, the API only uses the request path parameter.Properties of
pipeline
-
description
- (Optional, string) Description of the ingest pipeline.
-
on_failure
-
(Optional, array of processor objects) Processors to run immediately after a processor failure.
Each processor supports a processor-level
on_failure
value. If a processor without anon_failure
value fails, Elasticsearch uses this pipeline-level parameter as a fallback. The processors in this parameter run sequentially in the order specified. Elasticsearch will not attempt to run the pipeline’s remaining processors. -
processors
- (Required, array of processor objects) Processors used to perform transformations on documents before indexing. Processors run sequentially in the order specified.
-
version
-
(Optional, integer) Version number used by external systems to track ingest pipelines.
See the
if_version
parameter above for how the version attribute is used. -
_meta
- (Optional, object) Optional metadata about the ingest pipeline. May have any contents. This map is not automatically generated by Elasticsearch.
-
deprecated
- (Optional, boolean) Marks this ingest pipeline as deprecated. When a deprecated ingest pipeline is referenced as the default or final pipeline when creating or updating a non-deprecated index template, Elasticsearch will emit a deprecation warning.
-
-
docs
-
(Required, array of objects) Sample documents to test in the pipeline.
Properties of
docs
objects-
_id
-
(Optional, string)
Unique identifier for the document. This ID must be unique within the
_index
. -
_index
- (Optional, string) Name of the index containing the document.
-
_routing
-
(Optional, string)
Value used to send the document to a specific primary shard. See the
_routing
field. -
_source
- (Required, object) JSON body for the document.
-
Examples
editSpecify a pipeline as a path parameter
editPOST /_ingest/pipeline/my-pipeline-id/_simulate { "docs": [ { "_index": "index", "_id": "id", "_source": { "foo": "bar" } }, { "_index": "index", "_id": "id", "_source": { "foo": "rab" } } ] }
The API returns the following response:
{ "docs": [ { "doc": { "_id": "id", "_index": "index", "_version": "-3", "_source": { "field2": "_value", "foo": "bar" }, "_ingest": { "timestamp": "2017-05-04T22:30:03.187Z" } } }, { "doc": { "_id": "id", "_index": "index", "_version": "-3", "_source": { "field2": "_value", "foo": "rab" }, "_ingest": { "timestamp": "2017-05-04T22:30:03.188Z" } } } ] }
Specify a pipeline in the request body
editresponse = client.ingest.simulate( body: { pipeline: { description: '_description', processors: [ { set: { field: 'field2', value: '_value' } } ] }, docs: [ { _index: 'index', _id: 'id', _source: { foo: 'bar' } }, { _index: 'index', _id: 'id', _source: { foo: 'rab' } } ] } ) puts response
POST /_ingest/pipeline/_simulate { "pipeline" : { "description": "_description", "processors": [ { "set" : { "field" : "field2", "value" : "_value" } } ] }, "docs": [ { "_index": "index", "_id": "id", "_source": { "foo": "bar" } }, { "_index": "index", "_id": "id", "_source": { "foo": "rab" } } ] }
The API returns the following response:
{ "docs": [ { "doc": { "_id": "id", "_index": "index", "_version": "-3", "_source": { "field2": "_value", "foo": "bar" }, "_ingest": { "timestamp": "2017-05-04T22:30:03.187Z" } } }, { "doc": { "_id": "id", "_index": "index", "_version": "-3", "_source": { "field2": "_value", "foo": "rab" }, "_ingest": { "timestamp": "2017-05-04T22:30:03.188Z" } } } ] }
View verbose results
editYou can use the simulate pipeline API
to see how each processor affects the ingest document
as it passes through the pipeline.
To see the intermediate results
of each processor in the simulate request,
you can add the verbose
parameter to the request.
response = client.ingest.simulate( verbose: true, body: { pipeline: { description: '_description', processors: [ { set: { field: 'field2', value: '_value2' } }, { set: { field: 'field3', value: '_value3' } } ] }, docs: [ { _index: 'index', _id: 'id', _source: { foo: 'bar' } }, { _index: 'index', _id: 'id', _source: { foo: 'rab' } } ] } ) puts response
POST /_ingest/pipeline/_simulate?verbose=true { "pipeline" : { "description": "_description", "processors": [ { "set" : { "field" : "field2", "value" : "_value2" } }, { "set" : { "field" : "field3", "value" : "_value3" } } ] }, "docs": [ { "_index": "index", "_id": "id", "_source": { "foo": "bar" } }, { "_index": "index", "_id": "id", "_source": { "foo": "rab" } } ] }
The API returns the following response:
{ "docs" : [ { "processor_results" : [ { "processor_type" : "set", "status" : "success", "doc" : { "_index" : "index", "_id" : "id", "_version": "-3", "_source" : { "field2" : "_value2", "foo" : "bar" }, "_ingest" : { "pipeline" : "_simulate_pipeline", "timestamp" : "2020-07-30T01:21:24.251836Z" } } }, { "processor_type" : "set", "status" : "success", "doc" : { "_index" : "index", "_id" : "id", "_version": "-3", "_source" : { "field3" : "_value3", "field2" : "_value2", "foo" : "bar" }, "_ingest" : { "pipeline" : "_simulate_pipeline", "timestamp" : "2020-07-30T01:21:24.251836Z" } } } ] }, { "processor_results" : [ { "processor_type" : "set", "status" : "success", "doc" : { "_index" : "index", "_id" : "id", "_version": "-3", "_source" : { "field2" : "_value2", "foo" : "rab" }, "_ingest" : { "pipeline" : "_simulate_pipeline", "timestamp" : "2020-07-30T01:21:24.251863Z" } } }, { "processor_type" : "set", "status" : "success", "doc" : { "_index" : "index", "_id" : "id", "_version": "-3", "_source" : { "field3" : "_value3", "field2" : "_value2", "foo" : "rab" }, "_ingest" : { "pipeline" : "_simulate_pipeline", "timestamp" : "2020-07-30T01:21:24.251863Z" } } } ] } ] }