Using the Attachment Processor with arrays
editUsing the Attachment Processor with arrays
editTo use the attachment processor within an array of attachments the foreach processor is required. This enables the attachment processor to be run on the individual elements of the array.
For example, given the following source:
{
"attachments" : [
{
"filename" : "ipsum.txt",
"data" : "dGhpcyBpcwpqdXN0IHNvbWUgdGV4dAo="
},
{
"filename" : "test.txt",
"data" : "VGhpcyBpcyBhIHRlc3QK"
}
]
}
In this case, we want to process the data field in each element
of the attachments field and insert
the properties into the document so the following foreach
processor is used:
PUT _ingest/pipeline/attachment
{
"description" : "Extract attachment information from arrays",
"processors" : [
{
"foreach": {
"field": "attachments",
"processor": {
"attachment": {
"target_field": "_ingest._value.attachment",
"field": "_ingest._value.data"
}
}
}
}
]
}
PUT my_index/my_type/my_id?pipeline=attachment
{
"attachments" : [
{
"filename" : "ipsum.txt",
"data" : "dGhpcyBpcwpqdXN0IHNvbWUgdGV4dAo="
},
{
"filename" : "test.txt",
"data" : "VGhpcyBpcyBhIHRlc3QK"
}
]
}
GET my_index/my_type/my_id
Returns this:
{
"_index" : "my_index",
"_type" : "my_type",
"_id" : "my_id",
"_version" : 1,
"found" : true,
"_source" : {
"attachments" : [
{
"filename" : "ipsum.txt",
"data" : "dGhpcyBpcwpqdXN0IHNvbWUgdGV4dAo=",
"attachment" : {
"content_type" : "text/plain; charset=ISO-8859-1",
"language" : "en",
"content" : "this is\njust some text",
"content_length" : 24
}
},
{
"filename" : "test.txt",
"data" : "VGhpcyBpcyBhIHRlc3QK",
"attachment" : {
"content_type" : "text/plain; charset=ISO-8859-1",
"language" : "en",
"content" : "This is a test",
"content_length" : 16
}
}
]
}
}
Note that the target_field needs to be set, otherwise the
default value is used which is a top level field attachment. The
properties on this top level field will contain the value of the
first attachment only. However, by specifying the
target_field on to a value on _ingest._value it will correctly
associate the properties with the correct attachment.