Example: Enrich your data by matching a value to a range
editExample: Enrich your data by matching a value to a range
editA range enrich policy uses a term
query to match a number, date, or IP address in incoming documents to a range
of the same type in the enrich index. Matching a range to a range is not
supported.
The following example creates a range enrich policy that adds a descriptive network name and
responsible department to incoming documents based on an IP address. It then
adds the enrich policy to a processor in an ingest pipeline.
Use the create index API with the appropriate mappings to create a source index.
response = client.indices.create(
index: 'networks',
body: {
mappings: {
properties: {
range: {
type: 'ip_range'
},
name: {
type: 'keyword'
},
department: {
type: 'keyword'
}
}
}
}
)
puts response
PUT /networks
{
"mappings": {
"properties": {
"range": { "type": "ip_range" },
"name": { "type": "keyword" },
"department": { "type": "keyword" }
}
}
}
The following index API request indexes a new document to that index.
PUT /networks/_doc/1?refresh=wait_for
{
"range": "10.100.0.0/16",
"name": "production",
"department": "OPS"
}
Use the create enrich policy API to create an enrich policy with the
range policy type. This policy must include:
- One or more source indices
-
A
match_field, the field from the source indices used to match incoming documents - Enrich fields from the source indices you’d like to append to incoming documents
Since we plan to enrich documents based on an IP address, the policy’s
match_field must be an ip_range field.
response = client.enrich.put_policy(
name: 'networks-policy',
body: {
range: {
indices: 'networks',
match_field: 'range',
enrich_fields: [
'name',
'department'
]
}
}
)
puts response
PUT /_enrich/policy/networks-policy
{
"range": {
"indices": "networks",
"match_field": "range",
"enrich_fields": ["name", "department"]
}
}
Use the execute enrich policy API to create an enrich index for the policy.
POST /_enrich/policy/networks-policy/_execute
Use the create or update pipeline API to create an ingest pipeline. In the pipeline, add an enrich processor that includes:
- Your enrich policy.
-
The
fieldof incoming documents used to match documents from the enrich index. -
The
target_fieldused to store appended enrich data for incoming documents. This field contains thematch_fieldandenrich_fieldsspecified in your enrich policy.
PUT /_ingest/pipeline/networks_lookup
{
"processors" : [
{
"enrich" : {
"description": "Add 'network' data based on 'ip'",
"policy_name": "networks-policy",
"field" : "ip",
"target_field": "network",
"max_matches": "10"
}
}
]
}
Use the ingest pipeline to index a document. The incoming document should
include the field specified in your enrich processor.
PUT /my-index-000001/_doc/my_id?pipeline=networks_lookup
{
"ip": "10.100.34.1"
}
To verify the enrich processor matched and appended the appropriate field data, use the get API to view the indexed document.
GET /my-index-000001/_doc/my_id
The API returns the following response:
{
"_index" : "my-index-000001",
"_id" : "my_id",
"_version" : 1,
"_seq_no" : 0,
"_primary_term" : 1,
"found" : true,
"_source" : {
"ip" : "10.100.34.1",
"network" : [
{
"name" : "production",
"range" : "10.100.0.0/16",
"department" : "OPS"
}
]
}
}