Exclude cold and frozen data from individual rules
editExclude cold and frozen data from individual rules
editYour rule might perform slower or fail if it queries data from cold or frozen data tiers. To help Elasticsearch exclude cold and frozen data more efficiently, apply a Query DSL filter that ignores cold and frozen documents when your rule executes. You can add the filter when creating a new rule or updating an existing one.
To ensure that rules in your Kibana space exclude cold and frozen documents when executing, configure the excludedDataTiersForRuleExecution advanced setting. This setting does not apply to machine learning rules.
- This method is not supported for ES|QL and machine learning rules.
- Even when applying this filter, indicator match and event correlation rules may still fail if a frozen or cold shard that matches the rule’s specified index pattern is unavailable during rule executions. If failures occur, we recommend modifying the rule’s index patterns to only match indices containing hot tier data.
Here is a sample Query DSL filter that excludes frozen tier documents during rule execution:
{
"bool":{
"must_not":{
"terms":{
"_tier":[
"data_frozen"
]
}
}
}
}
Here is another sample Query DSL filter that excludes cold and frozen tier documents during rule execution:
{
"bool":{
"must_not":{
"terms":{
"_tier":[
"data_frozen", "data_cold"
]
}
}
}
}