Alerting Troubleshooting

This page describes how to resolve common problems you might encounter with Alerting. If your problem isn’t described here, please review open issues in the following GitHub repositories:

Have a question? Contact us in the discuss forum.

Problem:

Rules with a small check interval, such as every two seconds, run later than scheduled.

Resolution:

Rules run as background tasks at a cadence defined by their check interval. When a Rule check interval is smaller than the Task Manager poll_interval the rule will run late.

Either tweak the Kibana Task Manager settings or increase the check interval of the rules in question.

For more details, see Tasks with small schedule intervals run late.

Problem:

Scheduled rules run at an inconsistent cadence, often running late.

Actions run long after the status of a rule changes, sending a notification of the change too late.

Solution:

Rules and actions run as background tasks by each Kibana instance at a default rate of ten tasks every three seconds.

If many rules or actions are scheduled to run at the same time, pending tasks will queue in Elasticsearch. Each Kibana instance then polls for pending tasks at a rate of up to ten tasks at a time, at three second intervals. Because rules and actions are backed by tasks, it is possible for pending tasks in the queue to exceed this capacity and run late.

For details on diagnosing the underlying causes of such delays, see Tasks run late.

Alerting and action tasks are identified by their type.

When diagnosing issues related to Alerting, focus on the tasks that begin with alerting: and actions:.

For more details on monitoring and diagnosing task execution in Task Manager, see Health monitoring.