Rejected requests
editRejected requests
editWhen Elasticsearch rejects a request, it stops the operation and returns an error with a
429
response code. Rejected requests are commonly caused by:
-
A depleted thread pool. A depleted
search
orwrite
thread pool returns aTOO_MANY_REQUESTS
error message. - A circuit breaker error.
-
High indexing pressure that exceeds the
indexing_pressure.memory.limit
.
Check rejected tasks
editTo check the number of rejected tasks for each thread pool, use the
cat thread pool API. A high ratio of rejected
to
completed
tasks, particularly in the search
and write
thread pools, means
Elasticsearch regularly rejects requests.
resp = client.cat.thread_pool( v=True, h="id,name,queue,active,rejected,completed", ) print(resp)
const response = await client.cat.threadPool({ v: "true", h: "id,name,queue,active,rejected,completed", }); console.log(response);
GET /_cat/thread_pool?v=true&h=id,name,queue,active,rejected,completed
write
thread pool rejections frequently appear in the erring API and
correlating log as EsRejectedExecutionException
with either
QueueResizingEsThreadPoolExecutor
or queue capacity
.
These errors are often related to backlogged tasks.
Check circuit breakers
editTo check the number of tripped circuit breakers, use the node stats API.
resp = client.nodes.stats( metric="breaker", ) print(resp)
const response = await client.nodes.stats({ metric: "breaker", }); console.log(response);
GET /_nodes/stats/breaker
These statistics are cumulative from node startup. For more information, see circuit breaker errors.
Check indexing pressure
editTo check the number of indexing pressure rejections, use the node stats API.
resp = client.nodes.stats( human=True, filter_path="nodes.*.indexing_pressure", ) print(resp)
const response = await client.nodes.stats({ human: "true", filter_path: "nodes.*.indexing_pressure", }); console.log(response);
GET _nodes/stats?human&filter_path=nodes.*.indexing_pressure
These stats are cumulative from node startup.
Indexing pressure rejections appear as an
EsRejectedExecutionException
, and indicate that they were rejected due
to coordinating_and_primary_bytes
, coordinating
, primary
, or replica
.
These errors are often related to backlogged tasks,
bulk index sizing, or the ingest target’s
refresh_interval
setting.
Prevent rejected requests
editFix high CPU and memory usage
If Elasticsearch regularly rejects requests and other tasks, your cluster likely has high CPU usage or high JVM memory pressure. For tips, see High CPU usage and High JVM memory pressure.