Search APIs

Most search APIs are multi-index, with the exception of the Explain API endpoints.

When executing a search, Elasticsearch will pick the "best" copy of the data based on the adaptive replica selection formula. Which shards will be searched on can also be controlled by providing the routing parameter. For example, when indexing tweets, the routing value can be the user name:

POST /twitter/_doc?routing=kimchy
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

In such a case, if we want to search only on the tweets for a specific user, we can specify it as the routing, resulting in the search hitting only the relevant shard:

POST /twitter/_search?routing=kimchy
{
    "query": {
        "bool" : {
            "must" : {
                "query_string" : {
                    "query" : "some query string here"
                }
            },
            "filter" : {
                "term" : { "user" : "kimchy" }
            }
        }
    }
}

The routing parameter can be multi valued represented as a comma separated string. This will result in hitting the relevant shards where the routing values match to.

By default, Elasticsearch will use what is called adaptive replica selection. This allows the coordinating node to send the request to the copy deemed "best" based on a number of criteria:

This can be turned off by changing the dynamic cluster setting cluster.routing.use_adaptive_replica_selection from true to false:

PUT /_cluster/settings
{
    "transient": {
        "cluster.routing.use_adaptive_replica_selection": false
    }
}

If adaptive replica selection is turned off, searches are sent to the index/indices shards in a round robin fashion between all copies of the data (primaries and replicas).

A search can be associated with stats groups, which maintains a statistics aggregation per group. It can later be retrieved using the indices stats API specifically. For example, here is a search body request that associate the request with two different groups:

POST /_search
{
    "query" : {
        "match_all" : {}
    },
    "stats" : ["group1", "group2"]
}

Individual searches can have a timeout as part of the Request Body Search. Since search requests can originate from many sources, Elasticsearch has a dynamic cluster-level setting for a global search timeout that applies to all search requests that do not set a timeout in the request body. These requests will be cancelled after the specified time using the mechanism described in the following section on Search Cancellation. Therefore the same caveats about timeout responsiveness apply.

The setting key is search.default_search_timeout and can be set using the Cluster update settings endpoints. The default value is no global timeout. Setting this value to -1 resets the global search timeout to no timeout.

Searches can be cancelled using standard task cancellation mechanism and are also automatically cancelled when the http connection used to perform the request is closed by the client. It is fundamental that the http client sending requests closes connections whenever requests time out or are aborted.

By default Elasticsearch doesn’t reject any search requests based on the number of shards the request hits. While Elasticsearch will optimize the search execution on the coordinating node a large number of shards can have a significant impact CPU and memory wise. It is usually a better idea to organize data in such a way that there are fewer larger shards. In case you would like to configure a soft limit, you can update the action.search.shard_count.limit cluster setting in order to reject search requests that hit too many shards.

The request parameter max_concurrent_shard_requests can be used to control the maximum number of concurrent shard requests the search API will execute per node for the request. This parameter should be used to protect a single request from overloading a cluster (e.g., a default request will hit all indices in a cluster which could cause shard request rejections if the number of shards per node is high). This default value is 5.