Multi Search API
editMulti Search API
editThe multi search API allows to execute several search requests within
the same API. The endpoint for it is _msearch
.
The format of the request is similar to the bulk API format and makes use of the newline delimited JSON (NDJSON) format. The structure is as follows (the structure is specifically optimized to reduce parsing if a specific search ends up redirected to another node):
header\n body\n header\n body\n
NOTE: the final line of data must end with a newline character \n
. Each newline character
may be preceded by a carriage return \r
. When sending requests to this endpoint the
Content-Type
header should be set to application/x-ndjson
.
The header part includes which index / indices to search on, the search_type
, preference
,
and routing
. The body includes the typical search body request (including the query
,
aggregations
, from
, size
, and so on). Here is an example:
$ cat requests {"index" : "test"} {"query" : {"match_all" : {}}, "from" : 0, "size" : 10} {"index" : "test", "search_type" : "dfs_query_then_fetch"} {"query" : {"match_all" : {}}} {} {"query" : {"match_all" : {}}} {"query" : {"match_all" : {}}} {"search_type" : "dfs_query_then_fetch"} {"query" : {"match_all" : {}}}
$ curl -H "Content-Type: application/x-ndjson" -XGET localhost:9200/_msearch --data-binary "@requests"; echo
Note, the above includes an example of an empty header (can also be just without any content) which is supported as well.
The response returns a responses
array, which includes the search
response and status code for each search request matching its order in
the original multi search request. If there was a complete failure for that
specific search request, an object with error
message and corresponding
status code will be returned in place of the actual search response.
The endpoint allows to also search against an index/indices in the URI itself, in which case it will be used as the default unless explicitly defined otherwise in the header. For example:
GET twitter/_msearch {} {"query" : {"match_all" : {}}, "from" : 0, "size" : 10} {} {"query" : {"match_all" : {}}} {"index" : "twitter2"} {"query" : {"match_all" : {}}}
The above will execute the search against the twitter
index for all the
requests that don’t define an index, and the last one will be executed
against the twitter2
index.
The search_type
can be set in a similar manner to globally apply to
all search requests.
The msearch’s max_concurrent_searches
request parameter can be used to control
the maximum number of concurrent searches the multi search api will execute.
This default is based on the number of data nodes and the default search thread pool size.
The request parameter max_concurrent_shard_requests
can be used to control
the maximum number of concurrent shard requests that each sub search request
will execute per node. This parameter should be used to protect a single
request from overloading a cluster (e.g., a default request will hit all
indices in a cluster which could cause shard request rejections if the number
of shards per node is high). This default value is 5
.In certain scenarios
parallelism isn’t achieved through concurrent request such that this protection
will result in poor performance. For instance in an environment where only a
very low number of concurrent search requests are expected it might help to
increase this value to a higher number.
Security
editTemplate support
editMuch like described in Search Template for the _search resource, _msearch also provides support for templates. Submit them like follows:
GET _msearch/template {"index" : "twitter"} { "source" : "{ \"query\": { \"match\": { \"message\" : \"{{keywords}}\" } } } }", "params": { "query_type": "match", "keywords": "some message" } } {"index" : "twitter"} { "source" : "{ \"query\": { \"match_{{template}}\": {} } }", "params": { "template": "all" } }
for inline templates.
You can also create search templates:
POST /_scripts/my_template_1 { "script": { "lang": "mustache", "source": { "query": { "match": { "message": "{{query_string}}" } } } } }
POST /_scripts/my_template_2 { "script": { "lang": "mustache", "source": { "query": { "term": { "{{field}}": "{{value}}" } } } } }
and later use them in a _msearch:
GET _msearch/template {"index" : "main"} { "id": "my_template_1", "params": { "query_string": "some message" } } {"index" : "main"} { "id": "my_template_2", "params": { "field": "user", "value": "test" } }
Partial responses
editTo ensure fast responses, the multi search API will respond with partial results if one or more shards fail. See Shard failures for more information.