Profiling your search queries

If you need to dig deeper into your search query performance, you can enable request-level Elasticsearch profiling. Adding the X-Enterprise-Search-Profile:true header lets you profile Enterprise Search search queries for specific requests.

This returns detailed timing information about the execution of individual components in a search request, by leveraging the Elasticsearch Profile API:

"The Profile API gives the user insight into how search requests are executed at a low level so that the user can understand why certain requests are slow, and take steps to improve them."

The profiling information is returned directly in the response body and is also written into the Enterprise Search logs. This is an advanced tool and the output of the response is verbose. You will need to pretty-print the response and be comfortable analyzing Elasticsearch queries.

Use the X-Enterprise-Search-Profile: true header together with the X-Enterprise-Search-Debug: true header to view profiling data and the underlying Elasticsearch request.

Optionally, add the X-Request-ID header to make it easier to find profiling information for the request in application log files.

Here is an example curl request with the header X-Enterprise-Search-Debug: true, the header X-Enterprise-Search-Profile:true, and the X-Request-ID header set to pancakes:

curl -X POST '<ENTERPRISE_SEARCH_BASE_URL>/api/as/v1/engines/test/search' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer search-REDACTED' \
  -H 'X-Enterprise-Search-Debug: true' \
  -H 'X-Enterprise-Search-Profile: true' \
  -H 'X-Request-ID: pancakes' \
  -d '{"query": "great"}'

Here is what the response might look like. Note this is a truncated example of the meta.profile portion of the response body:

{
  "meta": {
    "alerts": [],
    "warnings": [],
    "precision": 2,
    "page": {
    "current": 1,
    "total_pages": 1,
    "total_results": 9,
    "size": 10
  },
  "engine": {
    "name": "test",
    "type": "default"
    },
  "profile": {
    "shards": [
      {
        "id": "[pancakes][.enterprise-search-engine-test-1][0]",
        "searches": [
          {
            "query": [
              {
                "type": "MatchAllDocsQuery",
                "description": "*:*",
                "time_in_nanos": 4228,
                "breakdown": {
                  "set_min_competitive_score_count": 0,
                  "match_count": 0,
                  "shallow_advance_count": 0,
                  "set_min_competitive_score": 0,
                  "next_doc": 270,
                  "match": 0,
                  "next_doc_count": 1,
                  "score_count": 1,
                  "compute_max_score_count": 0,
                  "compute_max_score": 0,
                  "advance": 663,
                  "advance_count": 1,
                  "score": 620,
                  "build_scorer_count": 2,
                  "create_weight": 512,
                  "shallow_advance": 0,
                  "create_weight_count": 1,
                  "build_scorer": 2163
               }
             }
           ],
           "rewrite_time": 1596,
           "collector": [
             {
              "name": "SimpleFieldCollector",
              "reason": "search_top_hits",
              "time_in_nanos": 15307
             }
           ]
         }
       ],
          "aggregations": []
      ...

You can visualize the profiling information using the Kibana Search Profiler UI. Search Profiler is automatically enabled in Kibana. Open the main menu, click Dev Tools, and then click Search Profiler to get started. Paste in the full JSON object surrounding the profile section, e.g.:

{
   "profile": {
      ...
   }
}

For difficult to diagnose issues, contact Elastic support for assistance.