Debugging your search queries

You can debug Enterprise Search queries by simply adding a header to your Search API requests. This allows you to analyze the queries Enterprise Search is sending to Elasticsearch and the responses returned. A common use case is to find the root cause of slow queries.

This is a fast and lightweight alternative to enabling monitoring and logging for your entire deployment.

If the X-Enterprise-Search-Debug header is present and set to true in your Search API request, this enables request-level debug logging. The response body now returns all the raw Elasticsearch queries associated with your Enterprise Search request, along with detailed timing data.

Here’s how the request looks using curl:

curl -X POST '<ENTERPRISE_SEARCH_BASE_URL>/api/as/v1/engines/test/search' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer search-REDACTED' \
  -H 'X-Enterprise-Search-Debug: true'

You can add another header to make the queries easy to track. The Search API automatically generates a unique ID for requests. However, if you add the X-Request-ID header, you can set this ID to an arbitrary unique value of your choice. This helps you find the request and all its associated Elasticsearch requests in Enterprise Search application logs. The following example sets the Request ID to pancakes.

Here is an example curl request with the header X-Enterprise-Search-Debug: true and the X-Request-ID header set to pancakes:

curl -X POST '<ENTERPRISE_SEARCH_BASE_URL>/api/as/v1/engines/test/search' \
  -H 'Content-Type: application/json' \
  -H 'Authorization: Bearer search-REDACTED' \
  -H 'X-Enterprise-Search-Debug: true' \
  -H 'X-Request-ID: pancakes' \
  -d '{"query": "great"}'

A simple way to see how the response has changed is to run the same search API request with and without the X-Enterprise-Search-Debug: true header and the X-Request-ID header. When you add the headers, the request response body will now have additional fields:

Under elasticsearch you will find:

You can search for pancakes in the Enterprise Search application logs to find all activity associated with this request. An example of a response body with this additional information looks like this:

{
  "meta": {
    "alerts": [],
    "warnings": [],
    "precision": 2,
    "page": { ...
    },
    "engine": {
      "name": "longevity",
      "type": "default"
    },
    "elasticsearch": {
      "request_header": {
        "index": "enterprise-search-engine-test",
        "preference": "de2f15d014d40b93578d255e6221fd60"
      },
      "request_body": {
        "query": { ...

        // The raw Elasticsearch query appears in this object...

        },
        "sort": [ ...
        ],
        "highlight": { ...
        },
        "size": 10,
        "from": 0,
        "timeout": "30000ms",
        "_source": [
          "title",
          "id"
        ]
      },
      "response": {

        // The amount of time Elasticsearch spent processing the raw query, in milliseconds:
        "took": 17,

        "timed_out": false,
        "_shards": {
          "total": 2,
          "successful": 2,
          "skipped": 0,
          "failed": 0
        },
        "hits": {
          "total": {
            "value": 4,
            "relation": "eq"
          },
          "max_score": null,
          "hits": [
          ]
        },
        "status": 200
      }
    },

    // The original X-Request-ID
    "request_id": "pancakes"
  },

You can run the raw Elasticsearch query directly against your Elasticsearch instance using the Elasticsearch search API. The Kibana dev console provides a UI where you can make search API requests against Elasticsearch.

You can also use the Elasticsearch search profile API parameter, profile: true, when querying Elasticsearch directly, to see more Elasticsearch performance profiling metrics. Paste the profiled Elasticsearch response into the Kibana Search Profiler UI to visualize it.

The debug information is also written into the Enterprise Search logs. The logs will include all Elasticsearch queries Enterprise Search performed up to and including the final raw search. For example, for App Search, the logs will show all queries to Elasticsearch to retrieve the engine’s saved synonyms and curations.

You will need to be comfortable analyzing raw Elasticsearch queries to debug using these tools. For difficult to diagnose issues, contact Elastic support for assistance.