IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

› › ›

ES|QL REST API

edit

ES|QL REST API

edit

Overview

edit

The ES|QL query API accepts an ES|QL query string in the query parameter, runs it, and returns the results. For example:

resp = client.esql.query(
    format="txt",
    query="FROM library | KEEP author, name, page_count, release_date | SORT page_count DESC | LIMIT 5",
)
print(resp)

const response = await client.esql.query({
  format: "txt",
  query:
    "FROM library | KEEP author, name, page_count, release_date | SORT page_count DESC | LIMIT 5",
});
console.log(response);

POST /_query?format=txt
{
  "query": "FROM library | KEEP author, name, page_count, release_date | SORT page_count DESC | LIMIT 5"
}

Which returns:

     author      |        name        |  page_count   | release_date
-----------------+--------------------+---------------+------------------------
Peter F. Hamilton|Pandora's Star      |768            |2004-03-02T00:00:00.000Z
Vernor Vinge     |A Fire Upon the Deep|613            |1992-06-01T00:00:00.000Z
Frank Herbert    |Dune                |604            |1965-06-01T00:00:00.000Z
Alastair Reynolds|Revelation Space    |585            |2000-03-15T00:00:00.000Z
James S.A. Corey |Leviathan Wakes     |561            |2011-06-02T00:00:00.000Z

Kibana Console

edit

If you are using Kibana Console (which is highly recommended), take advantage of the triple quotes """ when creating the query. This not only automatically escapes double quotes (") inside the query string but also supports multi-line requests:

resp = client.esql.query(
    format="txt",
    query="\n    FROM library\n    | KEEP author, name, page_count, release_date\n    | SORT page_count DESC\n    | LIMIT 5\n  ",
)
print(resp)

const response = await client.esql.query({
  format: "txt",
  query:
    "\n    FROM library\n    | KEEP author, name, page_count, release_date\n    | SORT page_count DESC\n    | LIMIT 5\n  ",
});
console.log(response);

POST /_query?format=txt
{
  "query": """
    FROM library
    | KEEP author, name, page_count, release_date
    | SORT page_count DESC
    | LIMIT 5
  """
}

Response formats

edit

ES|QL can return the data in the following human readable and binary formats. You can set the format by specifying the format parameter in the URL or by setting the Accept or Content-Type HTTP header.

The URL parameter takes precedence over the HTTP headers. If neither is specified then the response is returned in the same format as the request.

`format`	HTTP header	Description
Human readable
`csv`	`text/csv`	Comma-separated values
`json`	`application/json`	JSON (JavaScript Object Notation) human-readable format
`tsv`	`text/tab-separated-values`	Tab-separated values
`txt`	`text/plain`	CLI-like representation
`yaml`	`application/yaml`	YAML (YAML Ain’t Markup Language) human-readable format
Binary
`cbor`	`application/cbor`	Concise Binary Object Representation
`smile`	`application/smile`	Smile binary data format similar to CBOR
`arrow`	`application/vnd.apache.arrow.stream`	Experimental. Apache Arrow dataframes, IPC streaming format

The csv format accepts a formatting URL query attribute, delimiter, which indicates which character should be used to separate the CSV values. It defaults to comma (,) and cannot take any of the following values: double quote ("), carriage-return (\r) and new-line (\n). The tab (\t) can also not be used. Use the tsv format instead.

Filtering using Elasticsearch Query DSL

edit

Specify a Query DSL query in the filter parameter to filter the set of documents that an ES|QL query runs on.

resp = client.esql.query(
    format="txt",
    query="\n    FROM library\n    | KEEP author, name, page_count, release_date\n    | SORT page_count DESC\n    | LIMIT 5\n  ",
    filter={
        "range": {
            "page_count": {
                "gte": 100,
                "lte": 200
            }
        }
    },
)
print(resp)

const response = await client.esql.query({
  format: "txt",
  query:
    "\n    FROM library\n    | KEEP author, name, page_count, release_date\n    | SORT page_count DESC\n    | LIMIT 5\n  ",
  filter: {
    range: {
      page_count: {
        gte: 100,
        lte: 200,
      },
    },
  },
});
console.log(response);

POST /_query?format=txt
{
  "query": """
    FROM library
    | KEEP author, name, page_count, release_date
    | SORT page_count DESC
    | LIMIT 5
  """,
  "filter": {
    "range": {
      "page_count": {
        "gte": 100,
        "lte": 200
      }
    }
  }
}

Which returns:

    author     |                name                |  page_count   | release_date
---------------+------------------------------------+---------------+------------------------
Douglas Adams  |The Hitchhiker's Guide to the Galaxy|180            |1979-10-12T00:00:00.000Z

Columnar results

edit

By default, ES|QL returns results as rows. For example, FROM returns each individual document as one row. For the json, yaml, cbor and smile formats, ES|QL can return the results in a columnar fashion where one row represents all the values of a certain column in the results.

resp = client.esql.query(
    format="json",
    query="\n    FROM library\n    | KEEP author, name, page_count, release_date\n    | SORT page_count DESC\n    | LIMIT 5\n  ",
    columnar=True,
)
print(resp)

const response = await client.esql.query({
  format: "json",
  query:
    "\n    FROM library\n    | KEEP author, name, page_count, release_date\n    | SORT page_count DESC\n    | LIMIT 5\n  ",
  columnar: true,
});
console.log(response);

POST /_query?format=json
{
  "query": """
    FROM library
    | KEEP author, name, page_count, release_date
    | SORT page_count DESC
    | LIMIT 5
  """,
  "columnar": true
}

Which returns:

{
  "columns": [
    {"name": "author", "type": "text"},
    {"name": "name", "type": "text"},
    {"name": "page_count", "type": "integer"},
    {"name": "release_date", "type": "date"}
  ],
  "values": [
    ["Peter F. Hamilton", "Vernor Vinge", "Frank Herbert", "Alastair Reynolds", "James S.A. Corey"],
    ["Pandora's Star", "A Fire Upon the Deep", "Dune", "Revelation Space", "Leviathan Wakes"],
    [768, 613, 604, 585, 561],
    ["2004-03-02T00:00:00.000Z", "1992-06-01T00:00:00.000Z", "1965-06-01T00:00:00.000Z", "2000-03-15T00:00:00.000Z", "2011-06-02T00:00:00.000Z"]
  ]
}

Returning localized results

edit

Use the locale parameter in the request body to return results (especially dates) formatted per the conventions of the locale. If locale is not specified, defaults to en-US (English). Refer to JDK Supported Locales.

Syntax: the locale parameter accepts language tags in the (case-insensitive) format xy and xy-XY.

For example, to return a month name in French:

resp = client.esql.query(
    locale="fr-FR",
    query="\n          ROW birth_date_string = \"2023-01-15T00:00:00.000Z\"\n          | EVAL birth_date = date_parse(birth_date_string)\n          | EVAL month_of_birth = DATE_FORMAT(\"MMMM\",birth_date)\n          | LIMIT 5\n   ",
)
print(resp)

const response = await client.esql.query({
  locale: "fr-FR",
  query:
    '\n          ROW birth_date_string = "2023-01-15T00:00:00.000Z"\n          | EVAL birth_date = date_parse(birth_date_string)\n          | EVAL month_of_birth = DATE_FORMAT("MMMM",birth_date)\n          | LIMIT 5\n   ',
});
console.log(response);

POST /_query
{
  "locale": "fr-FR",
  "query": """
          ROW birth_date_string = "2023-01-15T00:00:00.000Z"
          | EVAL birth_date = date_parse(birth_date_string)
          | EVAL month_of_birth = DATE_FORMAT("MMMM",birth_date)
          | LIMIT 5
   """
}

Passing parameters to a query

edit

Values, for example for a condition, can be passed to a query "inline", by integrating the value in the query string itself:

resp = client.esql.query(
    query="\n    FROM library\n    | EVAL year = DATE_EXTRACT(\"year\", release_date)\n    | WHERE page_count > 300 AND author == \"Frank Herbert\"\n    | STATS count = COUNT(*) by year\n    | WHERE count > 0\n    | LIMIT 5\n  ",
)
print(resp)

const response = await client.esql.query({
  query:
    '\n    FROM library\n    | EVAL year = DATE_EXTRACT("year", release_date)\n    | WHERE page_count > 300 AND author == "Frank Herbert"\n    | STATS count = COUNT(*) by year\n    | WHERE count > 0\n    | LIMIT 5\n  ',
});
console.log(response);

POST /_query
{
  "query": """
    FROM library
    | EVAL year = DATE_EXTRACT("year", release_date)
    | WHERE page_count > 300 AND author == "Frank Herbert"
    | STATS count = COUNT(*) by year
    | WHERE count > 0
    | LIMIT 5
  """
}

To avoid any attempts of hacking or code injection, extract the values in a separate list of parameters. Use question mark placeholders (?) in the query string for each of the parameters:

resp = client.esql.query(
    query="\n    FROM library\n    | EVAL year = DATE_EXTRACT(\"year\", release_date)\n    | WHERE page_count > ? AND author == ?\n    | STATS count = COUNT(*) by year\n    | WHERE count > ?\n    | LIMIT 5\n  ",
    params=[
        300,
        "Frank Herbert",
        0
    ],
)
print(resp)

const response = await client.esql.query({
  query:
    '\n    FROM library\n    | EVAL year = DATE_EXTRACT("year", release_date)\n    | WHERE page_count > ? AND author == ?\n    | STATS count = COUNT(*) by year\n    | WHERE count > ?\n    | LIMIT 5\n  ',
  params: [300, "Frank Herbert", 0],
});
console.log(response);

POST /_query
{
  "query": """
    FROM library
    | EVAL year = DATE_EXTRACT("year", release_date)
    | WHERE page_count > ? AND author == ?
    | STATS count = COUNT(*) by year
    | WHERE count > ?
    | LIMIT 5
  """,
  "params": [300, "Frank Herbert", 0]
}

The parameters can be named parameters or positional parameters.

Named parameters use question mark placeholders (?) followed by a string.

resp = client.esql.query(
    query="\n    FROM library\n    | EVAL year = DATE_EXTRACT(\"year\", release_date)\n    | WHERE page_count > ?page_count AND author == ?author\n    | STATS count = COUNT(*) by year\n    | WHERE count > ?count\n    | LIMIT 5\n  ",
    params=[
        {
            "page_count": 300
        },
        {
            "author": "Frank Herbert"
        },
        {
            "count": 0
        }
    ],
)
print(resp)

const response = await client.esql.query({
  query:
    '\n    FROM library\n    | EVAL year = DATE_EXTRACT("year", release_date)\n    | WHERE page_count > ?page_count AND author == ?author\n    | STATS count = COUNT(*) by year\n    | WHERE count > ?count\n    | LIMIT 5\n  ',
  params: [
    {
      page_count: 300,
    },
    {
      author: "Frank Herbert",
    },
    {
      count: 0,
    },
  ],
});
console.log(response);

POST /_query
{
  "query": """
    FROM library
    | EVAL year = DATE_EXTRACT("year", release_date)
    | WHERE page_count > ?page_count AND author == ?author
    | STATS count = COUNT(*) by year
    | WHERE count > ?count
    | LIMIT 5
  """,
  "params": [{"page_count" : 300}, {"author" : "Frank Herbert"}, {"count" : 0}]
}

Positional parameters use question mark placeholders (?) followed by an integer.

resp = client.esql.query(
    query="\n    FROM library\n    | EVAL year = DATE_EXTRACT(\"year\", release_date)\n    | WHERE page_count > ?1 AND author == ?2\n    | STATS count = COUNT(*) by year\n    | WHERE count > ?3\n    | LIMIT 5\n  ",
    params=[
        300,
        "Frank Herbert",
        0
    ],
)
print(resp)

const response = await client.esql.query({
  query:
    '\n    FROM library\n    | EVAL year = DATE_EXTRACT("year", release_date)\n    | WHERE page_count > ?1 AND author == ?2\n    | STATS count = COUNT(*) by year\n    | WHERE count > ?3\n    | LIMIT 5\n  ',
  params: [300, "Frank Herbert", 0],
});
console.log(response);

POST /_query
{
  "query": """
    FROM library
    | EVAL year = DATE_EXTRACT("year", release_date)
    | WHERE page_count > ?1 AND author == ?2
    | STATS count = COUNT(*) by year
    | WHERE count > ?3
    | LIMIT 5
  """,
  "params": [300, "Frank Herbert", 0]
}

Running an async ES|QL query

edit

The ES|QL async query API lets you asynchronously execute a query request, monitor its progress, and retrieve results when they become available.

Executing an ES|QL query is commonly quite fast, however queries across large data sets or frozen data can take some time. To avoid long waits, run an async ES|QL query.

Queries initiated by the async query API may return results or not. The wait_for_completion_timeout property determines how long to wait for the results. If the results are not available by this time, a query id is returned which can be later used to retrieve the results. For example:

resp = client.esql.async_query(
    body={
        "query": "\n    FROM library\n    | EVAL year = DATE_TRUNC(1 YEARS, release_date)\n    | STATS MAX(page_count) BY year\n    | SORT year\n    | LIMIT 5\n  ",
        "wait_for_completion_timeout": "2s"
    },
)
print(resp)

const response = await client.transport.request({
  method: "POST",
  path: "/_query/async",
  body: {
    query:
      "\n    FROM library\n    | EVAL year = DATE_TRUNC(1 YEARS, release_date)\n    | STATS MAX(page_count) BY year\n    | SORT year\n    | LIMIT 5\n  ",
    wait_for_completion_timeout: "2s",
  },
});
console.log(response);

POST /_query/async
{
  "query": """
    FROM library
    | EVAL year = DATE_TRUNC(1 YEARS, release_date)
    | STATS MAX(page_count) BY year
    | SORT year
    | LIMIT 5
  """,
  "wait_for_completion_timeout": "2s"
}

If the results are not available within the given timeout period, 2 seconds in this case, no results are returned but rather a response that includes:

A query ID
An is_running value of true, indicating the query is ongoing

The query continues to run in the background without blocking other requests.

{
  "id": "FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
  "is_running": true
}

To check the progress of an async query, use the ES|QL async query get API with the query ID. Specify how long you’d like to wait for complete results in the wait_for_completion_timeout parameter.

resp = client.esql.async_query_get(
    id="FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=",
    wait_for_completion_timeout="30s",
    body=None,
)
print(resp)

response = client.esql.async_query_get(
  id: 'FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=',
  wait_for_completion_timeout: '30s'
)
puts response

const response = await client.transport.request({
  method: "GET",
  path: "/_query/async/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE&#x3D;",
  querystring: {
    wait_for_completion_timeout: "30s",
  },
});
console.log(response);

GET /_query/async/FmNJRUZ1YWZCU3dHY1BIOUhaenVSRkEaaXFlZ3h4c1RTWFNocDdnY2FSaERnUTozNDE=?wait_for_completion_timeout=30s

If the response’s is_running value is false, the query has finished and the results are returned.

{
  "is_running": false,
  "columns": ...
}

Use the ES|QL async query delete API to delete an async query before the keep_alive period ends. If the query is still running, Elasticsearch cancels it.

DELETE /_query/async/FmdMX2pIang3UWhLRU5QS0lqdlppYncaMUpYQ05oSkpTc3kwZ21EdC1tbFJXQToxOTI=

« Using ES|QL Using ES|QL in Kibana »