Using ES|QL in Kibana

edit

You can use ES|QL in Kibana to query and aggregate your data, create visualizations, and set up alerts.

This guide shows you how to use ES|QL in Kibana. To follow along with the queries, load the "Sample web logs" sample data set by clicking Try sample data from the Kibana Home, selecting Other sample data sets, and clicking Add data on the Sample web logs card.

Get started with ES|QL

edit

To get started with ES|QL in Discover, open the main menu and select Discover. Next, from the Data views menu, select Try ES|QL.

esql data view menu

The ability to select ES|QL from the Data views menu can be enabled and disabled using the discover:enableESQL setting from Advanced Settings.

The query bar

edit

After switching to ES|QL mode, the query bar shows a sample query. For example:

from kibana_sample_data_logs | limit 10

Every query starts with a source command. In this query, the source command is FROM. FROM retrieves data from data streams, indices, or aliases. In this example, the data is retrieved from kibana_sample_data_logs.

A source command can be followed by one or more processing commands. In this query, the processing command is LIMIT. LIMIT limits the number of rows that are retrieved.

Click the help icon (esql icon help) to open the in-product reference documentation for all commands and functions.

To make it easier to write queries, auto-complete offers suggestions with possible commands and functions:

esql kibana auto complete

ES|QL keywords are case-insensitive. The following query is identical to the previous one:

FROM kibana_sample_data_logs | LIMIT 10

Expand the query bar

edit

For readability, you can put each processing command on a new line. The following query is identical to the previous one:

FROM kibana_sample_data_logs
| LIMIT 10

To make it easier to write multi-line queries, click the double-headed arrow button (esql icon expand query bar) to expand the query bar:

esql expanded query bar

To return to a compact query bar, click the minimize editor button (esql icon minimize query bar).

Warnings

edit

A query may result in warnings, for example when querying an unsupported field type. When that happens, a warning symbol is shown in the query bar. To see the detailed warning, expand the query bar, and click warnings.

The results table

edit

For the example query, the results table shows 10 rows. Omitting the LIMIT command, the results table defaults to up to 1000 rows. Using LIMIT, you can increase the limit to up to 10,000 rows.

the 10,000 row limit only applies to the number of rows that are retrieved by the query and displayed in Discover. Any query or aggregation runs on the full data set.

Each row shows two columns for the example query: a column with the @timestamp field and a column with the full document. To display specific fields from the documents, use the KEEP command:

FROM kibana_sample_data_logs
| KEEP @timestamp, bytes, geo.dest

To display all fields as separate columns, use KEEP *:

FROM kibana_sample_data_logs
| KEEP *

The maximum number of columns in Discover is 50. If a query returns more than 50 columns, Discover only shows the first 50.

Sorting

edit

To sort on one of the columns, click the column name you want to sort on and select the sort order. Note that this performs client-side sorting. It only sorts the rows that were retrieved by the query, which may not be the full dataset because of the (implicit) limit. To sort the full data set, use the SORT command:

FROM kibana_sample_data_logs
| KEEP @timestamp, bytes, geo.dest
| SORT bytes DESC

Time filtering

edit

To display data within a specified time range, use the time filter. The time filter is only enabled when the indices you’re querying have a field called @timestamp.

If your indices do not have a timestamp field called @timestamp, you can limit the time range using the WHERE command and the NOW function. For example, if the timestamp field is called timestamp, to query the last 15 minutes of data:

FROM kibana_sample_data_logs
| WHERE timestamp > NOW() - 15minutes

Analyze and visualize data

edit

Between the query bar and the results table, Discover shows a date histogram visualization. If the indices you’re querying do not contain an @timestamp field, the histogram is not shown.

The visualization adapts to the query. A query’s nature determines the type of visualization. For example, this query aggregates the total number of bytes per destination country:

FROM kibana_sample_data_logs
| STATS total_bytes = SUM(bytes) BY geo.dest
| SORT total_bytes DESC
| LIMIT 3

The resulting visualization is a bar chart showing the top 3 countries:

esql kibana bar chart

To change the visualization into another type, click the visualization type dropdown:

esql kibana visualization type

To make other changes to the visualization, like the axes and colors, click the pencil button (esql icon edit visualization). This opens an in-line editor:

esql kibana in line editor

You can save the visualization to a new or existing dashboard by clicking the save button (esql icon save visualization). Once saved to a dashboard, you can continue to make changes to visualization. Click the options button in the top-right (esql icon options) and select Edit ESQL visualization to open the in-line editor:

esql kibana edit on dashboard

Create an enrich policy

edit

The ES|QL ENRICH command enables you to enrich your query dataset with fields from another dataset. Before you can use ENRICH, you need to create and execute an enrich policy. If a policy exists, it will be suggested by auto-complete. If not, click Click to create to create one.

esql kibana enrich autocomplete

Next, you can enter a policy name, the policy type, source indices, and optionally a query:

esql kibana enrich step 1

Click Next to select the match field and enrich fields:

esql kibana enrich step 2

Finally, click Create and execute.

Now, you can use the enrich policy in an ES|QL query:

esql kibana enriched data

Create an alerting rule

edit

You can use ES|QL queries to create alerts. From Discover, click Alerts and select Create search threshold rule. This opens a panel that enables you to create a rule using an ES|QL query. Next, you can test the query, add a connector, and save the rule.

esql kibana create rule

Limitations

edit
  • The user interface to filter data is not enabled when Discover is in ES|QL mode. To filter data, write a query that uses the WHERE command instead.
  • In ES|QL mode, clicking a field in the field list in Discover does not show quick statistics for that field.
  • Discover shows no more than 10,000 rows. This limit only applies to the number of rows that are retrieved by the query and displayed in Discover. Queries and aggregations run on the full data set.
  • Discover shows no more than 50 columns. If a query returns more than 50 columns, Discover only shows the first 50.
  • CSV export from Discover shows no more than 10,000 rows. This limit only applies to the number of rows that are retrieved by the query and displayed in Discover. Queries and aggregations run on the full data set.
  • Querying many indices at once without any filters can cause an error in kibana which looks like [esql] > Unexpected error from Elasticsearch: The content length (536885793) is bigger than the maximum allowed string (536870888). The response from ES|QL is too long. Use DROP or KEEP to limit the number of fields returned.