Improving the ES|QL editor experience in Kibana

We’re going to cover the guiding principles behind improving the ES|QL editor experience in Kibana and what we did to achieve that goal. We’ll cover features like live validation, improved autocomplete and quick fixes which all streamline the ES|QL experience.

Background

Since Elastic 8.11, a technical preview is now available of Elastic’s new piped query language, ES|QL (Elasticsearch Query Language), which transforms, enriches, and simplifies data investigations. Powered by a new query engine, ES|QL delivers advanced search capabilities with concurrent processing, improving speed and efficiency, irrespective of data source and structure. Accelerate resolution by creating aggregations and visualizations from one screen, delivering an iterative, uninterrupted workflow.

As a developer, learning a new language can be both an interesting challenge and a frustrating scenario. For a query language having nice syntax, lots of documentation and examples makes it click, but then moving from the walled garden of the documentation examples into the real world queries can be challenging.

When adopting a new language as a developer I’m interested in quickly iterating and jumping from a trial and error environment to the documentation to check more in-depth topics about syntax, limits and caveats.

Writing a correct ES|QL query should be easy

With ES|QL we want to provide the best possible experience for developers to push on all of the possibilities that modern web editors can provide.

As such, the ES|QL editor in Kibana has a critical role as it is one of the main mediums for users to approach the new language. Improving its user experience is of high importance to us.

In order to improve the user experience in the editor these four principles have been identified:

The user should not need to memorize all of the knowledge regarding indices/fields/policies/functions etc…
It should take seconds, not minutes, to understand what’s wrong with a query.
Autocomplete should make it easy for users to build the right queries.
The user should not be blamed for errors, rather the editor should help to fix them.

Catching ES|QL errors early on (and fix them)

In 8.13 ES|QL in Discover offers a complete client side validation engine, making it easy to catch potential errors before submitting a query to Elasticsearch.

The validation runs while typing and offers immediate feedback for the incorrect parts of the query:

(When expanded it is possible to inspect specific errors in the ES|QL editor with cursor hovering)

The validation has some resiliency to syntax errors and can still provide useful information to the user with incomplete queries.

(The ES|QL can validate the entire query at multiple points: errors are collected and fully reported on demand)

As a developer that is comfortable using IDEs in my daily coding environment, I’m used to the quick fix menu that provides suggestions on how to address common problems like spelling errors or using the wrong quotes.

Kibana uses the Monaco editor under the hood, which is a smaller version of the VSCode editor, and that provides an interface to deliver a similar feature also on the web.

An initial quick fix feature has been developed and some basic suggestions are already supported:

(The new ES|QL will leverage internal knowledge to propose a quick fix with existing indexes)

Current list of supported quick fixes includes:

Wrong field quoting
Wrong literals quoting
Index, field (and metafields), function, policies typos
…and more will be added in subsequent versions

The quick fixes function is still in its initial development and we are looking for feedback and enhancement requests.

Better ES|QL auto-complete

Since its’ release ES|QL has been shipped with a basic auto-complete feature in the Kibana editor, which already provides some useful suggestions for first time users.

In 8.13 the autocomplete logic has been refactored with improved feedback for the user, leveraging all field and function types, together with a deep knowledge of its ES|QL implementation in Elasticsearch.

In simple terms this means that from 8.13 on autocomplete will only suggest the right “thing” in many scenarios that were before uncovered.

A list (not necessarily complete) of covered features are:

Propose the right function, even when used within another function:

Propose the right function ES|QL auto-complete

(The ES|QL autocomplete knows what functions are compatible with each other, even when nested)

Propose the right argument for a function, either filtering out fields by type or proposing the right constants

Propose the right argument for a function - ES|QL autocomplete

(Autocomplete can help with special enums for particular functions, listing all of them directly)

Know when to quote or not a field/index name

The new autocomplete attempts to reduce the amount of information the user has to keep in mind in order to build a query, with both the application of many contextual type filters and leveraging some deep knowledge of the new language syntax.

Provide more contextual help

The new autocomplete contains the hidden gem of providing a full contextual help for any proposed suggestion, in particular for functions or commands with examples.

Provide more contextual help 2 - esql auto compleate

(Autocomplete can provide full inline documentation with examples on demand for commands and functions)

Another useful way to get more information within the editor is hover over specific parts of the query, like the policy name to gather more metadata information about it.

(Contextual tooltips helps with quick summaries of enrich policies with same basic informations)

Make the most of your data with ES|QL

In this post, we showcased some of the new ES|QL Kibana editor features. In summary, the list of features are as follows:

Users can get immediate feedback when typing a query about syntax and/or invalid query statement
Users can quickly get fix suggestions on some specific errors
Index, fields and policies are automatically suggested to the users in the right place
Help is provided inline with full documentation and examples.

Elastic invites SREs and developers to experience this editor feature firsthand and unlock new horizons in their data tasks. Try it today at https://ela.st/free-trial now.

The release and timing of any features or functionality described in this post remain at Elastic's sole discretion. Any features or functionality not currently available may not be delivered on time or at all.

Elasticsearch is packed with new features to help you build the best search solutions for your use case. Dive into our sample notebooks to learn more, start a free cloud trial, or try Elastic on your local machine now.

Report an issue