EQL syntax reference

EQL queries require an event category and a matching condition. The where keyword connects them.

event_category where condition

An event category is an indexed value of the event category field. By default, the EQL search API uses the event.category field from the Elastic Common Schema (ECS). You can specify another event category field using the API’s event_category_field parameter.

For example, the following EQL query matches events with an event category of process and a process.name of svchost.exe:

process where process.name == "svchost.exe"

To match events of any category, use the any keyword. You can also use the any keyword to search for documents without a event category field.

For example, the following EQL query matches any documents with a network.protocol field value of http:

any where network.protocol == "http"

Use enclosing double quotes (") or three enclosing double quotes (""") to escape event categories that:

".my.event.category"
"my-event-category"
"my event category"
"6eventcategory"

""".my.event.category"""
"""my-event-category"""
"""my event category"""
"""6eventcategory"""

Use enclosing backticks (`) to escape field names that:

`my-field`
`my field`
`6myfield`

Use double backticks (``) to escape any backticks (`) in the field name.

my`field -> `my``field`

A condition consists of one or more criteria an event must match. You can specify and combine these criteria using the following operators. Most EQL operators are case-sensitive by default.

<   <=   ==   :   !=   >=   >

my_field like  "VALUE*"         // case-sensitive wildcard matching
my_field like~ "value*"         // case-insensitive wildcard matching

my_field regex  "VALUE[^Z].?"   // case-sensitive regex matching
my_field regex~ "value[^z].?"   // case-insensitive regex matching

You cannot chain comparisons. Instead, use a logical operator between comparisons. For example, foo < bar <= baz is not supported. However, you can rewrite the expression as foo < bar and bar <= baz, which is supported.

You also cannot compare a field to another field, even if the fields are changed using a function.

Example
The following EQL query compares the process.parent_name field value to a static value, foo. This comparison is supported.

However, the query also compares the process.parent.name field value to the process.name field. This comparison is not supported and will return an error for the entire query.

process where process.parent.name == "foo" and process.parent.name == process.name

Instead, you can rewrite the query to compare both the process.parent.name and process.name fields to static values.

process where process.parent.name == "foo" and process.name == "foo"

and  or  not

my_field in ("Value-1", "VALUE2", "VAL3")                 // case-sensitive
my_field in~ ("value-1", "value2", "val3")                // case-insensitive

my_field not in ("Value-1", "VALUE2", "VAL3")             // case-sensitive
my_field not in~ ("value-1", "value2", "val3")            // case-insensitive

my_field : ("value-1", "value2", "val3")                  // case-insensitive

my_field like  ("Value-*", "VALUE2", "VAL?")              // case-sensitive
my_field like~ ("value-*", "value2", "val?")              // case-insensitive

my_field regex  ("[vV]alue-[0-9]", "VALUE[^2].?", "VAL3") // case-sensitive
my_field regex~  ("value-[0-9]", "value[^2].?", "val3")   // case-sensitive

+  -  *  /  %

To match events solely on event category, use the where true condition.

For example, the following EQL query matches any file events:

file where true

To match any event, you can combine the any keyword with the where true condition:

any where true

By default, an EQL query can only contain fields that exist in the dataset you’re searching. A field exists in a dataset if it has an explicit, dynamic, or runtime mapping. If an EQL query contains a field that doesn’t exist, it returns an error.

If you aren’t sure if a field exists in a dataset, use the ? operator to mark the field as optional. If an optional field doesn’t exist, the query replaces it with null instead of returning an error.

Example
In the following query, the user.id field is optional.

network where ?user.id != null

If the user.id field exists in the dataset you’re searching, the query matches any network event that contains a user.id value. If the user.id field doesn’t exist in the dataset, EQL interprets the query as:

network where null != null

In this case, the query matches no events.

To match events containing any value for a field, compare the field to null using the != operator:

?my_field != null

To match events that do not contain a field value, compare the field to null using the == operator:

?my_field == null

Strings are enclosed in double quotes (").

"hello world"

Strings enclosed in single quotes (') are not supported.

When used within a string, special characters, such as a carriage return or double quote ("), must be escaped with a preceding backslash (\).

"example \r of \" escaped \n characters"

You can escape Unicode characters using a hexadecimal \u{XXXXXXXX} escape sequence. The hexadecimal value can be 2-8 characters and is case-insensitive. Values shorter than 8 characters are zero-padded. You can use these escape sequences to include non-printable or right-to-left (RTL) characters in your strings. For example, you can escape a right-to-left mark (RLM) as \u{200f}, \u{200F}, or \u{0000200f}.

Raw strings treat special characters, such as backslashes (\), as literal characters. Raw strings are enclosed in three double quotes (""").

"""Raw string with a literal double quote " and blackslash \ included"""

A raw string cannot contain three consecutive double quotes ("""). Instead, use a regular string with the \" escape sequence.

"String containing \"\"\" three double quotes"

For string comparisons using the : operator or like keyword, you can use the * and ? wildcards to match specific patterns. The * wildcard matches zero or more characters:

my_field : "doc*"     // Matches "doc", "docs", or "document" but not "DOS"
my_field : "*doc"     // Matches "adoc" or "asciidoc"
my_field : "d*c"      // Matches "doc" or "disc"

my_field like "DOC*"  // Matches "DOC", "DOCS", "DOCs", or "DOCUMENT" but not "DOS"
my_field like "D*C"   // Matches "DOC", "DISC", or "DisC"

The ? wildcard matches exactly one character:

my_field : "doc?"     // Matches "docs" but not "doc", "document", or "DOS"
my_field : "?doc"     // Matches "adoc" but not "asciidoc"
my_field : "d?c"      // Matches "doc" but not "disc"

my_field like "DOC?"  // Matches "DOCS" or "DOCs" but not "DOC", "DOCUMENT", or "DOS"
my_field like "D?c"   // Matches "DOC" but not "DISC"

The : operator and like keyword also support wildcards in list lookups:

my_field : ("doc*", "f*o", "ba?", "qux")
my_field like ("Doc*", "F*O", "BA?", "QUX")

You can use EQL sequences to describe and match an ordered series of events. Each item in a sequence is an event category and event condition, surrounded by square brackets ([ ]). Events are listed in ascending chronological order, with the most recent event listed last.

sequence
  [ event_category_1 where condition_1 ]
  [ event_category_2 where condition_2 ]
  ...

Example
The following EQL sequence query matches this series of ordered events:

sequence
  [ file where file.extension == "exe" ]
  [ process where true ]

You can use with maxspan to constrain a sequence to a specified timespan. All events in a matching sequence must occur within this duration, starting at the first event’s timestamp.

maxspan accepts time value arguments.

sequence with maxspan=30s
  [ event_category_1 where condition_1 ] by field_baz
  [ event_category_2 where condition_2 ] by field_bar
  ...

Example
The following sequence query uses a maxspan value of 15m (15 minutes). Events in a matching sequence must occur within 15 minutes of the first event’s timestamp.

sequence with maxspan=15m
  [ file where file.extension == "exe" ]
  [ process where true ]

Use the by keyword in a sequence query to only match events that share the same values, even if those values are in different fields. These shared values are called join keys. If a join key should be in the same field across all events, use sequence by.

sequence by field_foo
  [ event_category_1 where condition_1 ] by field_baz
  [ event_category_2 where condition_2 ] by field_bar
  ...

Example
The following sequence query uses the by keyword to constrain matching events to:

sequence
  [ file where file.extension == "exe" ] by user.name, file.path
  [ process where true ] by user.name, process.executable

Because the user.name field is shared across all events in the sequence, it can be included using sequence by. The following sequence is equivalent to the prior one.

sequence by user.name
  [ file where file.extension == "exe" ] by file.path
  [ process where true ] by process.executable

You can combine sequence by and with maxspan to constrain a sequence by both field values and a timespan.

sequence by field_foo with maxspan=30s
  [ event_category_1 where condition_1 ]
  [ event_category_2 where condition_2 ]
  ...

Example
The following sequence query uses sequence by and with maxspan to only match a sequence of events that:

sequence by user.name with maxspan=15m
  [ file where file.extension == "exe" ]
  [ process where true ]

By default, a join key must be a non-null field value. To allow null join keys, use the ? operator to mark the by field as optional. This is also helpful if you aren’t sure the dataset you’re searching contains the by field.

Example
The following sequence query uses sequence by to constrain matching events to:

sequence by process.pid, ?process.entity_id
  [process where process.name == "regsvr32.exe"]
  [network where true]

You can use the until keyword to specify an expiration event for a sequence. If this expiration event occurs between matching events in a sequence, the sequence expires and is not considered a match. If the expiration event occurs after matching events in a sequence, the sequence is still considered a match. The expiration event is not included in the results.

sequence
  [ event_category_1 where condition_1 ]
  [ event_category_2 where condition_2 ]
  ...
until [ event_category_3 where condition_3 ]

Example
A dataset contains the following event sequences, grouped by shared IDs:

A, B
A, B, C
A, C, B

The following EQL query searches the dataset for sequences containing event A followed by event B. Event C is used as an expiration event.

sequence by ID
  A
  B
until C

The query matches sequences A, B and A, B, C but not A, C, B.

sequence by process.pid
  [ process where event.type == "start" and process.name == "cmd.exe" ]
  [ process where file.extension == "exe" ]

sequence by process.pid
  [ process where event.type == "start" and process.name == "cmd.exe" ]
  [ process where file.extension == "exe" ]
until [ process where event.type == "stop" ]

Use a with runs statement to run the same event criteria successively within a sequence query. For example:

sequence
  [ process where event.type == "creation" ]
  [ library where process.name == "regsvr32.exe" ] with runs=3
  [ registry where true ]

is equivalent to:

sequence
  [ process where event.type == "creation" ]
  [ library where process.name == "regsvr32.exe" ]
  [ library where process.name == "regsvr32.exe" ]
  [ library where process.name == "regsvr32.exe" ]
  [ registry where true ]

The runs value must be between 1 and 100 (inclusive).

You can use a with runs statement with the by keyword. For example:

sequence
  [ process where event.type == "creation" ] by process.executable
  [ library where process.name == "regsvr32.exe" ] by dll.path with runs=3

You can use EQL functions to convert data types, perform math, manipulate strings, and more. For a list of supported functions, see Function reference.

Most EQL functions are case-sensitive by default. To make a function case-insensitive, use the ~ operator after the function name:

stringContains(process.name,".exe")  // Matches ".exe" but not ".EXE" or ".Exe"
stringContains~(process.name,".exe") // Matches ".exe", ".EXE", or ".Exe"

Using functions in EQL queries can result in slower search speeds. If you often use functions to transform indexed data, you can speed up search by making these changes during indexing instead. However, that often means slower index speeds.

Example
An index contains the file.path field. file.path contains the full path to a file, including the file extension.

When running EQL searches, users often use the endsWith function with the file.path field to match file extensions:

file where endsWith(file.path,".exe") or endsWith(file.path,".dll")

While this works, it can be repetitive to write and can slow search speeds. To speed up search, you can do the following instead:

These changes may slow indexing but allow for faster searches. Users can use the file.extension field instead of multiple endsWith function calls:

file where file.extension in ("exe", "dll")

We recommend testing and benchmarking any indexing changes before deploying them in production. See Tune for indexing speed and Tune for search speed.

EQL pipes filter, aggregate, and post-process events returned by an EQL query. You can use pipes to narrow down EQL query results or make them more specific.

Pipes are delimited using the pipe (|) character.

event_category where condition | pipe

Example
The following EQL query uses the tail pipe to return only the 10 most recent events matching the query.

authentication where agent.id == 4624
| tail 10

You can pass the output of a pipe to another pipe. This lets you use multiple pipes with a single query.

For a list of supported pipes, see Pipe reference.

EQL has the following limitations.

EQL retrieves field values using the search API’s fields parameter. Any limitations on the fields parameter also apply to EQL queries. For example, if _source is disabled for any returned fields or at index level, the values cannot be retrieved.

You cannot use EQL comparison operators to compare a field to another field. This applies even if the fields are changed using a function.

EQL searches do not support text fields. To a search a text field, use the EQL search API’s Query DSL filter parameter.

You cannot use EQL to search the values of a nested field or the sub-fields of a nested field. However, data streams and indices containing nested field mappings are otherwise supported.

Elasticsearch EQL differs from the Elastic Endgame EQL syntax as follows:

Sequence queries don’t find all potential matches for a sequence. This approach would be too slow and costly for large event data sets. Instead, a sequence query handles pending sequence matches as a state machine:

Example

A data set contains the following process events in ascending chronological order:

{ "index" : { "_id": "1" } }
{ "user": { "name": "root" }, "process": { "name": "attrib" }, ...}
{ "index" : { "_id": "2" } }
{ "user": { "name": "root" }, "process": { "name": "attrib" }, ...}
{ "index" : { "_id": "3" } }
{ "user": { "name": "elkbee" }, "process": { "name": "bash" }, ...}
{ "index" : { "_id": "4" } }
{ "user": { "name": "root" }, "process": { "name": "bash" }, ...}
{ "index" : { "_id": "5" } }
{ "user": { "name": "root" }, "process": { "name": "bash" }, ...}
{ "index" : { "_id": "6" } }
{ "user": { "name": "elkbee" }, "process": { "name": "attrib" }, ...}
{ "index" : { "_id": "7" } }
{ "user": { "name": "root" }, "process": { "name": "attrib" }, ...}
{ "index" : { "_id": "8" } }
{ "user": { "name": "elkbee" }, "process": { "name": "bash" }, ...}
{ "index" : { "_id": "9" } }
{ "user": { "name": "root" }, "process": { "name": "cat" }, ...}
{ "index" : { "_id": "10" } }
{ "user": { "name": "elkbee" }, "process": { "name": "cat" }, ...}
{ "index" : { "_id": "11" } }
{ "user": { "name": "root" }, "process": { "name": "cat" }, ...}

An EQL sequence query searches the data set:

sequence by user.name
  [process where process.name == "attrib"]
  [process where process.name == "bash"]
  [process where process.name == "cat"]

The query’s event items correspond to the following states:

• State A: [process where process.name == "attrib"]
• State B: [process where process.name == "bash"]
• Complete: [process where process.name == "cat"]

To find matching sequences, the query uses separate state machines for each unique user.name value. Based on the data set, you can expect two state machines: one for the root user and one for elkbee.

Pending sequence matches move through each machine’s states as follows:

{ "index" : { "_id": "1" } }
{ "user": { "name": "root" }, "process": { "name": "attrib" }, ...}
// Creates sequence [1] in state A for the "root" user.
//
// +------------------------"root"------------------------+
// |  +-----------+     +-----------+     +------------+  |
// |  |  State A  |     |  State B  |     |  Complete  |  |
// |  +-----------+     +-----------+     +------------+  |
// |  |    [1]    |     |           |     |            |  |
// |  +-----------+     +-----------+     +------------+  |
// +------------------------------------------------------+

{ "index" : { "_id": "2" } }
{ "user": { "name": "root" }, "process": { "name": "attrib" }, ...}
// Creates sequence [2] in state A for "root", overwriting sequence [1].
//
// +------------------------"root"------------------------+
// |  +-----------+     +-----------+     +------------+  |
// |  |  State A  |     |  State B  |     |  Complete  |  |
// |  +-----------+     +-----------+     +------------+  |
// |  |    [2]    |     |           |     |            |  |
// |  +-----------+     +-----------+     +------------+  |
// +------------------------------------------------------+

{ "index" : { "_id": "3" } }
{ "user": { "name": "elkbee" }, "process": { "name": "bash" }, ...}
// Nothing happens. The "elkbee" user has no pending sequence to move
// from state A to state B.
//
// +-----------------------"elkbee"-----------------------+
// |  +-----------+     +-----------+     +------------+  |
// |  |  State A  |     |  State B  |     |  Complete  |  |
// |  +-----------+     +-----------+     +------------+  |
// |  |           |     |           |     |            |  |
// |  +-----------+     +-----------+     +------------+  |
// +------------------------------------------------------+

{ "index" : { "_id": "4" } }
{ "user": { "name": "root" }, "process": { "name": "bash" }, ...}
// Sequence [2] moves out of state A for "root".
// State B for "root" now contains [2, 4].
// State A for "root" is empty.
//
// +------------------------"root"------------------------+
// |  +-----------+     +-----------+     +------------+  |
// |  |  State A  |     |  State B  |     |  Complete  |  |
// |  +-----------+ --> +-----------+     +------------+  |
// |  |           |     |   [2, 4]  |     |            |  |
// |  +-----------+     +-----------+     +------------+  |
// +------------------------------------------------------+

{ "index" : { "_id": "5" } }
{ "user": { "name": "root" }, "process": { "name": "bash" }, ...}
// Nothing happens. State A is empty for "root".
//
// +------------------------"root"------------------------+
// |  +-----------+     +-----------+     +------------+  |
// |  |  State A  |     |  State B  |     |  Complete  |  |
// |  +-----------+     +-----------+     +------------+  |
// |  |           |     |   [2, 4]  |     |            |  |
// |  +-----------+     +-----------+     +------------+  |
// +------------------------------------------------------+

{ "index" : { "_id": "6" } }
{ "user": { "name": "elkbee" }, "process": { "name": "attrib" }, ...}
// Creates sequence [6] in state A for "elkbee".
//
// +-----------------------"elkbee"-----------------------+
// |  +-----------+     +-----------+     +------------+  |
// |  |  State A  |     |  State B  |     |  Complete  |  |
// |  +-----------+     +-----------+     +------------+  |
// |  |    [6]    |     |           |     |            |  |
// |  +-----------+     +-----------+     +------------+  |
// +------------------------------------------------------+

{ "index" : { "_id": "7" } }
{ "user": { "name": "root" }, "process": { "name": "attrib" }, ...}
// Creates sequence [7] in state A for "root".
// Sequence [2, 4] remains in state B for "root".
//
// +------------------------"root"------------------------+
// |  +-----------+     +-----------+     +------------+  |
// |  |  State A  |     |  State B  |     |  Complete  |  |
// |  +-----------+     +-----------+     +------------+  |
// |  |    [7]    |     |   [2, 4]  |     |            |  |
// |  +-----------+     +-----------+     +------------+  |
// +------------------------------------------------------+

{ "index" : { "_id": "8" } }
{ "user": { "name": "elkbee" }, "process": { "name": "bash" }, ...}
// Sequence [6, 8] moves to state B for "elkbee".
// State A for "elkbee" is now empty.
//
// +-----------------------"elkbee"-----------------------+
// |  +-----------+     +-----------+     +------------+  |
// |  |  State A  |     |  State B  |     |  Complete  |  |
// |  +-----------+ --> +-----------+     +------------+  |
// |  |           |     |   [6, 8]  |     |            |  |
// |  +-----------+     +-----------+     +------------+  |
// +------------------------------------------------------+

{ "index" : { "_id": "9" } }
{ "user": { "name": "root" }, "process": { "name": "cat" }, ...}
// Sequence [2, 4, 9] is complete for "root".
// State B for "root" is now empty.
// Sequence [7] remains in state A.
//
// +------------------------"root"------------------------+
// |  +-----------+     +-----------+     +------------+  |
// |  |  State A  |     |  State B  |     |  Complete  |  |
// |  +-----------+     +-----------+ --> +------------+  |
// |  |    [7]    |     |           |     |  [2, 4, 9] |
// |  +-----------+     +-----------+     +------------+  |
// +------------------------------------------------------+

{ "index" : { "_id": "10" } }
{ "user": { "name": "elkbee" }, "process": { "name": "cat" }, ...}
// Sequence [6, 8, 10] is complete for "elkbee".
// State A and B for "elkbee" are now empty.
//
// +-----------------------"elkbee"-----------------------+
// |  +-----------+     +-----------+     +------------+  |
// |  |  State A  |     |  State B  |     |  Complete  |  |
// |  +-----------+     +-----------+ --> +------------+  |
// |  |           |     |           |     | [6, 8, 10] |
// |  +-----------+     +-----------+     +------------+  |
// +------------------------------------------------------+

{ "index" : { "_id": "11" } }
{ "user": { "name": "root" }, "process": { "name": "cat" }, ...}
// Nothing happens.
// The machines for "root" and "elkbee" remain the same.
//
// +------------------------"root"------------------------+
// |  +-----------+     +-----------+     +------------+  |
// |  |  State A  |     |  State B  |     |  Complete  |  |
// |  +-----------+     +-----------+     +------------+  |
// |  |    [7]    |     |           |     |  [2, 4, 9] |
// |  +-----------+     +-----------+     +------------+  |
// +------------------------------------------------------+
//
// +-----------------------"elkbee"-----------------------+
// |  +-----------+     +-----------+     +------------+  |
// |  |  State A  |     |  State B  |     |  Complete  |  |
// |  +-----------+     +-----------+     +------------+  |
// |  |           |     |           |     | [6, 8, 10] |
// |  +-----------+     +-----------+     +------------+  |
// +------------------------------------------------------+