EQL syntax reference
editEQL syntax reference
editThis functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
Elasticsearch supports a subset of EQL syntax. See Limitations.
Basic syntax
editEQL queries require an event category and a matching condition. The where
keyword connects them.
event_category where condition
For example, the following EQL query matches process
events with a
process.name
field value of svchost.exe
:
process where process.name == "svchost.exe"
Event categories
editIn Elasticsearch, an event category is a valid, indexed value of the
event category field. You can set the event category
field using the event_category_field
parameter of the EQL search API.
Match any event category
editTo match events of any category, use the any
keyword. You can also use the
any
keyword to search for documents without a event category field.
For example, the following EQL query matches any documents with a
network.protocol
field value of http
:
any where network.protocol == "http"
Escape an event category or field name
editEvent categories or field names that contain a hyphen (-
), contain a space, or
start with a numeral must be escaped with enclosing backticks (`).
`my-field` `my field` `6myfield`
Conditions
editA condition consists of one or more criteria an event must match. You can specify and combine these criteria using the following operators:
Comparison operators
edit< <= == != >= >
Definitions
-
<
(less than) -
Returns
true
if the value to the left of the operator is less than the value to the right. Otherwise returnsfalse
. -
<=
(less than or equal) -
Returns
true
if the value to the left of the operator is less than or equal to the value to the right. Otherwise returnsfalse
. -
==
(equal) -
Returns
true
if the values to the left and right of the operator are equal. Otherwise returnsfalse
. -
!=
(not equal) -
Returns
true
if the values to the left and right of the operator are not equal. Otherwise returnsfalse
. -
>=
(greater than or equal) -
Returns
true
if the value to the left of the operator is greater than or equal to the value to the right. Otherwise returnsfalse
. -
>
(greater than) -
Returns
true
if the value to the left of the operator is greater than the value to the right. Otherwise returnsfalse
.
You also cannot use comparison operators to compare a field to another field. This applies even if the fields are changed using a function.
Example
The following EQL query compares the process.parent_name
field
value to a static value, foo
. This comparison is supported.
However, the query also compares the process.parent.name
field value to the
process.name
field. This comparison is not supported and will return an
error for the entire query.
process where process.parent.name == "foo" and process.parent.name == process.name
Instead, you can rewrite the query to compare both the process.parent.name
and process.name
fields to static values.
process where process.parent.name == "foo" and process.name == "foo"
Avoid using the ==
operator to perform exact matching on text
field
values.
By default, Elasticsearch changes the values of text
fields as part of analysis. This can make finding exact matches for text
field values
difficult.
To search text
fields, consider using a query
DSL filter that contains a match
query.
Logical operators
editand or not
Definitions
-
and
-
Returns
true
only if the condition to the left and right both returntrue
. Otherwise returns `false. -
or
-
Returns
true
if one of the conditions to the left or righttrue
. Otherwise returns `false. -
not
-
Returns
true
if the condition to the right isfalse
.
Lookup operators
edituser.name in ("Administrator", "SYSTEM", "NETWORK SERVICE") user.name not in ("Administrator", "SYSTEM", "NETWORK SERVICE")
Definitions
-
in
-
Returns
true
if the value is contained in the provided list. -
not in
-
Returns
true
if the value is not contained in the provided list.
Math operators
edit+ - * / %
Definitions
-
+
(add) - Adds the values to the left and right of the operator.
-
-
(subtract) - Subtracts the value to the right of the operator from the value to the left.
-
*
(multiply) - Multiplies the values to the left and right of the operator.
-
/
(divide) - Divides the value to the left of the operator by the value to the right.
-
%
(modulo) - Divides the value to the left of the operator by the value to the right. Returns only the remainder.
If both the dividend and divisor are integers, the divide (\
) operation
rounds down any returned floating point numbers to the nearest integer.
EQL queries in Elasticsearch should account for this rounding. To avoid rounding, convert either the dividend or divisor to a float.
Example
The process.args_count
field is a long
integer field containing a
count of process arguments.
A user might expect the following EQL query to only match events with a
process.args_count
value of 4
.
process where ( 4 / process.args_count ) == 1
However, the EQL query matches events with a process.args_count
value of 3
or 4
.
For events with a process.args_count
value of 3
, the divide operation
returns a float of 1.333...
, which is rounded down to 1
.
To match only events with a process.args_count
value of 4
, convert
either the dividend or divisor to a float.
The following EQL query changes the integer 4
to the equivalent float 4.0
.
process where ( 4.0 / process.args_count ) == 1
Strings
editStrings are enclosed with double quotes ("
) or single quotes ('
).
"hello world" "hello world with 'substring'"
Wildcards
editWhen comparing strings using the ==
or !=
operators, you can use the *
operator within the string to match specific patterns:
field == "example*wildcard" field != "example*wildcard"
Match any condition
editTo match events solely on event category, use the where true
condition.
For example, the following EQL query matches any file
events:
file where true
To match any event, you can combine the any
keyword with the where true
condition:
any where true
Escape characters in a string
editWhen used within a string, special characters, such as a carriage return or
double quote ("
), must be escaped with a preceding backslash (\
).
"example \t of \n escaped \r characters"
Escape sequences
Escape sequence | Literal character |
---|---|
|
A newline (linefeed) character |
|
A carriage return character |
|
A tab character |
|
A backslash ( |
|
A double quote ( |
|
A single quote ( |
Raw strings
editRaw strings are preceded by a question mark (?
) and treat backslashes (\
) as
literal characters.
?"String with a literal 'blackslash' \ character included"
You can escape single quotes ('
) and double quotes ("
) with a backslash, but
the backslash remains in the resulting string.
?"\""
Raw strings cannot contain only a single backslash or end in an odd number of backslashes.
Sequences
editYou can use EQL sequences to describe and match an ordered series of events.
Each item in a sequence is an event category and event condition,
surrounded by square brackets ([ ]
). Events are listed in ascending
chronological order, with the most recent event listed last.
sequence [ event_category_1 where condition_1 ] [ event_category_2 where condition_2 ] ...
Example
The following EQL sequence query matches this series of ordered events:
-
Start with an event with:
-
An event category of
file
-
A
file.extension
ofexe
-
An event category of
-
Followed by an event with an event category of
process
sequence [ file where file.extension == "exe" ] [ process where true ]
with maxspan
keywords
editYou can use the with maxspan
keywords to constrain a sequence to a specified
timespan. All events in a matching sequence must occur within this duration,
starting at the first event’s timestamp.
The maxspan
keyword accepts time value arguments.
sequence with maxspan=30s [ event_category_1 where condition_1 ] by field_baz [ event_category_2 where condition_2 ] by field_bar ...
Example
The following sequence query uses a maxspan
value of 15m
(15 minutes).
Events in a matching sequence must occur within 15 minutes of the first event’s
timestamp.
sequence with maxspan=15m [ file where file.extension == "exe" ] [ process where true ]
by
keyword
editYou can use the by
keyword with sequences to only match events that share the
same field values. If a field value should be shared across all events, you
can use sequence by
.
sequence by field_foo [ event_category_1 where condition_1 ] by field_baz [ event_category_2 where condition_2 ] by field_bar ...
Example
The following sequence query uses the by
keyword to constrain matching events
to:
-
Events with the same
user.name
value -
file
events with afile.path
value equal to the followingprocess
event’sprocess.path
value.
sequence [ file where file.extension == "exe" ] by user.name, file.path [ process where true ] by user.name, process.path
Because the user.name
field is shared across all events in the sequence, it
can be included using sequence by
. The following sequence is equivalent to the
prior one.
sequence by user.name [ file where file.extension == "exe" ] by file.path [ process where true ] by process.path
You can combine the sequence by
and with maxspan
keywords to constrain a
sequence by both field values and a timespan.
sequence by field_foo with maxspan=30s [ event_category_1 where condition_1 ] by field_baz [ event_category_2 where condition_2 ] by field_bar ...
Example
The following sequence query uses the sequence by
keyword and with maxspan
keywords to match only a sequence of events that:
-
Share the same
user.name
field values -
Occur within
15m
(15 minutes) of the first matching event
sequence by user.name with maxspan=15m [ file where file.extension == "exe" ] by file.path [ process where true ] by process.path
until
keyword
editYou can use the until
keyword to specify an expiration event for a sequence.
If this expiration event occurs between matching events in a sequence, the
sequence expires and is not considered a match. If the expiration event occurs
after matching events in a sequence, the sequence is still considered a
match. The expiration event is not included in the results.
sequence [ event_category_1 where condition_1 ] [ event_category_2 where condition_2 ] ... until [ event_category_3 where condition_3 ]
Example
A dataset contains the following event sequences, grouped by shared IDs:
A, B A, B, C A, C, B
The following EQL query searches the dataset for sequences containing
event A
followed by event B
. Event C
is used as an expiration event.
sequence by ID A B until C
The query matches sequences A, B
and A, B, C
but not A, C, B
.
The until
keyword can be useful when searching for process sequences in
Windows event logs.
In Windows, a process ID (PID) is unique only while a process is running. After a process terminates, its PID can be reused.
You can search for a sequence of events with the same PID value using the by
and sequence by
keywords.
Example
The following EQL query uses the sequence by
keyword to match a
sequence of events that share the same process.pid
value.
sequence by process.pid [ process where event.type == "start" and process.name == "cmd.exe" ] [ process where file.extension == "exe" ]
However, due to PID reuse, this can result in a matching sequence that
contains events across unrelated processes. To prevent false positives, you can
use the until
keyword to end matching sequences before a process termination
event.
The following EQL query uses the until
keyword to end sequences before
process
events with an event.type
of stop
. These events indicate a process
has been terminated.
sequence by process.pid [ process where event.type == "start" and process.name == "cmd.exe" ] [ process where file.extension == "exe" ] until [ process where event.type == "stop" ]
Functions
editElasticsearch supports several of EQL’s built-in functions. You can use these functions to convert data types, perform math, manipulate strings, and more.
For a list of supported functions, see Function reference.
Using functions in EQL queries can result in slower search speeds. If you often use functions to transform indexed data, you can speed up search by making these changes during indexing instead. However, that often means slower index speeds.
Example
An index contains the file.path
field. file.path
contains the full path to a
file, including the file extension.
When running EQL searches, users often use the endsWith
function with the
file.path
field to match file extensions:
file where endsWith(file.path,".exe") or endsWith(file.path,".dll")
While this works, it can be repetitive to write and can slow search speeds. To speed up search, you can do the following instead:
-
Add a new field,
file.extension
, to the index. Thefile.extension
field will contain only the file extension from thefile.path
field. -
Use an ingest pipeline containing the
grok
processor or another preprocessor tool to extract the file extension from thefile.path
field before indexing. -
Index the extracted file extension to the
file.extension
field.
These changes may slow indexing but allow for faster searches. Users
can use the file.extension
field instead of multiple endsWith
function
calls:
file where file.extension in ("exe", "dll")
We recommend testing and benchmarking any indexing changes before deploying them in production. See Tune for indexing speed and Tune for search speed.
Pipes
editEQL pipes filter, aggregate, and post-process events returned by an EQL query. You can use pipes to narrow down EQL query results or make them more specific.
Pipes are delimited using the pipe (|
) character.
event_category where condition | pipe
Example
The following EQL query uses the tail
pipe to return only the 10 most recent
events matching the query.
authentication where agent.id == 4624 | tail 10
You can pass the output of a pipe to another pipe. This lets you use multiple pipes with a single query.
For a list of supported pipes, see Pipe reference.
Limitations
editElasticsearch EQL does not support the following features and syntax.
Comparing fields
editIn Elasticsearch EQL, you cannot use comparison operators to compare a field to another field. This applies even if the fields are changed using a function.
EQL search on nested fields
editYou cannot use EQL to search the values of a nested
field or the
sub-fields of a nested
field. However, data streams and indices containing
nested
field mappings are otherwise supported.
Unsupported syntax
editElasticsearch supports a subset of EQL syntax. Elasticsearch cannot run EQL queries that contain: