EQL search API
editEQL search API
editThis functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.
Returns search results for an Event Query Language (EQL) query.
In Elasticsearch, EQL assumes each document in a data stream or index corresponds to an event.
GET /my-index-000001/_eql/search { "query": """ process where process.name == "regsvr32.exe" """ }
Path parameters
edit-
<target>
-
(Required, string) Comma-separated list of data streams, indices, or index aliases used to limit the request. Accepts wildcard (
*
) expressions.To search all data streams and indices in a cluster, use
_all
or*
.
Query parameters
edit-
allow_no_indices
-
(Optional, Boolean) If
false
, the request returns an error when a wildcard expression, index alias, or_all
value targets only missing or closed indices.Defaults to
false
. -
expand_wildcards
-
(Optional, string) Controls what kind of indices that wildcard expressions can expand to. Multiple values are accepted when separated by a comma, as in
open,hidden
. Valid values are:-
all
- Expand to open and closed indices, including hidden indices.
-
open
- Expand only to open indices.
-
closed
- Expand only to closed indices.
-
hidden
-
Expansion of wildcards will include hidden indices.
Must be combined with
open
,closed
, or both. -
none
- Wildcard expressions are not accepted.
Defaults to
open
. -
-
ignore_unavailable
-
(Optional, Boolean) If
false
, the request returns an error if it targets a missing or closed index. Defaults tofalse
. -
keep_alive
-
(Optional, time value) Period for which the search and its results are stored on the cluster. Defaults to
5d
(five days).When this period expires, the search and its results are deleted, even if the search is still ongoing.
If the
keep_on_completion
parameter isfalse
, Elasticsearch only stores async searches that do not complete within the period set by thewait_for_completion_timeout
parameter, regardless of this value.You can also specify this value using the
keep_alive
request body parameter. If both parameters are specified, only the query parameter is used. -
keep_on_completion
-
(Optional, Boolean) If
true
, the search and its results are stored on the cluster.If
false
, the search and its results are stored on the cluster only if the request does not complete during the period set by thewait_for_completion_timeout
parameter. Defaults tofalse
.You can also specify this value using the
keep_on_completion
request body parameter. If both parameters are specified, only the query parameter is used. -
wait_for_completion_timeout
-
(Optional, time value) Timeout duration to wait for the request to finish. Defaults to no timeout, meaning the request waits for complete search results.
If this parameter is specified and the request completes during this period, complete search results are returned.
If the request does not complete during this period, the search becomes an async search.
You can also specify this value using the
wait_for_completion_timeout
request body parameter. If both parameters are specified, only the query parameter is used.
Request body
edit-
case_sensitive
-
(Optional, Boolean)
If
true
, matching for the EQL query is case sensitive. Defaults tofalse
. -
event_category_field
-
(Required*, string) Field containing the event classification, such as
process
,file
, ornetwork
.Defaults to
event.category
, as defined in the Elastic Common Schema (ECS). If a data stream or index does not contain theevent.category
field, this value is required.The event category field is typically mapped as a field type in the
keyword
family. -
fetch_size
-
(Optional, integer) Maximum number of events to search at a time for sequence queries. Defaults to
1000
.This value must be greater than
2
but cannot exceed the value of theindex.max_result_window
setting, which defaults to10000
.Internally, a sequence query fetches and paginates sets of events to search for matches. This parameter controls the size of those sets. This parameter does not limit the total number of events searched or the number of matching events returned.
A greater
fetch_size
value often increases search speed but uses more memory. -
filter
- (Optional, query DSL object) Query, written in query DSL, used to filter the events on which the EQL query runs.
-
keep_alive
-
(Optional, time value) Period for which the search and its results are stored on the cluster. Defaults to
5d
(five days).When this period expires, the search and its results are deleted, even if the search is still ongoing.
If the
keep_on_completion
parameter isfalse
, Elasticsearch only stores async searches that do not complete within the period set by thewait_for_completion_timeout
parameter, regardless of this value.You can also specify this value using the
keep_alive
query parameter. If both parameters are specified, only the query parameter is used.
-
keep_on_completion
-
(Optional, Boolean) If
true
, the search and its results are stored on the cluster.If
false
, the search and its results are stored on the cluster only if the request does not complete during the period set by thewait_for_completion_timeout
parameter. Defaults tofalse
.You can also specify this value using the
keep_on_completion
query parameter. If both parameters are specified, only the query parameter is used.
-
query
-
(Required, string) EQL query you wish to run.
This parameter supports a subset of EQL syntax. See Unsupported syntax.
-
size
-
(Optional, integer or float) For basic queries, the maximum number of matching events to return.
For sequence queries, the maximum number of matching sequences to return.
Defaults to
10
. This value must be greater than0
.You cannot use pipes, such as
head
ortail
, to exceed this value.
-
tiebreaker_field
- (Optional, string) Field used to sort hits with the same timestamp in ascending, lexicographic order. See Specify a sort tiebreaker.
-
timestamp_field
-
(Required*, string) Field containing event timestamp.
Defaults to
@timestamp
, as defined in the Elastic Common Schema (ECS). If a data stream or index does not contain the@timestamp
field, this value is required.Events in the API response are sorted by this field’s value, converted to milliseconds since the Unix epoch, in ascending order.
The timestamp field should be mapped as a
date
. Thedate_nanos
field type is not supported.
-
wait_for_completion_timeout
-
(Optional, time value) Timeout duration to wait for the request to finish. Defaults to no timeout, meaning the request waits for complete search results.
If this parameter is specified and the request completes during this period, complete search results are returned.
If the request does not complete during this period, the search becomes an async search.
You can also specify this value using the
wait_for_completion_timeout
query parameter. If both parameters are specified, only the query parameter is used.
Response body
edit-
id
-
(string) Identifier for the search.
This search ID is only provided if one of the following conditions is met:
-
A search request does not return complete results during the
wait_for_completion_timeout
parameter’s timeout period, becoming an async search. -
The search request’s
keep_on_completion
parameter istrue
.
You can use this ID with the get async EQL search API to get the current status and available results for the search.
-
A search request does not return complete results during the
-
is_partial
-
(Boolean)
If
true
, the response does not contain complete search results. -
is_running
-
(Boolean) If
true
, the search request is still executing.If this parameter and the
is_partial
parameter aretrue
, the search is an ongoing async search. If thekeep_alive
period does not pass, the complete search results will be available when the search completes.If
is_partial
istrue
butis_running
isfalse
, the search returned partial results due to a failure. Only some shards returned results or the node coordinating the search failed. -
took
-
(integer) Milliseconds it took Elasticsearch to execute the request.
This value is calculated by measuring the time elapsed between receipt of a request on the coordinating node and the time at which the coordinating node is ready to send the response.
Took time includes:
- Communication time between the coordinating node and data nodes
-
Time the request spends in the
search
thread pool, queued for execution - Actual execution time
Took time does not include:
- Time needed to send the request to Elasticsearch
- Time needed to serialize the JSON response
- Time needed to send the response to a client
-
timed_out
-
(Boolean)
If
true
, the request timed out before completion. -
hits
-
(object) Contains matching events and sequences. Also contains related metadata.
Properties of
hits
-
total
-
(object) Metadata about the number of matching events or sequences.
Properties of
total
-
value
-
(integer) For basic queries, the total number of matching events.
For sequence queries, the total number of matching sequences.
-
relation
-
(string) Indicates whether the number of events or sequences returned is accurate or a lower bound.
Returned values are:
-
eq
- Accurate
-
gte
- Lower bound, including returned events or sequences
-
-
-
sequences
-
(array of objects) Contains event sequences matching the query. Each object represents a matching sequence. This parameter is only returned for EQL queries containing a sequence.
Properties of
sequences
objects-
join_keys
-
(array of values)
Shared field values used to constrain matches in the sequence. These are defined
using the
by
keyword in the EQL query syntax. -
events
-
(array of objects) Contains events matching the query. Each object represents a matching event.
Properties of
events
objects-
_index
- (string) Name of the index containing the event.
-
_id
- (string) Unique identifier for the event. This ID is only unique within the index.
-
_version
- (integer) Version of the document (event). This version is incremented each time the document is updated.
-
_seq_no
-
(integer) Sequence number assigned to the document (event).
Sequence numbers are used to ensure an older version of a document doesn’t overwrite a newer version. See Optimistic concurrency control.
-
_primary_term
- (integer) Primary term assigned to the document. See Optimistic concurrency control.
-
_score
- (float) Positive 32-bit floating point number used to determine the relevance of the event. See Relevance scores.
-
_source
- (object) Original JSON body passed for the event at index time.
-
-
-
events
-
(array of objects) Contains events matching the query. Each object represents a matching event.
Properties of
events
objects-
_index
- (string) Name of the index containing the event.
-
_id
- (string) (string) Unique identifier for the event. This ID is only unique within the index.
-
_score
- (float) Positive 32-bit floating point number used to determine the relevance of the event. See Relevance scores.
-
_source
- (object) Original JSON body passed for the event at index time.
-
-
Examples
editBasic query example
editThe following EQL search request searches for events with an event.category
of
process
that meet the following conditions:
-
A
process.name
ofcmd.exe
-
An
process.pid
other than2013
GET /my-index-000001/_eql/search { "query": """ process where (process.name == "cmd.exe" and process.pid != 2013) """ }
The API returns the following response. Matching events in the hits.events
property are sorted by timestamp, converted
to milliseconds since the Unix epoch,
in ascending order.
If two or more events share the same timestamp, the
tiebreaker_field
field is used to sort
the events in ascending, lexicographic order.
{ "is_partial": false, "is_running": false, "took": 6, "timed_out": false, "hits": { "total": { "value": 2, "relation": "eq" }, "events": [ { "_index": "my-index-000001", "_type": "_doc", "_id": "babI3XMBI9IjHuIqU0S_", "_score": null, "_source": { "@timestamp": "2099-12-06T11:04:05.000Z", "event": { "category": "process", "id": "edwCRnyD", "sequence": 1 }, "process": { "pid": 2012, "name": "cmd.exe", "executable": "C:\\Windows\\System32\\cmd.exe" } } }, { "_index": "my-index-000001", "_type": "_doc", "_id": "b6bI3XMBI9IjHuIqU0S_", "_score": null, "_source": { "@timestamp": "2099-12-07T11:06:07.000Z", "event": { "category": "process", "id": "cMyt5SZ2", "sequence": 3 }, "process": { "pid": 2012, "name": "cmd.exe", "executable": "C:\\Windows\\System32\\cmd.exe" } } } ] } }
Sequence query example
editThe following EQL search request matches a sequence of events that:
-
Start with an event with:
-
An
event.category
offile
-
A
file.name
ofcmd.exe
-
An
process.pid
other than2013
-
An
-
Followed by an event with:
-
An
event.category
ofprocess
-
A
process.executable
that contains the substringregsvr32
-
An
These events must also share the same process.pid
value.
GET /my-index-000001/_eql/search { "query": """ sequence by process.pid [ file where file.name == "cmd.exe" and process.pid != 2013 ] [ process where stringContains(process.executable, "regsvr32") ] """ }
The API returns the following response. Matching sequences are included in the
hits.sequences
property. The hits.sequences.join_keys
property contains the
shared process.pid
value for each matching event.
{ "is_partial": false, "is_running": false, "took": 6, "timed_out": false, "hits": { "total": { "value": 1, "relation": "eq" }, "sequences": [ { "join_keys": [ 2012 ], "events": [ { "_index": "my-index-000001", "_type": "_doc", "_id": "AtOJ4UjUBAAx3XR5kcCM", "_version": 1, "_seq_no": 1, "_primary_term": 1, "_score": null, "_source": { "@timestamp": "2099-12-06T11:04:07.000Z", "event": { "category": "file", "id": "dGCHwoeS", "sequence": 2 }, "file": { "accessed": "2099-12-07T11:07:08.000Z", "name": "cmd.exe", "path": "C:\\Windows\\System32\\cmd.exe", "type": "file", "size": 16384 }, "process": { "pid": 2012, "name": "cmd.exe", "executable": "C:\\Windows\\System32\\cmd.exe" } } }, { "_index": "my-index-000001", "_type": "_doc", "_id": "OQmfCaduce8zoHT93o4H", "_version": 1, "_seq_no": 3, "_primary_term": 1, "_score": null, "_source": { "@timestamp": "2099-12-07T11:07:09.000Z", "event": { "category": "process", "id": "aR3NWVOs", "sequence": 4 }, "process": { "pid": 2012, "name": "regsvr32.exe", "command_line": "regsvr32.exe /s /u /i:https://...RegSvr32.sct scrobj.dll", "executable": "C:\\Windows\\System32\\regsvr32.exe" } } } ] } ] } }