- Elasticsearch Guide: other versions:
- Getting Started
- Set up Elasticsearch
- Installing Elasticsearch
- Configuring Elasticsearch
- Important Elasticsearch configuration
- Important System Configuration
- Bootstrap Checks
- Heap size check
- File descriptor check
- Memory lock check
- Maximum number of threads check
- Max file size check
- Maximum size virtual memory check
- Maximum map count check
- Client JVM check
- Use serial collector check
- System call filter check
- OnError and OnOutOfMemoryError checks
- Early-access check
- G1GC check
- All permission check
- Starting Elasticsearch
- Stopping Elasticsearch
- Adding nodes to your cluster
- Installing X-Pack
- Set up X-Pack
- Configuring X-Pack Java Clients
- X-Pack Settings
- Bootstrap Checks for X-Pack
- Upgrade Elasticsearch
- API Conventions
- Document APIs
- Search APIs
- Aggregations
- Metrics Aggregations
- Avg Aggregation
- Weighted Avg Aggregation
- Cardinality Aggregation
- Extended Stats Aggregation
- Geo Bounds Aggregation
- Geo Centroid Aggregation
- Max Aggregation
- Min Aggregation
- Percentiles Aggregation
- Percentile Ranks Aggregation
- Scripted Metric Aggregation
- Stats Aggregation
- Sum Aggregation
- Top Hits Aggregation
- Value Count Aggregation
- Bucket Aggregations
- Adjacency Matrix Aggregation
- Auto-interval Date Histogram Aggregation
- Intervals
- Children Aggregation
- Composite Aggregation
- Date Histogram Aggregation
- Date Range Aggregation
- Diversified Sampler Aggregation
- Filter Aggregation
- Filters Aggregation
- Geo Distance Aggregation
- GeoHash grid Aggregation
- Global Aggregation
- Histogram Aggregation
- IP Range Aggregation
- Missing Aggregation
- Nested Aggregation
- Range Aggregation
- Reverse nested Aggregation
- Sampler Aggregation
- Significant Terms Aggregation
- Significant Text Aggregation
- Terms Aggregation
- Pipeline Aggregations
- Avg Bucket Aggregation
- Derivative Aggregation
- Max Bucket Aggregation
- Min Bucket Aggregation
- Sum Bucket Aggregation
- Stats Bucket Aggregation
- Extended Stats Bucket Aggregation
- Percentiles Bucket Aggregation
- Moving Average Aggregation
- Moving Function Aggregation
- Cumulative Sum Aggregation
- Bucket Script Aggregation
- Bucket Selector Aggregation
- Bucket Sort Aggregation
- Serial Differencing Aggregation
- Matrix Aggregations
- Caching heavy aggregations
- Returning only aggregation results
- Aggregation Metadata
- Returning the type of the aggregation
- Metrics Aggregations
- Indices APIs
- Create Index
- Delete Index
- Get Index
- Indices Exists
- Open / Close Index API
- Shrink Index
- Split Index
- Rollover Index
- Put Mapping
- Get Mapping
- Get Field Mapping
- Types Exists
- Index Aliases
- Update Indices Settings
- Get Settings
- Analyze
- Index Templates
- Indices Stats
- Indices Segments
- Indices Recovery
- Indices Shard Stores
- Clear Cache
- Flush
- Refresh
- Force Merge
- cat APIs
- Cluster APIs
- Query DSL
- Mapping
- Analysis
- Anatomy of an analyzer
- Testing analyzers
- Analyzers
- Normalizers
- Tokenizers
- Standard Tokenizer
- Letter Tokenizer
- Lowercase Tokenizer
- Whitespace Tokenizer
- UAX URL Email Tokenizer
- Classic Tokenizer
- Thai Tokenizer
- NGram Tokenizer
- Edge NGram Tokenizer
- Keyword Tokenizer
- Pattern Tokenizer
- Char Group Tokenizer
- Simple Pattern Tokenizer
- Simple Pattern Split Tokenizer
- Path Hierarchy Tokenizer
- Path Hierarchy Tokenizer Examples
- Token Filters
- Standard Token Filter
- ASCII Folding Token Filter
- Flatten Graph Token Filter
- Length Token Filter
- Lowercase Token Filter
- Uppercase Token Filter
- NGram Token Filter
- Edge NGram Token Filter
- Porter Stem Token Filter
- Shingle Token Filter
- Stop Token Filter
- Word Delimiter Token Filter
- Word Delimiter Graph Token Filter
- Multiplexer Token Filter
- Conditional Token Filter
- Predicate Token Filter Script
- Stemmer Token Filter
- Stemmer Override Token Filter
- Keyword Marker Token Filter
- Keyword Repeat Token Filter
- KStem Token Filter
- Snowball Token Filter
- Phonetic Token Filter
- Synonym Token Filter
- Synonym Graph Token Filter
- Compound Word Token Filters
- Reverse Token Filter
- Elision Token Filter
- Truncate Token Filter
- Unique Token Filter
- Pattern Capture Token Filter
- Pattern Replace Token Filter
- Trim Token Filter
- Limit Token Count Token Filter
- Hunspell Token Filter
- Common Grams Token Filter
- Normalization Token Filter
- CJK Width Token Filter
- CJK Bigram Token Filter
- Delimited Payload Token Filter
- Keep Words Token Filter
- Keep Types Token Filter
- Exclude mode settings example
- Classic Token Filter
- Apostrophe Token Filter
- Decimal Digit Token Filter
- Fingerprint Token Filter
- Minhash Token Filter
- Remove Duplicates Token Filter
- Character Filters
- Modules
- Index Modules
- Ingest Node
- Pipeline Definition
- Ingest APIs
- Accessing Data in Pipelines
- Conditional Execution in Pipelines
- Handling Failures in Pipelines
- Processors
- Append Processor
- Bytes Processor
- Convert Processor
- Date Processor
- Date Index Name Processor
- Dissect Processor
- Drop Processor
- Dot Expander Processor
- Fail Processor
- Foreach Processor
- Grok Processor
- Gsub Processor
- Join Processor
- JSON Processor
- KV Processor
- Lowercase Processor
- Pipeline Processor
- Remove Processor
- Rename Processor
- Script Processor
- Set Processor
- Set Security User Processor
- Split Processor
- Sort Processor
- Trim Processor
- Uppercase Processor
- URL Decode Processor
- SQL Access
- Monitor a cluster
- Rolling up historical data
- Set up a cluster for high availability
- Secure a cluster
- Overview
- Configuring security
- Encrypting communications in Elasticsearch
- Encrypting communications in an Elasticsearch Docker Container
- Enabling cipher suites for stronger encryption
- Separating node-to-node and client traffic
- Configuring an Active Directory realm
- Configuring a file realm
- Configuring an LDAP realm
- Configuring a native realm
- Configuring a PKI realm
- Configuring a SAML realm
- Configuring a Kerberos realm
- FIPS 140-2
- Security settings
- Security files
- Auditing settings
- How security works
- User authentication
- Built-in users
- Internal users
- Realms
- Realm chains
- Active Directory user authentication
- File-based user authentication
- LDAP user authentication
- Native user authentication
- PKI user authentication
- SAML authentication
- Kerberos authentication
- Integrating with other authentication systems
- Enabling anonymous access
- Controlling the user cache
- Configuring SAML single-sign-on on the Elastic Stack
- User authorization
- Auditing security events
- Encrypting communications
- Restricting connections with IP filtering
- Cross cluster search, tribe, clients, and integrations
- Tutorial: Getting started with security
- Tutorial: Encrypting communications
- Troubleshooting
- Can’t log in after upgrading to 6.5.4
- Some settings are not returned via the nodes settings API
- Authorization exceptions
- Users command fails due to extra arguments
- Users are frequently locked out of Active Directory
- Certificate verification fails for curl on Mac
- SSLHandshakeException causes connections to fail
- Common SSL/TLS exceptions
- Common Kerberos exceptions
- Common SAML issues
- Internal Server Error in Kibana
- Setup-passwords command fails due to connection failure
- Failures due to relocation of the configuration files
- Limitations
- Alerting on Cluster and Index Events
- Command line tools
- How To
- Testing
- Glossary of terms
- X-Pack APIs
- Info API
- Cross-cluster replication APIs
- Explore API
- Licensing APIs
- Migration APIs
- Machine learning APIs
- Add events to calendar
- Add jobs to calendar
- Close jobs
- Create calendar
- Create datafeeds
- Create filter
- Create jobs
- Delete calendar
- Delete datafeeds
- Delete events from calendar
- Delete filter
- Delete forecast
- Delete jobs
- Delete jobs from calendar
- Delete model snapshots
- Find file structure
- Flush jobs
- Forecast jobs
- Get calendars
- Get buckets
- Get overall buckets
- Get categories
- Get datafeeds
- Get datafeed statistics
- Get influencers
- Get jobs
- Get job statistics
- Get machine learning info
- Get model snapshots
- Get scheduled events
- Get filters
- Get records
- Open jobs
- Post data to jobs
- Preview datafeeds
- Revert model snapshots
- Start datafeeds
- Stop datafeeds
- Update datafeeds
- Update filter
- Update jobs
- Update model snapshots
- Rollup APIs
- Security APIs
- Authenticate
- Change passwords
- Clear cache
- Clear roles cache
- Create or update application privileges
- Create or update role mappings
- Create or update roles
- Create or update users
- Delete application privileges
- Delete role mappings
- Delete roles
- Delete users
- Disable users
- Enable users
- Get application privileges
- Get role mappings
- Get roles
- Get token
- Get users
- Has privileges
- Invalidate token
- SSL certificate
- Watcher APIs
- Definitions
- Release Highlights
- Breaking changes
- Release Notes
- Elasticsearch version 6.5.4
- Elasticsearch version 6.5.3
- Elasticsearch version 6.5.2
- Elasticsearch version 6.5.1
- Elasticsearch version 6.5.0
- Elasticsearch version 6.4.3
- Elasticsearch version 6.4.2
- Elasticsearch version 6.4.1
- Elasticsearch version 6.4.0
- Elasticsearch version 6.3.2
- Elasticsearch version 6.3.1
- Elasticsearch version 6.3.0
- Elasticsearch version 6.2.4
- Elasticsearch version 6.2.3
- Elasticsearch version 6.2.2
- Elasticsearch version 6.2.1
- Elasticsearch version 6.2.0
- Elasticsearch version 6.1.4
- Elasticsearch version 6.1.3
- Elasticsearch version 6.1.2
- Elasticsearch version 6.1.1
- Elasticsearch version 6.1.0
- Elasticsearch version 6.0.1
- Elasticsearch version 6.0.0
- Elasticsearch version 6.0.0-rc2
- Elasticsearch version 6.0.0-rc1
- Elasticsearch version 6.0.0-beta2
- Elasticsearch version 6.0.0-beta1
- Elasticsearch version 6.0.0-alpha2
- Elasticsearch version 6.0.0-alpha1
- Elasticsearch version 6.0.0-alpha1 (Changes previously released in 5.x)
Audit event types
editAudit event types
editWhen you are auditing security events, each request can generate multiple audit events.
The following is a list of the events that can be generated:
|
Logged when a request is denied due to a missing authentication token. |
||
|
Logged when a user successfully authenticates. |
||
|
Logged when the authentication token cannot be matched to a known user. |
||
|
Logged for every realm that fails to present a valid
authentication token. |
||
|
Logged when an authenticated user attempts to execute an action they do not have the necessary privilege to perform. |
||
|
Logged when an authenticated user attempts to execute
an action they have the necessary privilege to perform.
When the |
||
|
Logged when an authenticated user attempts to run as another user that they have the necessary privileges to do. |
||
|
Logged when an authenticated user attempts to run as another user action they do not have the necessary privilege to do so. |
||
|
Logged when the security features detect that the request has
been tampered with. Typically relates to |
||
|
Logged when an incoming TCP connection passes the IP Filter for a specific profile. |
||
|
Logged when an incoming TCP connection does not pass the IP Filter for a specific profile. |
Audit event attributes
editIn 6.5.0, there is a new logfile
audit output format.
This format also brings in a few changes for audit event attributes.
The new format is output to the <clustername>_audit.log
file.
The audit entries are formatted as flat JSON documents (that is to say, no
nested objects), one per line. Hence, the attribute names are JSON keys and they
follow a dotted name syntax. Any attributes that lack a value (null
) are not
output.
The following list shows attributes that are common to all audit events.
Their names and values are analogous to those in the deprecated logfile
or
index
output formats. However, it is expected that the formats will evolve
independently during the 6.x releases, so it is advisable to follow the attribute
descriptions for the format that you are using.
-
@timestamp
- The time, in ISO9601 format, when the event occurred.
-
node.name
-
The name of the node. This can be changed
in the
elasticsearch.yml
config file. -
node.id
- The node id. This is automatically generated and is persistent across full cluster restarts.
-
host.ip
- The bound IP address of the node, with which the node can be communicated with.
-
host.name
- The unresolved node’s hostname.
-
origin.address
- The source IP address of the request associated with this event. This could be the address of the remote client, the address of another cluster node, or the local node’s bound address, if the request originated locally. Unless the remote client connects directly to the cluster, the client address will actually be the address of the first OSI layer 3 proxy in front of the cluster.
-
origin.type
-
The origin type of the request associated with this event:
rest
(request originated from a REST API request),transport
(request was received on the transport channel), orlocal_node
(the local node issued the request). -
event.type
-
The internal processing layer that generated the event:
rest
,transport
orip_filter
. This is different fromorigin.type
because a request originating from the REST API is translated to a number of transport messages, generating audit events withorigin.type: rest
andevent.type: transport
. -
event.action
-
The type of event that occurred:
anonymous_access_denied
,authentication_failed
,authentication_success
,realm_authentication_failed
,access_denied
,access_granted
,connection_denied
,connection_granted
,tampered_request
,run_as_denied
, orrun_as_granted
. -
opaque_id
-
The value of the
X-Opaque-Id
HTTP header (if present) of the request associated with this event. This header can be used freely by the client to mark API calls, as it has no semantics in Elasticsearch.
Audit event attributes of the REST event type
editThe events with event.type
equal to rest
have one of the following event.action
attribute values: authentication_success
, anonymous_access_denied
, authentication_failed
,
realm_authentication_failed
, tampered_request
or run_as_denied
.
These event types also have the following extra attributes (in addition to the
common ones):
-
url.path
- The path part of the URL (between the port and the query string) of the REST request associated with this event. This is URL encoded.
-
url.query
- The query part of the URL (after "?", if present) of the REST request associated with this event. This is URL encoded.
-
request.body
- The full content of the REST request associated with this event, if enabled. This contains the query body. The body is escaped according to the JSON RFC 4627.
Audit event attributes of the transport event type
editThe events with event.type
equal to transport
have one of the following event.action
attribute values: authentication_success
, anonymous_access_denied
, authentication_failed
,
realm_authentication_failed
, access_granted
, access_denied
, run_as_granted
,
run_as_denied
, or tampered_request
.
These event types also have the following extra attributes (in addition to the common
ones):
-
action
- The name of the transport action that was executed. This is like the URL for a REST request.
-
indices
- The indices names array that the request associated with this event pertains to (when applicable).
-
request.name
- The name of the request handler that was executed.
Audit event attributes of the ip_filter event type
editThe events with event.type
equal to ip_filter
have one of the following event.action
attribute values: connection_granted
or connection_denied
.
These event types also have the following extra attributes (in addition to the common
ones):
-
transport_profile
- The transport profile the request targeted.
-
rule
- The IP filtering rule that denied the request.
Extra audit event attributes for specific events
editThere are a few events that have some more attributes in addition to those that have been previously described:
-
authentication_success
:-
realm
- The name of the realm that successfully authenticated the user.
-
user.name
- The name of the effective user. This is usually the same as the authenticated user, but if using the run as authorization functionality this instead denotes the name of the impersonated user.
-
user.run_by.name
- This attribute is present only if the request is using the run as authorization functionality and denotes the name of the authenticated user, which is also known as the impersonator.
-
-
authentication_failed
:-
user.name
- The name of the user that failed authentication. If the request authentication token is invalid or unparsable, this information might be missing.
-
-
realm_authentication_failed
:-
user.name
- The name of the user that failed authentication.
-
realm
- The name of the realm that rejected this authentication. This event is generated for each consulted realm in the chain.
-
-
run_as_denied
andrun_as_granted
:-
user.roles
- The role names of the user as an array.
-
user.name
- The name of the authenticated user which is being granted or denied the impersonation action.
-
user.realm
- The realm name that the authenticated user belongs to.
-
user.run_as.name
- The name of the user as which the impersonation action is granted or denied.
-
user.run_as.realm
- The realm name of that the impersonated user belongs to.
-
-
access_granted
oraccess_denied
:-
user.roles
- The role names of the user as an array.
-
user.name
- The name of the effective user that is being authorized or unauthorized. This is usually the authenticated user, but if using the run as authorization functionality this instead denotes the name of the impersonated user.
-
user.realm
- The realm name that the effective user belongs to.
-
user.run_by.name
- This attribute is present only if the request is using the run as authorization functionality and denoted the name of the authenticated user, which is also known as the impersonator.
-
user.run_by.realm
- This attribute is present only if the request is using the run as authorization functionality and denotes the name of the realm that the authenticated (impersonator) user belongs to.
-
Audit event attributes for the deprecated formats
editThe following table shows the common attributes that can be associated with
every event, when it is output to the <clustername>_access.log
file or to the
index.
Table 60. Common attributes
Attribute | Description |
---|---|
|
When the event occurred. |
|
The name of the node. |
|
The hostname of the node. |
|
The IP address of the node. |
|
The layer from which this event originated: |
|
The type of event that occurred: |
For an event in the audit log file output, these are positional attributes, which are printed at the beginning of each log line and are not adjoined by the attribute name. As a matter of course, the names are present for each attribute when the event is forwarded to the index audit output.
The attribute origin_address
is also common to every audit event. It is always
named, that is, it is not positional. It denotes the source IP address of the
request associated with this event. This might be the address of the client, the
address of another cluster node, or the local node’s bound address (if the request
originated locally). Unless the client connects directly to the cluster, the
client address is the address of the first OSI layer 3 proxy in front of the
cluster.
In addition, every event might have the opaque_id
attribute, with the value as
it has been passed in by the X-Opaque-Id
HTTP request header. This header can
be used freely by the client to mark API calls, as it has no semantics in
Elasticsearch. Every audit event, generated as part of handling a request thus
marked, contains the opaque_id
attribute.
The following tables show the attributes that can are associated with each type of event, in addition to the common ones previously described:
Table 61. REST anonymous_access_denied attributes
Attribute | Description |
---|---|
|
The REST endpoint URI. |
|
The body of the request, if enabled. |
Table 62. REST authentication_success attributes
Attribute | Description |
---|---|
|
The effective (impersonated) username. Usually this is the same as the authenticated username. |
|
The authenticated (impersonator) username.
This attribute is present only if the request is
using the run as authorization functionality.
Otherwise, the effective user is the same as the
authenticated one, which is indicated by the |
|
The realm that authenticated the user. |
|
The REST endpoint URI. |
|
The REST URI query parameters. |
|
The body of the request, if enabled. |
Table 63. REST authentication_failed attributes
Attribute | Description |
---|---|
|
The principal (username) that failed authentication. If the request’s authentication token is invalid, this information might be missing. |
|
The REST endpoint URI. |
|
The body of the request, if enabled. |
Table 64. REST realm_authentication_failed attributes
Attribute | Description |
---|---|
|
The realm that failed to authenticate the user. A separate entry is logged for each consulted realm. |
|
The principal (username) that failed authentication. |
|
The REST endpoint URI. |
|
The body of the request, if enabled. |
Table 65. REST tampered_request attributes
Attribute | Description |
---|---|
|
The REST endpoint URI. |
|
The body of the request, if enabled. |
Table 66. Transport anonymous_access_denied attributes
Attribute | Description |
---|---|
|
Where the request originated: |
|
The name of the action that was executed. |
|
A comma-separated list of indices this request pertains to (when applicable). |
|
The type of request that was executed. |
Table 67. Transport authentication_success attributes
Attribute | Description |
---|---|
|
Where the request originated: |
|
The effective (impersonated) username. Usually this is the same as the authenticated username. |
|
The authenticated (impersonator) username.
This attribute is present only if the request is
using the run as authorization functionality.
Otherwise, the effective and the authenticated
users are equivalent and are indicated by the
|
|
The realm that authenticated the user. |
|
The name of the action that was executed. |
|
A comma-separated list of indices this request pertains to (when applicable). |
|
The type of request that was executed. |
Table 68. Transport authentication_failed attributes
Attribute | Description |
---|---|
|
Where the request originated: |
|
The effective (impersonated) username. Usually this is the same as the authenticated username. If the request’s authentication token is invalid, this information might be missing. |
|
The authenticated (impersonator) username.
This attribute is present only if the request is
using the run as authorization functionality.
Otherwise, the effective and the authenticated
users are equivalent and are indicated by the
|
|
The name of the action that was executed. |
|
A comma-separated list of indices this request pertains to (when applicable). |
|
The type of request that was executed. |
Table 69. Transport realm_authentication_failed attributes
Attribute | Description |
---|---|
|
The realm that failed to authenticate the user. A separate entry is logged for each consulted realm. |
|
Where the request originated: |
|
The principal (username) that failed authentication. |
|
The name of the action that was executed. |
|
A comma-separated list of indices this request pertains to (when applicable). |
|
The type of request that was executed. |
Table 70. Transport access_granted attributes
Attribute | Description |
---|---|
|
Where the request originated: |
|
The effective (impersonated) username for which authorization succeeded. Unless the request is using the run as authorization functionality, the effective and authenticated usernames are equivalent. |
|
The realm name that |
|
The authenticated (impersonator) username.
This attribute is present only if the request is
using the run as authorization functionality.
Otherwise, the effective and the authenticated
usernames are equivalent and are indicated by the
|
|
The realm name that |
|
The set of roles granting permissions. |
|
The name of the action that was executed. |
|
A comma-separated list of indices this request pertains to (when applicable). |
|
The type of request that was executed. |
Table 71. Transport access_denied attributes
Attribute | Description |
---|---|
|
Where the request originated: |
|
The effective (impersonated) username for which authorization failed. Unless the request is using the run as authorization functionality, the effective and the authenticated usernames are equivalent. |
|
The realm name that |
|
The authenticated (impersonator) username.
This attribute is present only if the request is
using the run as authorization functionality.
Otherwise, the effective and the authenticated
usernames are equivalent and are indicated by the
|
|
The realm name that |
|
The set of roles granting permissions. |
|
The name of the action that was executed. |
|
A comma-separated list of indices this request relates to (when applicable). |
|
The type of request that was executed. |
Table 72. Transport run_as_granted attributes
Attribute | Description |
---|---|
|
Where the request originated: |
|
The authenticated (impersonator) username for which the impersonation operation was granted. |
|
The realm name that the authenticated user belongs to. |
|
The impersonated username. |
|
The realm name that the impersonated username belongs to. |
|
The set of roles granting permissions. |
|
The name of the action that was executed. |
|
A comma-separated list of indices this request relates to (when applicable). |
|
The type of request that was executed. |
Table 73. Transport run_as_denied attributes
Attribute | Description |
---|---|
|
Where the request originated: |
|
The authenticated (impersonator) username for which the impersonation operation was denied. |
|
The realm name that the authenticated user belongs to. |
|
The impersonated username. |
|
The realm name that the impersonated username belongs to. |
|
The set of roles granting permissions. |
|
The name of the action that was executed. |
|
A comma-separated list of indices this request relates to (when applicable). |
|
The type of request that was executed. |
Table 74. Transport tampered_request attributes
Attribute | Description |
---|---|
|
Where the request originated: |
|
The effective (impersonated) username. Unless the request is using the run as authorization functionality, the effective and the authenticated usernames are equivalent. If the requests’s authentication token is invalid, this information might be missing. |
|
The authenticated (impersonator) username.
This attribute is present only if the request is
using the run as authorization functionality.
Otherwise, the effective and the authenticated usernames
are equivalent and are indicated by the |
|
The name of the action that was executed. |
|
A comma-separated list of indices this request pertains to (when applicable). |
|
The type of request that was executed. |
Table 75. IP filter connection_granted attributes
Attribute | Description |
---|---|
|
The transport profile the request targeted. |
|
The IP filtering rule that granted the request. |
Table 76. IP filter connection_denied attributes
Attribute | Description |
---|---|
|
The transport profile the request targeted. |
|
The IP filtering rule that denied the request. |
On this page