Security anomaly detection configurations
editSecurity anomaly detection configurations
editThese anomaly detection jobs appear by default in the Anomaly Detection interface of the Elastic Security app in Kibana. They help you automatically detect file system and network anomalies on your hosts. However, if you don’t use Beats, you need to map your data to the ECS fields that are listed for every job.
For more details, see the
datafeed and job definitions in the siem_*
folders in
GitHub.
Security Auditbeat
editDetect suspicious network activity and unusual processes in Auditbeat data.
- rare_process_by_host_linux_ecs
-
Identifies rare processes that do not usually run on individual hosts, which can indicate execution of unauthorized services, malware, or persistence mechanisms.
Processes are considered rare when they only run occasionally as compared with other processes running on the host.
- Job details
-
-
Analyzes host activity logs where
agent.type
isauditbeat
(Linux). - Models occurrences of process activities on the host.
-
Detects unusually rare processes compared to other processes on the host (using the
rare
function).
-
Analyzes host activity logs where
- Required Beats
-
- Auditbeat
- Required ECS fields when not using Beats
-
-
host.name
-
process.name
-
user.name
-
event.action
-
agent.type
-
- linux_anomalous_network_activity_ecs
-
Identifies OS processes that do not usually use the network but have unexpected network activity, which can indicate command-and-control, lateral movement, persistence, or data exfiltration activity.
A process with unusual network activity can denote process exploitation or injection, where the process is used to run persistence mechanisms that allow a malicious actor remote access or control of the host, data exfiltration, and execution of unauthorized network applications.
- Job details
-
-
Analyzes network activity logs where
agent.type
isauditbeat
. - Models the occurrences of processes that cause network activity.
-
Detects network activity caused by processes that occur rarely compared to
other processes (using the
rare
function).
-
Analyzes network activity logs where
- Required Beats
-
- Auditbeat
- Required ECS fields when not using Beats
-
-
destination.ip
-
host.name
-
process.name
-
user.name
-
event.action
-
agent.type
-
- linux_anomalous_network_port_activity_ecs
-
Identifies unusual destination port activity that can indicate command-and-control, persistence mechanism, or data exfiltration activity.
Rarely used destination port activity is generally unusual in Linux fleets, and can indicate unauthorized access or threat actor activity.
- Job details
-
-
Analyzes network activity logs where
agent.type
isauditbeat
. - Models destination port activity.
-
Detects destination port activity that occurs rarely compared to other port
activities (using the
rare
function).
-
Analyzes network activity logs where
- Required Beats
-
- Auditbeat (Linux)
This job is available only when you use Auditbeat to ship data.
- linux_anomalous_network_service
-
Searches for unusual listening ports that can indicate execution of unauthorized services, backdoors, or persistence mechanisms.
- Job details
-
-
Analyzes network activity logs where
agent.type
isauditbeat
. - Models listening port activity.
-
Detects listening port activity that occurs rarely compared to
other port activities (using the
rare
function).
-
Analyzes network activity logs where
- Required Beats
-
- Auditbeat (Linux)
This job is available only when you use Auditbeat to ship data.
- linux_anomalous_network_url_activity_ecs
-
Searches for unusual web URL requests from hosts, which can indicate malware delivery and execution.
Wget and cURL are commonly used by Linux programs to download code and data. Most of the time, their usage is entirely normal. Generally, because they use a list of URLs, they repeatedly download from the same locations. However, Wget and cURL are sometimes used to deliver Linux exploit payloads, and threat actors use these tools to download additional software and code. For these reasons, unusual URLs can indicate unauthorized downloads or threat activity.
- Job details
-
-
Analyzes network activity logs where
agent.type
isauditbeat
. - Models the occurrences of URL requests.
-
Detects a web URL request that is rare compared to other web URL
requests (using the
rare
function).
-
Analyzes network activity logs where
- Required Beats
-
- Auditbeat (Linux)
- Required ECS fields when not using Beats
-
-
destination.ip
-
destination.port
-
host.name
-
process.name
-
process.title
-
agent.type
-
- linux_anomalous_process_all_hosts_ecs
-
Searches for rare processes running on multiple hosts in an entire fleet or network.
This reduces the detection of false positives since automated maintenance processes usually only run occasionally on a single machine but are common to all or many hosts in a fleet.
- Job details
-
-
Analyzes host activity logs where
agent.type
isauditbeat
. - Models the occurrences of processes on all hosts.
-
Detects processes that occur rarely compared to other processes on all
hosts (using the
rare
function).
-
Analyzes host activity logs where
- Required Beats
-
- Auditbeat
- Required ECS fields when not using Beats
-
-
host.name
-
process.name
-
user.name
-
process.executable
-
event.action
-
agent.type
-
- linux_anomalous_user_name_ecs
-
Searches for activity from users who are not normally active, which can indicate unauthorized changes, activity by unauthorized users, lateral movement, and compromised credentials.
In organizations, new usernames are not often created apart from specific types of system activities, such as creating new accounts for new employees. These user accounts quickly become active and routine.
Events from rarely used usernames can point to suspicious activity. Additionally, automated Linux fleets tend to see activity from rarely used usernames only when personnel log in to make authorized or unauthorized changes, or threat actors have acquired credentials and log in for malicious purposes. Unusual usernames can also indicate pivoting, where compromised credentials are used to try and move laterally from one host to another.
- Job details
-
-
Analyzes host activity logs where
agent.type
isauditbeat
. - Models user activity.
-
Detects users that are rarely or unusually active compared to other users
(using the
rare
function).
-
Analyzes host activity logs where
- Required Beats
-
- Auditbeat
- Required ECS fields when not using Beats
-
-
host.name
-
process.name
-
user.name
-
event.action
-
agent.type
-
Security Auditbeat authentication
editDetect suspicious authentication events in Auditbeat data.
- suspicious_login_activity_ecs
-
Identifies an unusually high number of authentication attempts.
- Job details
-
-
Analyzes host activity logs where
agent.type
isauditbeat
. -
Models occurrences of authentication attempts (
partition_field_name
ishost.name
). -
Detects unusually high number of authentication attempts (using the
high_non_zero_count
function).
-
Analyzes host activity logs where
- Required Beats
-
- Auditbeat (Linux)
- Required ECS fields when not using Beats
-
-
source.ip
-
host.name
-
user.name
-
event.category
-
agent.type
-
Security CloudTrail
editDetect suspicious activity recorded in your CloudTrail logs.
- high_distinct_count_error_message
-
Looks for a spike in the rate of an error message. These spikes might simply indicate an impending service failure but they can also be byproducts of attempted or successful persistence, privilege escalation, defense evasion, discovery, lateral movement, or collection activity by a threat actor.
- Job details
-
-
Detects anomalies where the number of distinct values in
the
aws.cloudtrail.error_message
field is unusual (using thehigh_distinct_count
function).
-
Detects anomalies where the number of distinct values in
the
- Required Beats
-
- Filebeat
- Required ECS fields when not using Beats
-
-
source.geo.city_name
-
source.ip
-
- rare_error_code
-
Looks for unusual errors. Rare and unusual errors might simply indicate an impending service failure but they can also be byproducts of attempted or successful persistence, privilege escalation, defence evasion, discovery, lateral movement, or collection activity by a threat actor.
- Job details
-
-
Detects
aws.cloudtrail.error_code
values that have never or rarely occurred before (using therare
function).
-
Detects
- Required Beats
-
- Filebeat
- Required ECS fields when not using Beats
-
-
source.geo.city_name
-
source.ip
-
- rare_method_for_a_city
-
Looks for AWS API calls that—while not inherently suspicious or abnormal—are sourcing from a geolocation (city) that is unusual. These calls can be the result of compromised credentials or keys.
- Job details
-
-
Detects unusually rare
event.action
values compared to other cities (using therare
function).
-
Detects unusually rare
- Required Beats
-
- Filebeat
- Required ECS fields when not using Beats
-
-
event.action
-
source.geo.city_name
-
source.ip
-
- rare_method_for_a_country
-
Looks for AWS API calls that—while not inherently suspicious or abnormal—are sourcing from a geolocation (country) that is unusual. These calls can be the result of compromised credentials or keys.
- Job details
-
-
Detects unusually rare
event.action
values compared to other countries (using therare
function).
-
Detects unusually rare
- Required Beats
-
- Filebeat
- Required ECS fields when not using Beats
-
-
event.action
-
source.geo.country_iso_code
-
source.ip
-
- rare_method_for_a_username
-
Looks for AWS API calls that—while not inherently suspicious or abnormal—are sourcing from a user context that does not normally call the method. These calls can be the result of compromised credentials or keys as someone uses a valid account to persist, move laterally, or exfil data.
- Job details
-
-
Detects unusually rare
event.action
values compared to other users (using therare
function).
-
Detects unusually rare
- Required Beats
-
- Filebeat
- Required ECS fields when not using Beats
-
-
event.action
-
source.geo.city_name
-
source.ip
-
user.name
-
Security Packetbeat
editDetect suspicious network activity in Packetbeat data.
- packetbeat_dns_tunneling
-
Searches for unusually large numbers of DNS queries for a single top-level DNS domain, which is often used for DNS tunneling.
DNS tunneling can be used for command-and-control, persistence, or data exfiltration activity. For example,
dnscat
tends to generate many DNS questions for a top-level domain as it uses the DNS protocol to tunnel data.- Job details
-
-
Analyzes network activity logs where
agent.type
ispacketbeat
. - Models occurrences of DNS activity.
-
Detects unusual DNS activity (using the
high_info_content
function).
-
Analyzes network activity logs where
- Required Beats
-
- Packetbeat (Windows and Linux)
- Required ECS fields when not using Beats
-
-
destination.ip
-
dns.question.registered_domain
-
host.name
-
dns.question.name
-
event.dataset
-
agent.type
-
This job uses the Packetbeat
dns.question.etld_plus_one
field, which is not defined in ECS. Instead, map your network data to thedns.question.registered_domain
ECS field. - packetbeat_rare_dns_question
-
Searches for rare and unusual DNS queries that indicate network activity with unusual domains is about to occur. This can be due to initial access, persistence, command-and-control, or exfiltration activity.
For example, when a user clicks on a link in a phishing email or opens a malicious document, a request may be sent to download and run a payload from an uncommon domain. When malware is already running, it may send requests to an uncommon DNS domain the malware uses for command-and-control communication.
- Job details
-
-
Analyzes network activity logs where
agent.type
ispacketbeat
. - Models occurrences of DNS activity.
-
Detects DNS activity that is rare compared to other DNS activities (using the
rare
function).
-
Analyzes network activity logs where
- Required Beats
-
- Packetbeat (Windows and Linux)
- Required ECS fields when not using Beats
-
-
host.name
-
dns.question.name
-
dns.question.type
-
event.dataset
-
agent.type
-
- packetbeat_rare_server_domain
-
Searches for rare and unusual DNS queries that indicate network activity with unusual domains is about to occur. This can be due to initial access, persistence, command-and-control, or exfiltration activity.
For example, when a user clicks on a link in a phishing email or opens a malicious document, a request may be sent to download and run a payload from an uncommon HTTP or TLS server. When malware is already running, it may send requests to an uncommon DNS domain the malware uses for command-and-control communication.
- Job details
-
-
Analyzes network activity logs where
agent.type
ispacketbeat
. - Models HTTP or TLS domain activity.
-
Detects HTTP or TLS domain activity that is rare compared to other
activities (using the
rare
function).
-
Analyzes network activity logs where
- Required Beats
-
- Packetbeat (Windows and Linux)
- Required ECS fields when not using Beats
-
-
destination.ip
-
source.ip
-
host.name
-
server.domain
-
agent.type
-
- packetbeat_rare_urls
-
Searches for rare and unusual URLs that indicate unusual web browsing activity. This can be due to initial access, persistence, command-and-control, or exfiltration activity.
For example, in a strategic web compromise or watering hole attack, when a trusted website is compromised to target a particular sector or organization, targeted users may receive emails with uncommon URLs for trusted websites. These URLs can be used to download and run a payload. When malware is already running, it may send requests to uncommon URLs on trusted websites the malware uses for command-and-control communication. When rare URLs are observed being requested for a local web server by a remote source, these can be due to web scanning, enumeration or attack traffic, or they can be due to bots and web scrapers which are part of common Internet background traffic.
- Job details
-
-
Analyzes network activity logs where
agent.type
ispacketbeat
. - Models occurrences of web browsing URL activity.
-
Detects URL activity that rarely occurs compared to other URL activities
(using the
rare
function).
-
Analyzes network activity logs where
- Required Beats
-
- Packetbeat (Windows and Linux)
- Required ECS fields when not using Beats
-
-
destination.ip
-
host.name
-
url.full
-
agent.type
-
- packetbeat_rare_user_agent
-
Searches for rare and unusual user agents that indicate web browsing activity by an unusual process other than a web browser. This can be due to persistence, command-and-control, or exfiltration activity. Uncommon user agents coming from remote sources to local destinations are often the result of scanners, bots, and web scrapers, which are part of common internet background traffic.
Much of this is noise, but more targeted attacks on websites using tools like Burp or SQLmap can sometimes be discovered by spotting uncommon user agents. Uncommon user agents in traffic from local sources to remote destinations can be any number of things, including harmless programs like weather monitoring or stock-trading programs. However, uncommon user agents from local sources can also be due to malware or scanning activity.
- Job details
-
-
Analyzes network activity logs where
agent.type
ispacketbeat
. - Models occurrences of HTTP user agent activity.
-
Detects HTTP user agent activity that occurs rarely compared to other HTTP
user agent activities (using the
rare
function).
-
Analyzes network activity logs where
- Required Beats
-
- Packetbeat (Windows and Linux)
- Required ECS fields when not using Beats
-
-
destination.ip
-
host.name
-
event.dataset
-
user_agent.original
-
agent.type
-
Security Winlogbeat
editDetect unusual processes and network activity in Winlogbeat data.
- rare_process_by_host_windows_ecs
-
Identifies rare processes that do not usually run on individual hosts, which can indicate execution of unauthorized services, malware, or persistence mechanisms.
Processes are considered rare when they only run occasionally as compared with other processes running on the host.
- Job details
-
-
Analyzes host activity logs where
agent.type
iswinlogbeat
. - Models occurrences of process activities on the host.
-
Detects unusually rare processes compared to other processes on the host (using the
rare
function).
-
Analyzes host activity logs where
- Required Beats
-
- Winlogbeat
- Required ECS fields when not using Beats
-
-
host.name
-
process.name
-
user.name
-
event.action
-
agent.type
-
- windows_anomalous_network_activity_ecs
-
Identifies OS processes that do not usually use the network but have unexpected network activity, which can indicate command-and-control, lateral movement, persistence, or data exfiltration activity.
A process with unusual network activity can denote process exploitation or injection, where the process is used to run persistence mechanisms that allow a malicious actor remote access or control of the host, data exfiltration, and execution of unauthorized network applications.
- Job details
-
-
Analyzes network activity logs where
agent.type
iswinlogbeat
. - Models the occurrences of processes that cause network activity.
-
Detects network activity caused by processes that occur rarely compared to
other processes (using the
rare
function).
-
Analyzes network activity logs where
- Required Beats
-
- Winlogbeat
- Required ECS fields when not using Beats
-
-
destination.ip
-
host.name
-
process.name
-
user.name
-
event.action
-
agent.type
-
- windows_anomalous_path_activity_ecs
-
Identifies processes started from atypical folders in the file system, which might indicate malware execution or persistence mechanisms.
In corporate Windows environments, software installation is centrally managed and it is unusual for programs to be executed from user or temporary directories. Processes executed from these locations can denote that a user downloaded software directly from the internet or a malicious script/macro executed malware.
- Job details
-
-
Analyzes host activity logs where
agent.type
iswinlogbeat
. - Models occurrences of processes in paths.
-
Detects activity in unusual paths (using the
rare
function).
-
Analyzes host activity logs where
- Required Beats
-
- Winlogbeat (Windows)
- Required ECS fields when not using Beats
-
-
host.name
-
process.name
-
user.name
-
process.working_directory
-
event.action
-
agent.type
-
- windows_anomalous_process_all_hosts_ecs
-
Searches for rare processes running on multiple hosts in an entire fleet or network.
This reduces the detection of false positives since automated maintenance processes usually only run occasionally on a single machine but are common to all or many hosts in a fleet.
- Job details
-
-
Analyzes host activity logs where
agent.type
iswinlogbeat
(Windows). - Models the occurrences of processes on all hosts.
-
Detects processes that occur rarely compared to other processes on all
hosts (using the
rare
function).
-
Analyzes host activity logs where
- Required Beats
-
- Winlogbeat
- Required ECS fields when not using Beats
-
-
host.name
-
process.name
-
user.name
-
process.executable
-
event.action
-
agent.type
-
- windows_anomalous_process_creation
-
Identifies unusual parent-child process relationships that can indicate malware execution or persistence mechanisms.
Malicious scripts often call on other applications and processes as part of their exploit payload. For example, when a malicious Office document runs scripts as part of an exploit payload, Excel or Word may start a script interpreter process, which, in turn, runs a script that downloads and executes malware. Another common scenario is Outlook running an unusual process when malware is downloaded in an email.
Monitoring and identifying anomalous process relationships is an excellent way of detecting new and emerging malware that is not yet recognized by anti-virus scanners.
- Job details
-
-
Analyzes host activity logs where
agent.type
iswinlogbeat
. -
Models occurrences of process creation activities (
partition_field_name
isprocess.parent.name
). -
Detects process relationships that are rare compared to other process
relationships (using the
rare
function).
-
Analyzes host activity logs where
- Required Beats
-
- Winlogbeat (Windows)
- Required ECS fields when not using Beats
-
-
host.name
-
process.name
-
user.name
-
process.parent.name
-
event.action
-
agent.type
-
- windows_anomalous_script
-
Searches for PowerShell scripts with unusual data characteristics, such as obfuscation, that may be a characteristic of malicious PowerShell script text blocks.
- Job details
-
-
Analyzes host activity logs where
agent.type
iswinlogbeat
. - Models occurrences of PowerShell script activities.
-
Detects unusual PowerShell script execution compared to other PowerShell
script activities (using the
high_info_content
function).
-
Analyzes host activity logs where
- Required Beats
-
- Winlogbeat (Windows)
This job is available only when you use Winlogbeat to ship data.
- windows_anomalous_service
-
Searches for unusual Windows services that can indicate execution of unauthorized services, malware, or persistence mechanisms.
In corporate Windows environments, hosts do not generally run many rare or unique services. This job helps detect malware and persistence mechanisms that have been installed and run as a service.
- Job details
-
-
Analyzes host activity logs where
agent.type
iswinlogbeat
. - Models occurrences of Windows service activities.
-
Detects Windows service activities that occur rarely compared to other Windows service activities (using the
rare
function).
-
Analyzes host activity logs where
- Required Beats
-
- Winlogbeat (Windows)
This job is available only when you use Winlogbeat to ship data.
- windows_anomalous_user_name_ecs
-
Searches for activity from users who are not normally active, which can indicate unauthorized changes, activity by unauthorized users, lateral movement, and compromised credentials.
In organizations, new usernames are not often created apart from specific types of system activities, such as creating new accounts for new employees. These user accounts quickly become active and routine.
Events from rarely used usernames can point to suspicious activity. Additionally, automated Linux fleets tend to see activity from rarely used usernames only when personnel log in to make authorized or unauthorized changes, or threat actors have acquired credentials and log in for malicious purposes. Unusual usernames can also indicate pivoting, where compromised credentials are used to try and move laterally from one host to another.
- Job details
-
-
Analyzes host activity logs where
agent.type
iswinlogbeat
(Windows). - Models user activity.
-
Detects users that are rarely or unusually active compared to other users
(using the
rare
function).
-
Analyzes host activity logs where
- Required Beats
-
- Winlogbeat
- Required ECS fields when not using Beats
-
-
host.name
-
process.name
-
user.name
-
event.action
-
agent.type
-
- windows_rare_user_runas_event
-
Searches for unusual user context switches using the
runas
command or similar techniques, which can indicate account takeover or privilege escalation using compromised accounts. Privilege elevation using tools likerunas
is more common for domain and network administrators than professionals who are not members of the technology department.- Job details
-
-
Analyzes host activity logs where
agent.type
iswinlogbeat
. - Models occurrences of user context switches.
-
Detects user context switches that occur rarely compared to other user context switches (using the
rare
function).
-
Analyzes host activity logs where
- Required Beats
-
- Winlogbeat (Windows)
- Required ECS fields when not using Beats
-
-
process.name
-
host.name
-
user.name
-
event.code
-
agent.type
-
Security Winlogbeat authentication
editDetect suspicious authentication events in Winlogbeat data.
- windows_rare_user_type10_remote_login
-
Searches for unusual remote desktop protocol (RDP) logins, which can indicate account takeover or credentialed persistence using compromised accounts. RDP attacks, such as BlueKeep, also tend to use unusual usernames.
- Job details
-
-
Analyzes host activity logs where
agent.type
iswinlogbeat
. - Models occurrences of user remote login activities.
-
Detects user remote login activities that occur rarely compared to other
user remote login activities (using the
rare
function).
-
Analyzes host activity logs where
- Required Beats
-
- Winlogbeat (Windows)
This job is available only when you use Winlogbeat to ship data.