« Appendix G: Nginx anomaly detection configurations Appendix I: Uptime anomaly detection configurations »

›

Appendix H: Security anomaly detection configurations

These anomaly detection jobs automatically detect file system and network anomalies on your hosts. They appear in the Anomaly Detection interface of the Elastic Security app in Kibana when you have data that matches their configuration. For more information, refer to Anomaly detection with machine learning.

Security: Authentication

edit

Detect anomalous activity in your ECS-compatible authentication logs.

In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the manifest file. In the Elastic Security app, it looks in the data view specified in the securitySolution:defaultIndex advanced setting for data that matches the query.

By default, when you create these job in the Elastic Security app, it uses a data view that applies to multiple indices. To get the same results if you use the Machine Learning app, create a similar data view then select it in the job wizard.

Name	Description	Job	Datafeed
auth_high_count_logon_events	Looks for an unusually large spike in successful authentication events. This can be due to password spraying, user enumeration, or brute force activity.
auth_high_count_logon_events_for_a_source_ip	Looks for an unusually large spike in successful authentication events from a particular source IP address. This can be due to password spraying, user enumeration or brute force activity.
auth_high_count_logon_fails	Looks for an unusually large spike in authentication failure events. This can be due to password spraying, user enumeration, or brute force activity and may be a precursor to account takeover or credentialed access.
auth_rare_hour_for_a_user	Looks for a user logging in at a time of day that is unusual for the user. This can be due to credentialed access via a compromised account when the user and the threat actor are in different time zones. In addition, unauthorized user activity often takes place during non-business hours.
auth_rare_source_ip_for_a_user	Looks for a user logging in from an IP address that is unusual for the user. This can be due to credentialed access via a compromised account when the user and the threat actor are in different locations. An unusual source IP address for a username could also be due to lateral movement when a compromised account is used to pivot between hosts.
auth_rare_user	Looks for an unusual user name in the authentication logs. An unusual user name is one way of detecting credentialed access by means of a new or dormant user account. A user account that is normally inactive, because the user has left the organization, which becomes active, may be due to credentialed access using a compromised account password. Threat actors will sometimes also create new users as a means of persisting in a compromised web application.
suspicious_login_activity	Detect unusually high number of authentication attempts.

Security: CloudTrail

edit

Detect suspicious activity recorded in your CloudTrail logs.

Name	Description	Job	Datafeed
high_distinct_count_error_message	Looks for a spike in the rate of an error message which may simply indicate an impending service failure but these can also be byproducts of attempted or successful persistence, privilege escalation, defense evasion, discovery, lateral movement, or collection activity by a threat actor.
rare_error_code	Looks for unusual errors. Rare and unusual errors may simply indicate an impending service failure but they can also be byproducts of attempted or successful persistence, privilege escalation, defense evasion, discovery, lateral movement, or collection activity by a threat actor.
rare_method_for_a_city	Looks for AWS API calls that, while not inherently suspicious or abnormal, are sourcing from a geolocation (city) that is unusual. This can be the result of compromised credentials or keys.
rare_method_for_a_country	Looks for AWS API calls that, while not inherently suspicious or abnormal, are sourcing from a geolocation (country) that is unusual. This can be the result of compromised credentials or keys.
rare_method_for_a_username	Looks for AWS API calls that, while not inherently suspicious or abnormal, are sourcing from a user context that does not normally call the method. This can be the result of compromised credentials or keys as someone uses a valid account to persist, move laterally, or exfil data.

Security: Linux

edit

Anomaly detection jobs for Linux host-based threat hunting and detection.

Name	Description	Job	Datafeed
v3_linux_anomalous_network_activity	Looks for unusual processes using the network which could indicate command-and-control, lateral movement, persistence, or data exfiltration activity.
v3_linux_anomalous_network_port_activity	Looks for unusual destination port activity that could indicate command-and-control, persistence mechanism, or data exfiltration activity.
v3_linux_anomalous_process_all_hosts	Looks for processes that are unusual to all Linux hosts. Such unusual processes may indicate unauthorized software, malware, or persistence mechanisms.
v3_linux_anomalous_user_name	Rare and unusual users that are not normally active may indicate unauthorized changes or activity by an unauthorized user which may be credentialed access or lateral movement.
v3_linux_network_configuration_discovery	Looks for commands related to system network configuration discovery from an unusual user context. This can be due to uncommon troubleshooting activity or due to a compromised account. A compromised account may be used by a threat actor to engage in system network configuration discovery to increase their understanding of connected networks and hosts. This information may be used to shape follow-up behaviors such as lateral movement or additional discovery.
v3_linux_network_connection_discovery	Looks for commands related to system network connection discovery from an unusual user context. This can be due to uncommon troubleshooting activity or due to a compromised account. A compromised account may be used by a threat actor to engage in system network connection discovery to increase their understanding of connected services and systems. This information may be used to shape follow-up behaviors such as lateral movement or additional discovery.
v3_linux_rare_metadata_process	Looks for anomalous access to the metadata service by an unusual process. The metadata service may be targeted in order to harvest credentials or user data scripts containing secrets.
v3_linux_rare_metadata_user	Looks for anomalous access to the metadata service by an unusual user. The metadata service may be targeted in order to harvest credentials or user data scripts containing secrets.
v3_linux_rare_sudo_user	Looks for sudo activity from an unusual user context. Unusual user context changes can be due to privilege escalation.
v3_linux_rare_user_compiler	Looks for compiler activity by a user context which does not normally run compilers. This can be ad-hoc software changes or unauthorized software deployment. This can also be due to local privilege elevation via locally run exploits or malware activity.
v3_linux_system_information_discovery	Looks for commands related to system information discovery from an unusual user context. This can be due to uncommon troubleshooting activity or due to a compromised account. A compromised account may be used to engage in system information discovery to gather detailed information about system configuration and software versions. This may be a precursor to the selection of a persistence mechanism or a method of privilege elevation.
v3_linux_system_process_discovery	Looks for commands related to system process discovery from an unusual user context. This can be due to uncommon troubleshooting activity or due to a compromised account. A compromised account may be used to engage in system process discovery to increase their understanding of software applications running on a target host or network. This may be a precursor to the selection of a persistence mechanism or a method of privilege elevation.
v3_linux_system_user_discovery	Looks for commands related to system user or owner discovery from an unusual user context. This can be due to uncommon troubleshooting activity or due to a compromised account. A compromised account may be used to engage in system owner or user discovery to identify currently active or primary users of a system. This may be a precursor to additional discovery, credential dumping, or privilege elevation activity.
v3_rare_process_by_host_linux	Looks for processes that are unusual to a particular Linux host. Such unusual processes may indicate unauthorized software, malware, or persistence mechanisms.

Security: Network

edit

Detect anomalous network activity in your ECS-compatible network logs.

By default, when you create these jobs in the Elastic Security app, it uses a data view that applies to multiple indices. To get the same results if you use the Machine Learning app, create a similar data view then select it in the job wizard.

Name	Description	Job	Datafeed
high_count_by_destination_country	Looks for an unusually large spike in network activity to one destination country in the network logs. This could be due to unusually large amounts of reconnaissance or enumeration traffic. Data exfiltration activity may also produce such a surge in traffic to a destination country which does not normally appear in network traffic or business work-flows. Malware instances and persistence mechanisms may communicate with command-and-control (C2) infrastructure in their country of origin, which may be an unusual destination country for the source network.
high_count_network_denies	Looks for an unusually large spike in network traffic that was denied by network ACLs or firewall rules. Such a burst of denied traffic is usually either 1) a misconfigured application or firewall or 2) suspicious or malicious activity. Unsuccessful attempts at network transit, in order to connect to command-and-control (C2), or engage in data exfiltration, may produce a burst of failed connections. This could also be due to unusually large amounts of reconnaissance or enumeration traffic. Denial-of-service attacks or traffic floods may also produce such a surge in traffic.
high_count_network_events	Looks for an unusually large spike in network traffic. Such a burst of traffic, if not caused by a surge in business activity, can be due to suspicious or malicious activity. Large-scale data exfiltration may produce a burst of network traffic; this could also be due to unusually large amounts of reconnaissance or enumeration traffic. Denial-of-service attacks or traffic floods may also produce such a surge in traffic.
rare_destination_country	Looks for an unusual destination country name in the network logs. This can be due to initial access, persistence, command-and-control, or exfiltration activity. For example, when a user clicks on a link in a phishing email or opens a malicious document, a request may be sent to download and run a payload from a server in a country which does not normally appear in network traffic or business work-flows. Malware instances and persistence mechanisms may communicate with command-and-control (C2) infrastructure in their country of origin, which may be an unusual destination country for the source network.

Security: Packetbeat

edit

Detect suspicious network activity in Packetbeat data.

Name	Description	Job	Datafeed
packetbeat_dns_tunneling	Looks for unusual DNS activity that could indicate command-and-control or data exfiltration activity.
packetbeat_rare_dns_question	Looks for unusual DNS activity that could indicate command-and-control activity.
packetbeat_rare_server_domain	Looks for unusual HTTP or TLS destination domain activity that could indicate execution, persistence, command-and-control or data exfiltration activity.
packetbeat_rare_urls	Looks for unusual web browsing URL activity that could indicate execution, persistence, command-and-control or data exfiltration activity.
packetbeat_rare_user_agent	Looks for unusual HTTP user agent activity that could indicate execution, persistence, command-and-control or data exfiltration activity.

Security: Windows

edit

Anomaly detection jobs for Windows host-based threat hunting and detection.

If there are additional requirements such as installing the Windows System Monitor (Sysmon) or auditing process creation in the Windows security event log, they are listed for each job.

Name	Description	Job	Datafeed
v3_rare_process_by_host_windows	Looks for processes that are unusual to a particular Windows host. Such unusual processes may indicate unauthorized software, malware, or persistence mechanisms.
v3_windows_anomalous_network_activity	Looks for unusual processes using the network which could indicate command-and-control, lateral movement, persistence, or data exfiltration activity.
v3_windows_anomalous_path_activity	Looks for activity in unusual paths that may indicate execution of malware or persistence mechanisms. Windows payloads often execute from user profile paths.
v3_windows_anomalous_process_all_hosts	Looks for processes that are unusual to all Windows hosts. Such unusual processes may indicate execution of unauthorized software, malware, or persistence mechanisms.
v3_windows_anomalous_process_creation	Looks for unusual process relationships which may indicate execution of malware or persistence mechanisms.
v3_windows_anomalous_script	Looks for unusual powershell scripts that may indicate execution of malware, or persistence mechanisms.
v3_windows_anomalous_service	Looks for rare and unusual Windows service names which may indicate execution of unauthorized services, malware, or persistence mechanisms.
v3_windows_anomalous_user_name	Rare and unusual users that are not normally active may indicate unauthorized changes or activity by an unauthorized user which may be credentialed access or lateral movement.
v3_windows_rare_metadata_process	Looks for anomalous access to the metadata service by an unusual process. The metadata service may be targeted in order to harvest credentials or user data scripts containing secrets.
v3_windows_rare_metadata_user	Looks for anomalous access to the metadata service by an unusual user. The metadata service may be targeted in order to harvest credentials or user data scripts containing secrets.
v3_windows_rare_user_runas_event	Unusual user context switches can be due to privilege escalation.
v3_windows_rare_user_type10_remote_login	Unusual RDP (remote desktop protocol) user logins can indicate account takeover or credentialed access.

Security: Elastic Integrations

edit

Elastic Integrations are a streamlined way to add Elastic assets to your environment, such as data ingestion, transforms, and in this case, machine learning capabilities for Security.

The following Integrations use machine learning to analyze patterns of user and entity behavior, and help detect and alert when there is related suspicious activity in your environment.

Domain Generation Algorithm (DGA) Detection

Machine learning solution package to detect domain generation algorithm (DGA) activity in your network data. Refer to the subscription page to learn more about the required subscription.

To download, refer to the documentation.

Name	Description
dga_high_sum_probability	Detect domain generation algorithm (DGA) activity in your network data.

The job configurations and datafeeds can be found here.

Living off the Land Attack (LotL) Detection

Machine learning solution package to detect Living off the Land (LotL) attacks in your environment. Refer to the subscription page to learn more about the required subscription. (Also known as ProblemChild).

To download, refer to the documentation.

Name	Description
problem_child_rare_process_by_host	Looks for a process that has been classified as malicious on a host that does not commonly manifest malicious process activity.
problem_child_high_sum_by_host	Looks for a set of one or more malicious child processes on a single host.
problem_child_rare_process_by_user	Looks for a process that has been classified as malicious where the user context is unusual and does not commonly manifest malicious process activity.
problem_child_rare_process_by_parent	Looks for rare malicious child processes spawned by a parent process.
problem_child_high_sum_by_user	Looks for a set of one or more malicious processes, started by the same user.
problem_child_high_sum_by_parent	Looks for a set of one or more malicious child processes spawned by the same parent process.

The job configurations and datafeeds can be found here.

Data Exfiltration Detection (DED)

Machine learning package to detect data exfiltration in your network and file data. Refer to the subscription page to learn more about the required subscription.

To download, refer to the documentation.

Name	Description
ded_high_sent_bytes_destination_geo_country_iso_code	Detects data exfiltration to an unusual geo-location (by country iso code).
ded_high_sent_bytes_destination_ip	Detects data exfiltration to an unusual geo-location (by IP address).
ded_high_sent_bytes_destination_port	Detects data exfiltration to an unusual destination port.
ded_high_sent_bytes_destination_region_name	Detects data exfiltration to an unusual geo-location (by region name).
ded_high_bytes_written_to_external_device	Detects data exfiltration activity by identifying high bytes written to an external device.
ded_rare_process_writing_to_external_device	Detects data exfiltration activity by identifying a file write started by a rare process to an external device.
ded_high_bytes_written_to_external_device_airdrop	Detects data exfiltration activity by identifying high bytes written to an external device via Airdrop.

The job configurations and datafeeds can be found here.

Lateral Movement Detection (LMD)

Machine learning package to detect lateral movement based on file transfer activity and Windows RDP events. Refer to the subscription page to learn more about the required subscription.

To download, refer to the documentation.

Name	Description
lmd_high_count_remote_file_transfer	Detects unusually high file transfers to a remote host in the network.
lmd_high_file_size_remote_file_transfer	Detects unusually high size of files shared with a remote host in the network.
lmd_rare_file_extension_remote_transfer	Detects data exfiltration to an unusual destination port.
lmd_rare_file_path_remote_transfer	Detects unusual folders and directories on which a file is transferred.
lmd_high_mean_rdp_session_duration	Detects unusually high mean of RDP session duration.
lmd_high_var_rdp_session_duration	Detects unusually high variance in RDP session duration.
lmd_high_sum_rdp_number_of_processes	Detects unusually high number of processes started in a single RDP session.
lmd_unusual_time_weekday_rdp_session_start	Detects an RDP session started at an usual time or weekday.
lmd_high_rdp_distinct_count_source_ip_for_destination	Detects a high count of source IPs making an RDP connection with a single destination IP.
lmd_high_rdp_distinct_count_destination_ip_for_source	Detects a high count of destination IPs establishing an RDP connection with a single source IP.
lmd_high_mean_rdp_process_args	Detects unusually high number of process arguments in an RDP session.

The job configurations and datafeeds can be found here.

« Appendix G: Nginx anomaly detection configurations Appendix I: Uptime anomaly detection configurations »