Lateral Movement Detection Model

edit

Lateral Movement Detection Model

edit

Version

2.1.4 (View all)

Compatible Kibana version(s)

8.9.0 or higher

Supported Serverless project types
What’s this?

Security

Subscription level
What’s this?

Platinum

Level of support
What’s this?

Elastic

The Lateral movement detection model package contains assets that detect lateral movement based on file transfer activity and Windows RDP events. This package requires a Platinum subscription. Please ensure that you have a Trial, Platinum, or Enterprise subscription before proceeding. This package is licensed under Elastic License 2.0.

For more detailed information refer to the following blogs:

Installation

edit
  1. Upgrading: If upgrading from a version below v2.0.0, see the section v2.0.0 and beyond.
  2. Add the Integration Package: Install the package via Management > Integrations > Add Lateral Movement Detection. Configure the integration name and agent policy. Click Save and Continue.
  3. Check the health of the transform: The transform is scheduled to run every hour. This transform creates the index ml-rdp-lmd. To check the health of the transform go to Management > Stack Management > Data > Transforms under logs-lmd.pivot_transform-default-<FLEET-TRANSFORM-VERSION>.
  4. Create data views for anomaly detection jobs: The anomaly detection jobs under this package rely on two indices. One has file transfer events (logs-*), and the other index (ml-rdp-lmd) collects RDP session information from a transform. Before enabling the anomaly detection jobs, create a data view with both index patterns.

    1. Go to Stack Management > Kibana > Data Views and click Create data view.
    2. Enter the name of your respective index patterns in the Index pattern box, i.e., logs-*, ml-rdp-lmd, and copy the same in the Name field.
    3. Select @timestamp under the Timestamp field and click on Save data view to Kibana.
    4. Use the new data view (logs-*, ml-rdp-lmd) to create anomaly detection jobs for this package.
  5. Add preconfigured anomaly detection jobs: In Machine Learning > Anomaly Detection, when you create a job, you should see an option to Use preconfigured jobs with a card for Lateral Movement Detection. When you select the card, you will see pre-configured anomaly detection jobs that you can enable depending on what makes the most sense for your environment. Note: In the Machine Learning app, these configurations are available only when data exists that matches the query specified in the lmd-ml file. For example, this would be available in logs-endpoint.events.* if you used Elastic Defend to collect events.
  6. Data view configuration for Dashboards: For the dashboard to work as expected, the following settings need to be configured in Kibana.

    1. You have started the above anomaly detection jobs.
    2. You have read access to .ml-anomalies-shared index or are assigned the machine_learning_user role. For more information on roles, please refer to Built-in roles in Elastic. Please be aware that a user who has access to the underlying machine learning results indices can see the results of all jobs in all spaces. Be mindful of granting permissions if you use Kibana spaces to control which users can see which machine learning results. For more information on machine learning privileges, refer to setup-privileges.
    3. After enabling the jobs, go to Management > Stack Management > Kibana > Data Views. Click on Create data view with the following settings:

      • Name: .ml-anomalies-shared
      • Index pattern : .ml-anomalies-shared
      • Select Show Advanced settings enable Allow hidden and system indices
      • Custom data view ID: .ml-anomalies-shared
  7. Enabling detection rules: You can also enable detection rules to alert on Lateral Movement activity in your environment, based on anomalies flagged by the above ML jobs. As of version 2.0.0 of this package, these rules are available as part of the Detection Engine, and can be found using the tag Use Case: Lateral Movement Detection. See this documentation for more information on importing and enabling the rules.
  8. Use with Living off the Land Detection: This integration package can be used along with Living off the Land detection, see the section Install Living off the Land package to detect malicious processes.
Data Exfiltration Detection Rules

In Security > Rules, filtering with the “Use Case: Lateral Movement Detection” tag

Dashboard

edit

After the anomaly detectors and the data views for the dashboard are configured, the Lateral Movement Detection Dashboard is available under Analytics > Dashboard. This dashboard gives an overview of anomalies triggered for the lateral movement detection package.

Install ProblemChild package to detect malicious processes

edit

To detect malicious RDP processes started in a session, install the Living off the Land Attack (LotL) Detection package. Follow the steps under the package overview to install the related assets. Use the below filter query to examine model predictions on RDP events only.

Clone the anomaly detection jobs available under the Living off the Land Attack (LotL) Detection package and follow the below steps to customize them only to process Windows RDP events in the datafeed:

  1. Click on the Actions panel at the right-most corner of the anomaly detection job and then select the Edit job option.
  2. Under the Datafeed panel, enter the below query to filter malicious RDP processes.
{
  "bool": {
    "minimum_should_match": 1,
    "should": [
      {
        "match": {
          "problemchild.prediction": 1
        }
      },
      {
        "match": {
          "blocklist_label": 1
        }
      }
    ],
    "must_not": [
      {
        "terms": {
          "user.name": [
            "system"
          ]
        }
      }
    ],
    "filter": [
      {
        "exists": {
          "field": "process.Ext.session_info.client_address"
        }
      },
      {
        "exists": {
          "field": "process.Ext.authentication_id"
        }
      },
      {
        "exists": {
          "field": "host.ip"
        }
      },
      {
        "term": {
          "event.category": "process"
        }
      },
      {
        "term": {
          "process.Ext.session_info.logon_type": "RemoteInteractive"
        }
      }
    ]
  }
}

Anomaly Detection Jobs

edit

Detects potential lateral movement activity by identifying malicious file transfers and RDP sessions in an environment.

Job Description

lmd_high_count_remote_file_transfer

Detects unusually high file transfers to a remote host in the network.

lmd_high_file_size_remote_file_transfer

Detects unusually high size of files shared with a remote host in the network.

lmd_rare_file_extension_remote_transfer

Detects rare file extensions shared with a remote host in the network.

lmd_rare_file_path_remote_transfer

Detects unusual folders and directories on which a file is transferred (by a host).

lmd_high_mean_rdp_session_duration

Detects unusually high mean of RDP session duration.

lmd_high_var_rdp_session_duration

Detects unusually high variance in RDP session duration.

lmd_high_sum_rdp_number_of_processes

Detects unusually high number of processes started in a single RDP session.

lmd_unusual_time_weekday_rdp_session_start

Detects an RDP session started at an usual time or weekday.

lmd_high_rdp_distinct_count_source_ip_for_destination

Detects a high count of source IPs making an RDP connection with a single destination IP.

lmd_high_rdp_distinct_count_destination_ip_for_source

Detects a high count of destination IPs establishing an RDP connection with a single source IP.

lmd_high_mean_rdp_process_args

Detects unusually high number of process arguments in an RDP session.

v2.0.0 and beyond

edit

v2.0.0 of the package introduces breaking changes, namely deprecating detection rules from the package. To continue receiving updates to Lateral Movement Detection, we recommend upgrading to v2.0.0 after doing the following:

  • Delete existing ML jobs: Navigate to Machine Learning > Anomaly Detection and delete jobs corresponding to the following IDs:

    • high-count-remote-file-transfer
    • high-file-size-remote-file-transfer
    • rare-file-extension-remote-transfer
    • rare-file-path-remote-transfer
    • high-mean-rdp-session-duration
    • high-var-rdp-session-duration
    • high-sum-rdp-number-of-processes
    • unusual-time-weekday-rdp-session-start
    • high-rdp-distinct-count-source-ip-for-destination
    • high-rdp-distinct-count-destination-ip-for-source
    • high-mean-rdp-process-args

Depending on the version of the package you’re using, you might also be able to search for the above jobs using the group lateral_movement.

  • Uninstall existing rules associated with this package: Navigate to Security > Rules and delete the following rules:

    • Spike in Remote File Transfers
    • Unusual Remote File Size
    • Unusual Remote File Directory
    • Unusual Remote File Extension
    • Malicious Remote File Creation
    • Remote File Creation on a Sensitive Directory
    • Spike in number of processes in an RDP session
    • High mean of RDP session duration
    • High variance in RDP session duration
    • Unusually high number of process arguments in an RDP session
    • Spike in number of connections made to a source IP
    • Spike in number of connections made to a destination IP
    • Unusual time or day for an RDP session start

Depending on the version of the package you’re using, you might also be able to search for the above rules using the tag Lateral Movement.

  • Upgrade the Lateral Movement Detection package to v2.0.0 using the steps here
  • Install the new rules as described in the Enabling detection rules section below

In version 2.1.2, the package ignores data in cold and frozen data tiers to reduce heap memory usage, avoid running on outdated data, and to follow best practices.

Licensing

edit

Usage in production requires that you have a license key that permits use of machine learning features.

Changelog

edit
Changelog
Version Details Kibana version(s)

2.1.4

Enhancement (View pull request)
Remove scripted_metric aggregation from pivot transform

8.9.0 or higher

2.1.3

Enhancement (View pull request)
Improve package installation documentation

8.9.0 or higher

2.1.2

Enhancement (View pull request)
Add query settings to ignore frozen and cold data tiers

8.9.0 or higher

2.1.1

Enhancement (View pull request)
Add unattended flag for Serverless

8.9.0 or higher

2.1.0

Enhancement (View pull request)
Add serverless support

8.9.0 or higher

2.0.0

Enhancement (View pull request)
Removing detection rules from the package

8.9.0 or higher

1.0.2

Enhancement (View pull request)
Added additional ml jobs and security rules to the package

8.8.0 or higher

1.0.1

Enhancement (View pull request)
Add the Advanced Analytics (UEBA) subcategory

8.5.0 or higher

1.0.0

Enhancement (View pull request)
Added dashboard and released the package in production

8.5.0 or higher

0.0.1

Enhancement (View pull request)
Initial release of the package