The SIEM app is now a part of the Elastic Security solution. Click here to view the current documentation.
IMPORTANT: No additional bug fixes or documentation updates will be released for this version.
« Configuring map data Prebuilt job reference »
Elastic Docs ›SIEM Guide [7.8]

Anomaly Detection with Machine Learning

edit

Anomaly Detection with Machine Learning

edit

Machine learning functionality is available when you have the appropriate license, are using a cloud deployment, or are testing out a Free Trial.

You can view the details of detected anomalies within the Anomalies table widget shown on the Hosts, Network and associated Details pages, or even narrow to the specific date range of an anomaly from the Max Anomaly Score details in the overview of the Host and IP Details pages. Each of these interfaces also offer the ability to drag and drop details of the anomaly to Timeline, such as the Entity itself, or any of the associated Influencers.

ml ui

Manage machine learning jobs

edit

For users with the machine_learning_admin role, the ML job settings interface on the Detections page can be used for for viewing, starting, and stopping SIEM machine learning jobs.

To add a custom job to the ML job settings interface, add SIEM to the job’s Groups field (Kibana → Machine learning → Create/Edit job → Job details).

Prebuilt jobs

edit

The SIEM app comes with prebuilt machine learning anomaly detection jobs for automatically detecting host and network anomalies. The jobs are displayed in the Anomaly Detection interface. They are available when either:

  • You ship data using Beats, and Kibana is configured with the required index patterns (auditbeat-*, filebeat-*, packetbeat-*, or winlogbeat-* via Kibana → Management → Index Patterns).

Or

  • Your shipped data is ECS-compliant, and Kibana is configured with the shipped data’s index patterns.

Prebuilt job reference describes all available machine learning jobs, and lists which ECS fields are required on your hosts when you are not using Beats to ship your data. For information on tuning anomaly results to reduce the number of false positive, see Optimizing anomaly results.

Machine learning jobs look back and analyse two weeks of historical data prior to the time they are enabled. After jobs are enabled, they continuously analyse incoming data. When jobs are stopped and restarted within the two week timeframe, previously analysed data is not processed again.

View detected anomalies

edit

To view the Anomalies table widget and Max Anomaly Score By Job details, the user must have the ml_admin or ml_user role.

To adjust the score threshold that determines which anomalies are shown, you can modify Kibana → Management → Advanced Settings → siem:defaultAnomalyScore.

« Configuring map data Prebuilt job reference »