8.7

edit

8.7.1

edit

Known issues

edit
  • After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (ManageRules), click the Custom rules filter next to the search bar, then select and delete the duplicate rules.
  • Index aliases and some data streams are not properly retrieved by the Elastic Security default data view.
  • The Add exceptions flyout loads indefinitely and an out of memory error displays when a rule has a large number of unmapped fields in multiple indices. To avoid this issue, use the exception APIs to manage exceptions.
  • If you modify an exception item using the update exception item API and only specify its item_id, the exception item is erroneously duplicated. To avoid this issue, you can either:

    If you’ve already encountered this issue and want to find erroneously duplicated exceptions, use the queries provided below.

    • Query for finding exception documents that were duplicated from only specifying the item_id:

      // Retrieve exception documents grouped by `item_id`.
      // Each bucket contains all duplicates of that document.
      GET .kibana*/_search
      {
        "query": {
          "bool": {
            "filter": [
              {
                "match": {
                  "type": "exception-list"
                }
              }
            ]
          }
        },
        "aggs": {
          "item_id_duplicates": {
            "terms": {
              "field": "exception-list.item_id",
              "min_doc_count": 2
            },
            "aggs": {
              "ids": {
                "top_hits": {
                  "size": 100, // Increase this if you may have more duplicates.
                  "_source": false
                }
              }
            }
          }
        },
        "size": 0
      }
    • Query for finding exception documents that were duplicated and have lost their item_id because their id was used to update them:

      // Each item returned lost its `item_id`, which is expected to be present and unique.
      GET .kibana*/_search
      {
        "query": {
          "bool": {
            "filter": [
              {
                "term": {
                  "exception-list.list_type": "item"
                }
              }
            ],
            "must_not": [
              {
                "exists": {
                  "field": "exception-list.item_id"
                }
              }
            ]
          }
        }
      }

Enhancements

edit

There are no user-facing changes in 8.7.1.

Bug fixes

edit
  • Fixes a bug that caused the Elastic Agent upgrade from 8.6.2 to 8.7.0 to fail for Elastic Agents running the Osquery Manager integration (#2448).

    To prevent upgrade issues while upgrading Elastic Agent from 8.6.2 or 8.7.0, delete the Osquery Manager integration from your Elastic Agents before upgrading them, then re-add it after upgrading.

    If you already upgraded your Elastic Agents and they’re stuck in the Updating state, you’ll need to upgrade them using the API instead of Fleet. First, remove the Osquery integration from the Elastic Agents, then use the Agent upgrade API to start the upgrade. After the upgrade completes, add the Osquery Manager integration to the Elastic Agents again.

  • Fixes a UI bug that affected the rule details page when rules had long queries (#153338).
  • Fixes exception operator logic for mapping field conflicts (#155071).
  • Fixes a bug that stopped Elastic Defend from working when malware protection was enabled on Linux hosts.
  • Fixes a bug that prevented policy artifacts from being properly updated after being added to the .fleet-artifacts index (#154810).

    If the error message Failed to download or validate user artifacts appears in the Policy Response section of your Elastic Defend integration policy, you can resolve the error by adding the affected endpoint policy artifacts as global trusted applications, endpoint exceptions, event filters, blocklist entries, or host isolation exceptions.

    To find the affected endpoint policy artifacts:

    1. Go to Fleet → Agent policies.
    2. Open the impacted Elastic Agent policy.
    3. Select Actions → View policy.
    4. Search for compression_algorithm: none in the policy details. This field-value pair will appear under the affected endpoint policy artifacts.

8.7.0

edit

Known issues

edit
  • After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (ManageRules), click the Custom rules filter next to the search bar, then select and delete the duplicate rules.
  • After alerts are generated for the first time, you may have to refresh your browser before your alert data appears on pages that use data views (for example, Timeline). Navigating between pages will not work (#3046).
  • The Elastic Agent upgrade from 8.6.2 to 8.7.0 might fail for Elastic Agents running the Osquery Manager integration (#2433). To prevent this, delete the Osquery Manager integration from your Elastic Agents before upgrading them to 8.7.0, then re-add it after upgrading.

    If you already upgraded your Elastic Agents, and they’re are stuck in the Updating state, you’ll need to upgrade them using the API instead of Fleet. First, remove the Osquery integration from the Elastic Agents, then use the Agent upgrade API to start the upgrade. After the upgrade completes, add the Osquery Manager integration to the Elastic Agents again.

    This problem can occur when upgrading Elastic Agents from 8.6.2 or 8.7.0 to any other version. Keep this in mind when upgrading Elastic Agents running 8.6.2 or 8.7.0 to newer versions.

  • Enabling malware protection on Linux hosts might cause Elastic Defend to enter a failed state when mount points are unmounted, which will stop the integration from working. To fix this, turn off malware protection on the Elastic Defend integration policy.
  • Index aliases and some data streams are not properly retrieved by the Elastic Security default data view.
  • The Add exceptions flyout loads indefinitely and an out of memory error displays when a rule has a large number of unmapped fields in multiple indices. To avoid this issue, use the exception APIs to manage exceptions.
  • If you modify an exception item using the update exception item API and only specify its item_id, the exception item is erroneously duplicated. To avoid this issue, you can either:

    If you’ve already encountered this issue and want to find erroneously duplicated exceptions, use the queries provided below.

    • Query for finding exception documents that were duplicated from only specifying the item_id:

      // Retrieve exception documents grouped by `item_id`.
      // Each bucket contains all duplicates of that document.
      GET .kibana*/_search
      {
        "query": {
          "bool": {
            "filter": [
              {
                "match": {
                  "type": "exception-list"
                }
              }
            ]
          }
        },
        "aggs": {
          "item_id_duplicates": {
            "terms": {
              "field": "exception-list.item_id",
              "min_doc_count": 2
            },
            "aggs": {
              "ids": {
                "top_hits": {
                  "size": 100, // Increase this if you may have more duplicates.
                  "_source": false
                }
              }
            }
          }
        },
        "size": 0
      }
    • Query for finding exception documents that were duplicated and have lost their item_id because their id was used to update them:

      // Each item returned lost its `item_id`, which is expected to be present and unique.
      GET .kibana*/_search
      {
        "query": {
          "bool": {
            "filter": [
              {
                "term": {
                  "exception-list.list_type": "item"
                }
              }
            ],
            "must_not": [
              {
                "exists": {
                  "field": "exception-list.item_id"
                }
              }
            ]
          }
        }
      }

Breaking changes

edit

There are no breaking changes in 8.7.0.

Deprecations

edit

There are no deprecations in 8.7.0.

New features

edit
  • Creates a new dashboard, Data Quality, which highlights any issues in your ECS field mappings (#150063).
  • Introduces a new event type (Credential access) to represent credential dumping attempts on Windows using tools like Mimikatz or fgdump.
  • Creates a Torq connector that can trigger Torq workflows. You must have at least a Platinum subscription to use this connector (#149405).
  • Adds more key performance indicator charts to the Alerts page (#150242, #149173, and #146938).
  • Allows you to set expiration dates for rule exceptions and choose whether to include expired exceptions when you export shared exception lists (#145180).
  • Adds two more inline actions (Copy to clipboard and Add to timeline investigation) to chart legends and tables (#146779).
  • Allows you to include connectors when exporting and importing rules (#148703).
  • Adds "Group by" functionality to the Alerts table (technical preview only)(#149145).
  • Improves the UI for building an Investigation Guide query (#150363).
  • Adds the ability to create a rule from a Timeline (#143020).
  • Adds the option to suppress custom query rule alerts during a specific time window. Duplicate alerts within that time window will be grouped (#148868).
  • Introduces the Cloud Security Posture Management (CSPM) feature, which detects misconfigured cloud resources in AWS accounts.

Enhancements

edit
  • Improves the formatting and readability of machine learning job names (#148974, #148780).
  • Improves sorting of the Rules table: allows you to sort it by any column, removes the Advanced sorting toggle, and removes the Version column (#149840).
  • Adds a Clear table filters button to the Rules page so you can clear all filters in one click (#150059).
  • Warns you about type conflicts and unmapped indices when creating rule exceptions (#149149).
  • Adds buttons to the Rules page that allow you to view only enabled or disabled rules (#150153).
  • The Related alerts by process ancestry section of the alert details flyout is now generally available (GA) (#152011).
  • Adds the option to suppress custom query rule alerts during a specific time window. Duplicate alerts within that time window will be grouped (#148868).
  • Reduces alert creation errors by stopping the detection engine from writing non-ECS-compliant fields to alerts from source events (#147628).
  • Simplifies the interface for navigating from a rule’s details page back to the Rules page (#147357).
  • Allows you to resize the Rule preview panel (#147351).
  • Improves the Bulk Edit API by adding a skipped property to rules that weren’t updated, and updates the toast message to show which were skipped (#147345).
  • Allows placeholder fields in Osquery queries (#146598).
  • Allows the Rules table state to persist even after you refresh or navigate to another page (#145111).
  • Improves data fetch performance throughout Elastic Security, especially for deployments with large indices and multiple integrations (#142904).
  • Introduces cross-cluster search support for Indicator Match rules by improving rule performance (#149113).
  • Improves the toast message that appears when you export an exception list (#152301).

Bug fixes

edit
  • Various bug fixes and UX enhancements for the Alerts page (#152402).
  • Fixes a bug that could cause your cursor to jump to the end of the text field when editing a rule action message (#150823).
  • Fixes a bug that could result in incorrect links to machine learning jobs from search results (#150881).
  • Fixes a bug that caused a fade in and out effect on rule descriptions (#150998).
  • Fixes a bug that caused the Alerts page to default to the wrong chart type (#151073).
  • Fixes a bug that could hide some shared exception lists when you changed the number of rows in the exceptions lists view (#151393).
  • Removes a blank option from the Field browser in the Add rule exception flyout (#151398).
  • Fixes a UI text bug that conflated Endpoint exceptions with regular rule exceptions (#151532).
  • Fixes a bug that could cause an unnecessary warning to display in the Add rule exception flyout (#151570).
  • Fixes a bug with the empty state that appears when your exception lists search yields no results (#151530).
  • Fixes a bug that sometimes prevented a Timeline from saving when it was created using the Investigate in timeline action on an alert (#151616).
  • Fixes a bug that could cause unnecessary validation errors in text entry fields in the Add rule exception flyout (#151654).
  • Fixes a bug that caused some module names to be partially hidden on the Overview dashboard (#151843).
  • Fixes a visual bug that affected empty rule previews (#151869).
  • Fixes a bug that could cause a rule’s related integrations to incorrectly appear as not installed on the Rules table and the rule details page (#152055, #149646).
  • Changes the Import list button name to Import value list (#152281).
  • Fixes a bug that broke the visual analyzer for sysmon data ingested via Elastic Agent (#152418).
  • Fixes a bug that incorrectly allowed you to use custom fields in the Add Endpoint Exception flyout (#152619).
  • Fixes a bug where the two breadcrumbs on shared exception lists pages did not use the same text (#152629).
  • Fixes an issue in the Update exception item API that incorrectly merged existing objects with updated objects (#151952).
  • Fixes a bug that affected the rule status refresh loading indicator (#147806).