« Upgrade Elastic Security to 8.16.0 Post-upgrade steps (optional) »
Elastic Docs ›Elastic Security Solution [8.16] ›Upgrade Elastic Security to 8.16.0

Upgrade from 7.17 to an 8.x version

edit

Upgrade from 7.17 to an 8.x version

edit

Why it’s important to upgrade

edit

Upgrading provides you access to Elastic Security’s latest features, enhancements, and bug fixes, many of which enable you to save your organization money, respond faster to potential threats, and improve the tools you use to investigate and analyze your data. For more benefits, check out our blog, Top 5 reasons to upgrade Elastic Security. Also, it’s important to ensure that your deployment is fully maintained and supported. For more information, refer to Elastic’s Product End of Life Dates.

Plan for your upgrade

edit

Before upgrading to Elastic Security 8.x, consider the following recommendations:

  • Plan for an appropriate amount of time to complete the upgrade. Depending on your configuration and the size of your cluster, the process can take up to a week to complete.
  • Open a support case with Elastic to alert our Elastic Support team of your system change. If you need additional assistance, Elastic Consulting Services provides the technical expertise and step-by-step approach for upgrading your Elastic deployment.
  • Choose a version to upgrade to. We recommend the latest minor and patch version. Be sure to upgrade your development or non-production deployment to the same version as your production deployment.
  • Ensure that you have stack monitoring enabled in Kibana. Take note of your current index and search rate.
  • Review your selected version’s features, Elastic connectors, integrations, and detection rules to determine if you can replace any customized content with out-of-the-box functionality. This can help reduce your workload and the complexity of your upgrade.
  • If your organization sends alerts (formerly known as signals) to an external SOAR (security orchestration, automation, and response) platform, you may need to change your workflows to accommodate the new alert schema used in 8.x.
  • Review release notes, deprecations, and breaking changes for Elastic Security, Elasticsearch, Kibana, and, if applicable, Fleet and Elastic Agent, Beats, and Logstash. Identify any issues that might affect your deployment. Work with your Elastic team on any questions you may have. Start with breaking changes for your solution and platform components, such as Elasticsearch and Kibana.
  • Schedule a system maintenance window within your organization.

Pre-upgrade steps

edit

To prepare for the upgrade process, follow these steps before you start:

  1. Do a software version inventory across your entire Elastic deployment, including Elasticsearch, Kibana, Elastic Agent, Beats, and Logstash.

  2. If you’re running any versions earlier than 7.17, you must first upgrade them all to the latest 7.17.x patch release before upgrading to 8.x. This enables you to use the Upgrade Assistant to upgrade to 8.x.

    If you’re managing your deployments using Elastic Cloud Enterprise (ECE) or Elastic Cloud on Kubernetes (ECK), you may need to upgrade the system clusters first.

  3. Note that alerts in 8.x are written using the kibana_system user instead of the current user, so any user-modified ingest pipelines configured against the alert index (this is uncommon) will use the kibana_system user privileges, which may break your ingest pipeline. If you encounter this issue, please do not update your ingest pipeline. Instead, contact your Elastic Support rep for assistance.

  4. Run the Upgrade Assistant. Take note of any critical error messages, then work with your Elastic Support rep to resolve them.

    Requirements

    To run the Upgrade Assistant, you must have the superuser role on the cluster.

  5. If you’re not using snapshot lifecycle management (SLM), you must set up and configure a policy, then run the policy to create at least one snapshot — a backup of indices taken from a running cluster. If you need to roll back during the upgrade process, use a recent snapshot to avoid data loss. Snapshots are incremental — depending on the cluster size and the input/output rate, the initial snapshot may take several hours to complete. If you’re using SLM, check the SLM history to ensure that snapshots are completing successfully.

Perform an 8.x upgrade on a deployment

edit

We strongly recommend performing the following steps on a non-production deployment first to address any potential issues before upgrading your production deployments. If you’re using a cross-cluster search environment, upgrade your remote deployments first.

  1. If you haven’t already done so, back up your cluster data to a snapshot.
  2. We recommend you export all your custom detection rules in case there are issues with the detection engine after the upgrade.

  3. Upgrade Elasticsearch.

    • If you’re using Elastic Cloud, we recommend upgrades with no downtime. Refer to these instructions.
    • If you’re using Elastic Cloud Enterprise (ECE), refer to these instructions.
    • If you’re using Elastic Cloud on Kubernetes (ECK), refer to these instructions.

    • If you’re upgrading a self-orchestrated deployment, refer to these instructions and upgrade the data nodes tier by tier in this order:

      1. Frozen tier
      2. Cold tier
      3. Warm tier
      4. Hot tier
      5. Any other nodes not in a tier
      6. All remaining nodes that are neither master-eligible nor data nodes
      7. Master-eligible nodes

  4. Upgrade Kibana. Refer to these instructions.

    If you’re using Elastic Cloud Hosted or Elastic Cloud Enterprise, this is already included in the Elasticsearch upgrade.

  5. Validate that Elasticsearch and Kibana are operating as expected by completing the following checks:

    1. For Elasticsearch:

      1. Check the status of your clusters and ensure that they’re green by running a GET _cat/health API request. For more information, refer to the cat health API documentation.

      2. Ensure that the index and search rate are close to what they were before upgrading. To view these, find Stack Monitoring in the navigation menu or by using the global search field, then select Elasticsearch → Overview.

        You can also check the index document count using the cat index API.

      3. Verify that snapshot lifecycle management SLM is taking snapshots by checking the SLM history.
      4. If you use machine learning, ensure that it is up and running.

    2. For Kibana:

      1. Ensure that you and your users can successfully log in to Kibana and access desired pages.
      2. Check Discover and verify that the index patterns you typically use are available.
      3. Verify that your commonly used dashboards are available and working properly.
      4. If you use any Watcher-based Kibana scheduled reporting, ensure that it’s working properly.
  6. Upgrade your ingest components (such as Logstash, Fleet and Elastic Agent, Beats, etc.). For details, refer to the Elastic Stack upgrade docs.

  7. Validate that ingest is operating correctly.

    1. Open Discover, go through data views for each of your expected ingest data streams, and ensure that data is being ingested in the expected format and volume.

  8. Validate that Elastic Security is operating correctly.

    1. On the Rules page, re-enable your desired SIEM detection rules (Rule Management tab), and ensure that enabled rules are running without errors or warnings (Rule Monitoring tab).
    2. Ensure that any SOAR workflows that consume alerts are working.
    3. Verify that any custom dashboards your team has created are working properly, especially if they operate on alert documents.
  9. If you performed these steps on a non-production deployment, repeat these same steps on your production environment. If you’re using a cross-cluster search environment and performed these steps on your remote clusters, repeat these same steps on your other deployments.
  10. Confirm with your appropriate stakeholders that the upgrade process has been successful.

Post-upgrade steps

edit

The following sections describe procedures to complete after upgrading Elastic Security to 8.x.

Re-enable disabled rules

edit

Any active rules when you upgrade from 7.17 to 8.0.1 or newer are automatically disabled, and a tag named auto_disabled_8.0 is added to those rules for tracking purposes. Once the upgrade is complete, you can filter rules by the new tag, then use bulk actions to re-enable them:

  1. Find Detection rules (SIEM) in the navigation menu or by using the global search field.
  2. From the Tags dropdown, search for auto_disabled_8.0.
  3. Click Select all x rules, or individually select the rules you want to re-enable.
  4. Click Bulk actions → Enable to re-enable the rules.

Alternatively, you can use the Bulk rule actions API to re-enable rules.

Full Disk Access (FDA) approval for Elastic Endpoint

edit

When you manually install Elastic Endpoint, you must approve a system extension, kernel extension, and enable Full Disk Access (FDA). There is a new FDA requirement in 8.x. Refer to Elastic Defend requirements to review the required permissions.

Requirements to display Data views in the Elastic Security app

edit

To make the Data view option appear in an environment with legacy alerts, a user with elevated role privileges must visit the Elastic Security app, open a page that displays alert data (at least one alert must be present), and then refresh the page. The user’s role privileges must allow them to enable the detections feature in a Kibana space. For more information, refer to Enable and access detections.

New alert schema

edit

The system index for detection alerts has been renamed from .siem-signals-<space-id> to .alerts-security.alerts-<space-id> and is now a hidden index. Therefore, the schema used for alert documents in Elastic Security has changed. Users that access documents in the .siem-signals indices using the Elastic Security API must modify their API queries and scripts to operate properly on the new 8.x alert documents. Refer to how to query alert indices and review the new Alert schema.

New privileges required to view alerts and preview rules

edit
  • To view alerts, users need manage, write, read, and view_index_metadata privileges for two new indices, .alerts-security.alerts and .internal.alerts-security.alerts. Existing users who are upgrading to 8.x can retain their privileges to the .siem-signals index.
  • To preview rules, users need read access to the new .preview.alerts-security.alerts index. Refer to Detections requirements for more information.

Updates to indicator match rules

edit

Changes to the indicator match rule type’s default threat indicator path might require you to update existing rules or create new ones after upgrading to 8.x. Be mindful of the following:

  • If an indicator match rule’s default threat indicator path was not defined before the upgrade, it will default to threatintel.indicator after the upgrade. This allows the rule to continue using indicator data ingested by Filebeat version 7.x. If a custom value was defined before the upgrade, the value will not change.
  • If an existing indicator match rule was configured to use threat indicator indices generated from Filebeat version 7.x, updating the default threat indicator path to threat.indicator after you upgrade to Elastic Stack version 8.x and Elastic Agent or Filebeat version 8.x configures the rule to use threat indicator indices generated by those later versions.
  • You must create separate rules to query threat intelligence indices created by Filebeat version 7.x and version 8.x because each version requires a different default threat indicator path value. Review the recommendations for querying alert indices.
« Upgrade Elastic Security to 8.16.0 Post-upgrade steps (optional) »