IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Installing the Elastic Stack
Elastic Docs ›Installation and Upgrade Guide [6.3]

Upgrading the Elastic Stack

edit

Upgrading the Elastic Stack

edit

When upgrading to a new version of Elasticsearch, you need to upgrade each of the products in your Elastic Stack. Beats and Logstash 5.6 are compatible with Elasticsearch 6.3.2 to give you flexibility in scheduling the upgrade.

The steps you need to take to upgrade differ depending on which products you are using. Want a list that’s tailored to your stack? Try out our Interactive Upgrade Guide.

If you are running a pre-6.0 version, we recommend upgrading to the most recent 5.6 before upgrading to 6.3.2. X-Pack 5.6 provides a free Upgrade Assistant that identifies issues you need to address before upgrading and simplifies migrating indices that need to be reindexed before you upgrade. The Upgrade Assistant is enabled with both Trial and Basic licenses. You can install X-Pack solely for the purpose of upgrading. In 6.3 and later, X-Pack is automatically installed when you install Elasticsearch, Kibana, and Logstash.

Rolling upgrades are supported when upgrading from Elasticsearch 5.6 and Elasticsearch 6.0-6.2 to 6.3.2. Upgrading from any version prior to 5.6 requires a full cluster restart.

  • 6.3.0 and 6.3.1 might disable security for trial licenses. If you have a 6.2.x or earlier cluster with a trial license, X-Pack security is disabled when you upgrade to 6.3.0 or 6.3.1. See https://www.elastic.co/blog/elastic-stack-6-3-0-and-6-3-1-may-disable-security-for-trial-licenses
  • 2.x indices are not compatible with 6.3.2. You must remove or reindex them on your 5.n cluster before upgrading to 6.3.2. The internal Kibana and X-Pack indices and the default Beats and Logstash mapping templates also need to be updated to work with 6.3.2.

Preparing to upgrade

edit

Before upgrading the Elastic Stack to 6.3.2:

  1. Back up your data. You cannot roll back to an earlier version unless you have a backup of your data. For information about creating snapshots, see Snapshot and Restore.
  2. Check the Elasticsearch deprecation log to see if you’re using any deprecated features and update your code accordingly. By default, deprecation log messages are enabled at the WARN level.

  3. Review the breaking changes for each product you use and make the necessary changes so your code is compatible with 6.3.2:

    • If you’re upgrading from 2.n, make sure you check the breaking changes from 2.n to 5.n, as well as from 5.n to 6.n!
    • If you are using machine learning datafeeds that contain discontinued search or query domain specific language (DSL), the upgrade will fail. In 5.6.5 and later, the Upgrade Assistant provides information about which datafeeds need to be updated.
  4. Reindex or delete any indices created on 2.n. We recommend upgrading to the most recent 5.6 and using the X-Pack Reindex Helper to reindex 2.n indices.
  5. If Kibana and X-Pack are part of your stack, upgrade the internal Kibana and X-Pack indices. We recommend using the X-Pack 5.6 Reindex Helper to upgrade the internal indices. If you’re performing a full cluster restart upgrade from an earlier version, you can also use the _xpack/migration/upgrade API directly to upgrade the internal indices after you install Elasticsearch 6.3.2.

  6. If you use Elastic Stack security features to secure your cluster:

    1. Make sure TLS is enabled to encrypt communications between nodes. TLS must be enabled to upgrade to 6.3.2. For more information, see Encrypting communications.

      Enabling TLS requires a full cluster restart. Nodes that have TLS enabled cannot communicate with nodes that do not have TLS enabled. You must restart all nodes to maintain communication across the cluster.

    2. Make sure real passwords are configured for the built-in elasticsearch, kibana, and logstash_system users. They cannot use the 5.n default password (changeme). For more information, see Built-in users.
  7. Stop any X-Pack machine learning jobs that are running before starting the upgrade process. See Stopping machine learning.

Test upgrades in a dev environment before upgrading your production cluster.

Upgrade order

edit

Upgrade the Elastic Stack products you use in the following order:

  1. Elasticsearch Hadoop: install instructions
  2. Elasticsearch: upgrade instructions
  3. Kibana: upgrade instructions
  4. Logstash: upgrade instructions
  5. Beats: upgrade instructions

Logstash 5.6 and 6.n and Beats 5.6 and 6.n are compatible with all 6.n versions of Elasticsearch. This provides flexibility in when you schedule the upgrades for your Logstash instances and Beats agents. We recommend upgrading Logstash and Beats as soon as possible to take advantage of performance improvements and other enhancements.

Upgrading to 6.3

edit

Starting in 6.3, the default distributions of Elasticsearch, Logstash, and Kibana include X-Pack and a free Basic license that never expires.

You can perform rolling upgrades to 6.3 from OSS-only clusters running 5.6 or 6.0-6.2. Basic features are operational once the cluster is fully upgraded. If you are already using X-Pack, your settings are preserved when you upgrade.

If you are using X-Pack for the first time, you must explicitly enable data collection after the upgrade to use monitoring. Set xpack.monitoring.collection.enabled to true with the _cluster/settings API:

PUT /_cluster/settings
{
    "persistent" : {
        "xpack.monitoring.collection.enabled" : "true"
    }
}

To take more of the Elastic Stack features for a spin, you can start a 30-day trial from Kibana or with the Start Trial API:

POST _xpack/license/start_trial

The 30-day trial enables you to try out the full set of platinum features, including security, machine learning, alerting, graph capabilities, and more.

Upgrading from 5.6

edit

X-Pack 5.6 provides migration and upgrade APIs for Elasticsearch and a Upgrade Assistant UI for Kibana. These tools are included with the trial license and the free basic license.

To upgrade to 6.3.2 from 5.6:

  1. Upgrade Elasticsearch to the most recent 5.6 and install X-Pack on all nodes in your cluster. If you are upgrading from an earlier 5.x release, you can perform a rolling upgrade. To upgrade from older versions you must perform a full cluster restart.

    If your trial license expires, register for a free Basic license. To apply the license, upload the license file with the license API:

    license -d @license.json

  2. If X-Pack IS NOT normally a part of your Elastic Stack, disable the Elasticsearch security features in elasticsearch.yml:

    xpack.security.enabled: false
  3. Upgrade Kibana to the most recent 5.6 and install X-Pack.

  4. If you disabled the Elasticsearch security features, also disable the Kibana security features in kibana.yml:

    xpack.security.enabled: false

  5. Use the Upgrade Assistant in Kibana to view incompatibilities that you need to fix, identify any 2.x indices that need to be migrated or deleted, and upgrade the internal indices to the 6.x index format.

    You can also call the Elasticsearch migration APIs directly:

    /_xpack/migration/assistance
    Runs a series of checks on your cluster, nodes, and indices and returns a list of issues that need to be fixed before you can upgrade to 6.3.2.
    /_xpack/migration/upgrade
    Upgrades the Watcher and Security indices to a single-type format compatible with Elasticsearch 6.x.
  6. Once you’ve resolved all of the migration issues, perform a rolling upgrade from Elasticsearch 5.6 to 6.3.2.

Upgrading from a pre-5.6 installation

edit

It is possible to upgrade directly to 6.x from a pre-5.6 installation, but it requires a full cluster restart and you must manually reindex any 2.x indices you need to carry forward to 6.x.

If you use Kibana or X-Pack, you also need to upgrade the internal Kibana and X-Pack indices. For information about upgrading them after you install Elasticsearch 6.3.2, see Upgrading internal indices.

To manually reindex a 2.x index:

  1. Create an index with 6.x compatible mappings.
  2. Use the reindex API to copy documents from the 2.x index into the new index. You can use a script to perform any necessary modifications to the document data and metadata during reindexing.
  3. Use the _aliases API to add the name of the 2.x index as alias for the new index and delete the 2.x index.

Upgrading internal indices for 6.x

edit

The format used for the internal indices used by Kibana and X-Pack has changed in 6.x. Before you can run Kibana and X-Pack in 6.3.2, these indices must be upgraded to the new format. If you are upgrading from a version prior to 5.6, you must upgrade them after after installing Elasticsearch 6.3.2.

To get a list of the indices that need to be upgraded, install X-Pack and use the _xpack/migration/assistance API:

GET /_xpack/migration/assistance

To upgrade the .security index:

  1. On a single node, add a temporary superuser account to the file realm.

  2. Use the _xpack/migration/upgrade API to upgrade the security index, submitting the request with the credentials for the temporary superuser:

    POST /_xpack/migration/upgrade/.security
  3. Delete the temporary superuser account from the file realm.

You can use your regular administration credentials to upgrade the other internal indices using the _xpack/migration/upgrade API.

Once you upgrade the .kibana index, you can run Kibana and use the X-Pack Reindex Helper UI to upgrade the other indices.

Upgrading on Elastic Cloud

edit

A single click in the Elastic Cloud console can upgrade a cluster to a newer version, add more processing capacity, change plugins, and enable or disable high availability, all at the same time. During the upgrade process, Elasticsearch, Kibana, X-Pack and the officially included plugins are upgraded simultaneously.

Although upgrading your Elastic Cloud clusters is easy, you still need to address breaking changes that affect your application. Minor version upgrades, upgrades from 5.6 to 6.x, and all other cluster configuration changes can be performed with no downtime.

To avoid downtime when a full cluster restart is required:

  1. Provision an additional cluster with the new Elasticsearch version, reindex your data, and send index requests to both clusters temporarily.
  2. Verify that the new cluster performs as expected, fix any problems, and then permanently swap in the new cluster.
  3. Delete the old cluster to stop incurring additional costs. You are billed only for the time that the new cluster runs in parallel with your old cluster. Usage is billed on an hourly basis.

To learn more about the upgrade process on Elastic Cloud, see Upgrade versions.

Elastic Cloud only supports upgrades to released versions. Preview releases and master snapshots are not supported.

« Installing the Elastic Stack