Upgrading with the Persistent Queue Enabled

If you have the persistent queue (PQ) enabled, please read the section that applies for your upgrade scenario.

The following applies only if you are upgrading from Logstash version 6.2.x or earlier with the persistent queue (PQ) enabled.

We strive to maintain backward compatibility within a given major release. Serialization issues in Logstash 6.2.x and earlier required us to break that compatibility in version 6.3.0 to ensure correctness of operation. For more technical details, please check our tracking github issue for this matter, #9494.

We strongly recommend that you drain or delete the persistent queue before you upgrade from version 6.2.x and earlier.

To drain the queue:

When the queue is empty:

We have resolved issues with data incompatibilities for version 6.3 and later. These steps won’t be required for future upgrades.

Upgrading Logstash with persistent queues enabled is supported. The persistent queue directory is self-contained and can be read by a new Logstash instance running the same pipeline. You can safely shut down the original Logstash instance, spin up a new instance, and set path.queue in the logstash.yml settings file to point to the original queue directory. You can also use a mounted drive to make this workflow easier.

Keep in mind that only one Logstash instance can write to path.queue. You cannot have the original instance and the new instance writing to the queue at the same time.