How to

SHA-512 checksums for Elastic Stack artifacts

By
Maxime Greau

Each time we do a release of the Elastic Stack, more than 100 binary artifacts are generated. Until now, we provided SHA-1 checksum files alongside those artifacts to verify file integrity. The first SHA-1 collision, announced this year, has increased the awareness to move to a safer alternative for all applications that rely on SHA-1 for digital signatures, file integrity, or file identification.

That's what we have done by using SHA-512 cryptographic hashes, from the SHA-2 family.

Elastic Stack 5.6.2+: SHA-1 and SHA-512 checksums

While we encourage you to move to the SHA-512 checksum files quickly, we still generate the SHA-1 checksums for 5.6.x releases for backward compatibility.

Elastic Stack 5.6.2 was the first version released with both SHA-1 and SHA-512 checksum files available. The same format is used to produce the contents of the .sha1 and .sha512 files.

Once you have downloaded one Elastic Stack artifact, e.g. Kibana 5.6.3, and its related checksum files from artifacts.elastic.co, you can check the file integrity:

$ curl -o kibana-5.6.3-linux-x86_64.tar.gz https://artifacts.elastic.co/downloads/kibana/kibana-5.6.3-linux-x86_64.tar.gz
$ curl -o kibana-5.6.3-linux-x86_64.tar.gz.sha512 https://artifacts.elastic.co/downloads/kibana/kibana-5.6.1-linux-x86_64.tar.gz.sha512

# Display the downloaded .sha512 file's content
$ cat kibana-5.6.3-linux-x86_64.tar.gz.sha512
fe957f0a2def9fb40be32ac77986d5eb7cc3373f828eed8eae074770ef965cf01c36028f2f9790f927e217ee59a283e63273eb768b9d2375aa0cdf15e4963c9a

# Generate the checksum locally with SHA-512 algorithm
$ shasum -a 512 kibana-5.6.3-linux-x86_64.tar.gz
fe957f0a2def9fb40be32ac77986d5eb7cc3373f828eed8eae074770ef965cf01c36028f2f9790f927e217ee59a283e63273eb768b9d2375aa0cdf15e4963c9a  kibana-5.6.3-linux-x86_64.tar.gz

Then you have to write a script to check that the checksum downloaded from artifacts.elastic.co is the same that the one generated locally, based on the binary artifact downloaded.

Elastic Stack 6.0.0 checksums: new SHA-512 format, no more SHA-1

With the upcoming Elastic Stack 6.0.0 release, we have decided to:

  1. No longer generate SHA-1 checksum files
  2. Generate SHA-512 checksum files with a format optimized for command line tools

You can already validate the integrity of Elastic Stack files with the new SHA-512 file format with the 6.0.0-rc2 artifacts released last week:

$ curl -s -o kibana-6.0.0-rc2-linux-x86_64.tar.gz https://artifacts.elastic.co/downloads/kibana/kibana-6.0.0-rc2-linux-x86_64.tar.gz
$ curl -s -o kibana-6.0.0-rc2-linux-x86_64.tar.gz.sha512 https://artifacts.elastic.co/downloads/kibana/kibana-6.0.0-rc2-linux-x86_64.tar.gz.sha512

$ ls
kibana-6.0.0-rc2-linux-x86_64.tar.gz        kibana-6.0.0-rc2-linux-x86_64.tar.gz.sha512

The new format, used by .sha512 files, contains the hash value and the artifact's filename associated on the same line (separated by two spaces):

# Display the downloaded .sha512 file's content new format 
$ cat kibana-6.0.0-rc2-linux-x86_64.tar.gz.sha512
318cd48d9f7aced508f8bb5aebf59d8d0a97696a26ce748d7381883059ba4ced45f3ebcb833e0e1b39746b249b9cdbc3c9186988c4f74dbf5263a81b658663ce  kibana-6.0.0-rc2-linux-x86_64.tar.gz

Now, with this format, it's easier to check the integrity of a downloaded Elastic Stack artifact, by simply using the -c option:

# Read the checksum from the .sha512 file and check it with the artifact file 
$ shasum -a 512 -c kibana-6.0.0-rc2-linux-x86_64.tar.gz.sha512
kibana-6.0.0-rc2-linux-x86_64.tar.gz: OK

If the SHA-512 checksums matches, OK is printed along the filename, while a FAILED message will be printed to the standard output in case of error:

$ shasum -a 512 -c kibana-6.0.0-rc2-linux-x86_64.tar.gz.sha512
kibana-6.0.0-rc2-linux-x86_64.tar.gz: FAILED
shasum: WARNING: 1 computed checksum did NOT match

TIP: use the -s, --status option to suppress output and use the command status code to check success or failure.

Elasticsearch plugins

The Elasticsearch team has updated the plugin installation component that checks the integrity of the plugins files before installing it. All Elasticsearch official plugins have been updated with SHA-512 checksum files. The expected behavior is based on the above explanations in this article, so:

  • For versions 5.6.2+, Elasticsearch supports and expects SHA-512 with the file only containing the hash. These hashes are now required for official plugins, while for community plugins, SHA-1 is still supported.
  • For versions 6.0+, Elasticsearch supports and expects SHA-512 with the file containing the hash and the filename separated by two spaces (both elements are validated). SHA-1 support is still there for non-official plugins (file containing only the hash) and will be removed from major versions in the future when Maven Central will support SHA-512 checksums.

Conclusion

To summarize, we encourage that you always check the integrity of the files you download. If you already have a script doing that with the Elastic Stack artifacts, please update it to use the SHA-512 files and format we now provide for all Elastic releases.