Configure a snapshot repository using AWS S3

Configure a custom snapshot repository using an S3 storage bucket in your AWS account.

Prepare an S3 bucket

Create the S3 bucket in your custom AWS account. Make sure to reserve this bucket to backup only one cluster, since AWS allows file overwrite for non-unique titles.

Next, create an IAM user, copy the access key ID and secret, and configure the following user policy. This is important to make sure the access keys, which you will need to provide to your cluster, can only access the intended bucket.

{
  "Version": "policy-language-YYYY-MM-dd",
  "Statement": [
    {
      "Action": [
        "s3:*"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::bucket-name",
        "arn:aws:s3:::bucket-name/*"
      ]
    }
  ]
}

The version of the policy language syntax rules. For more information, refer to the AWS documentation.

For more information on S3 and IAM, refer to AWS' S3-documentation and IAM-documentation.

For a full list of settings that are supported for your S3 bucket, refer to S3 repository in the Elasticsearch Guide.

Store your secrets in the keystore

edit

You can use the Elasticsearch Add-On for Heroku Keystore to store the credentials to access your AWS account.

Log in to the Elasticsearch Add-On for Heroku console.
Navigate to the Security page of the deployment you wish to configure.
Locate Elasticsearch keystore and select Add settings.
With Type set to Single string, add the following keys and their values:
- s3.client.secondary.access_key
- s3.client.secondary.secret_key
Perform a cluster restart to reload the secure settings.

Create the repository

edit

Open Kibana and go to Management > Snapshot and Restore.
On the Repositories tab, select Register a repository.
Provide a name for your repository and select type AWS S3.
Provide the following settings:
- Client: secondary
- Bucket: YOUR_S3_BUCKET_NAME
Add any other settings that you wish to configure.
Select Register.
Select Verify to confirm that your settings are correct and the deployment can connect to your repository.

Your snapshot repository is now set up using S3! You can use Kibana to manage your snapshots and begin sending Elasticsearch snapshots to your own bucket. For details refer to the Snapshot and Restore documentation.

« Snapshot and restore with custom repositories Configure a snapshot repository using GCS »