Azure repository

To enable Azure repositories, you have first to define your Azure storage settings as secure settings.

You can define these settings before the node is started, or call the Nodes reload secure settings API after the settings are defined to apply them to a running node.

bin/elasticsearch-keystore add azure.client.default.account
bin/elasticsearch-keystore add azure.client.default.key

Note that you can also define more than one account:

bin/elasticsearch-keystore add azure.client.default.account
bin/elasticsearch-keystore add azure.client.default.key
bin/elasticsearch-keystore add azure.client.secondary.account
bin/elasticsearch-keystore add azure.client.secondary.sas_token

For more information about these settings, see Client settings.

The client that you use to connect to Azure has a number of settings available. The settings have the form azure.client.CLIENT_NAME.SETTING_NAME. By default, azure repositories use a client named default, but this can be modified using the repository setting client. For example:

resp = client.snapshot.create_repository(
    name="my_backup",
    repository={
        "type": "azure",
        "settings": {
            "client": "secondary"
        }
    },
)
print(resp)

const response = await client.snapshot.createRepository({
  name: "my_backup",
  repository: {
    type: "azure",
    settings: {
      client: "secondary",
    },
  },
});
console.log(response);

PUT _snapshot/my_backup
{
  "type": "azure",
  "settings": {
    "client": "secondary"
  }
}

Most client settings can be added to the elasticsearch.yml configuration file. For example:

azure.client.default.timeout: 10s
azure.client.default.max_retries: 7
azure.client.default.endpoint_suffix: core.chinacloudapi.cn
azure.client.secondary.timeout: 30s

In this example, the client side timeout is 10s per try for the default account with 7 retries before failing. The endpoint suffix is core.chinacloudapi.cn and 30s per try for the secondary account with 3 retries.

The account, key, and sas_token storage settings are reloadable secure settings, which you add to the Elasticsearch keystore. For more information about creating and updating the Elasticsearch keystore, see Secure settings. After you reload the settings, the internal Azure clients, which are used to transfer the snapshot, utilize the latest settings from the keystore.

The following list contains the available client settings. Those that must be stored in the keystore are marked as "secure"; the other settings belong in the elasticsearch.yml file.

The Azure repository supports following settings:

Some examples, using scripts:

# The simplest one
PUT _snapshot/my_backup1
{
  "type": "azure"
}

# With some settings
PUT _snapshot/my_backup2
{
  "type": "azure",
  "settings": {
    "container": "backup-container",
    "base_path": "backups",
    "chunk_size": "32MB",
    "compress": true
  }
}


# With two accounts defined in elasticsearch.yml (my_account1 and my_account2)
PUT _snapshot/my_backup3
{
  "type": "azure",
  "settings": {
    "client": "secondary"
  }
}
PUT _snapshot/my_backup4
{
  "type": "azure",
  "settings": {
    "client": "secondary",
    "location_mode": "primary_only"
  }
}

Example using Java:

client.admin().cluster().preparePutRepository("my_backup_java1")
    .setType("azure").setSettings(Settings.builder()
        .put(Storage.CONTAINER, "backup-container")
        .put(Storage.CHUNK_SIZE, new ByteSizeValue(32, ByteSizeUnit.MB))
    ).get();

According to the containers naming guide, a container name must be a valid DNS name, conforming to the following naming rules:

The linearizable register implementation for Azure repositories is based on Azure’s support for strongly consistent leases. Each lease may only be held by a single node at any time. The node presents its lease when performing a read or write operation on a protected blob. Lease-protected operations fail if the lease is invalid or expired. To perform a compare-and-exchange operation on a register, Elasticsearch first obtains a lease on the blob, then reads the blob contents under the lease, and finally uploads the updated blob under the same lease. This process ensures that the read and write operations happen atomically.