IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Google Cloud Storage Repository Plugin Create a Repository »
Elastic Docs ›Elasticsearch Plugins and Integrations [5.2] ›Snapshot/Restore Repository Plugins ›Google Cloud Storage Repository Plugin

Getting started

edit

Getting started

edit

The plugin uses the Google Cloud Storage JSON API (v1) to connect to the Storage service. If this is the first time you use Google Cloud Storage, you first need to connect to the Google Cloud Platform Console and create a new project. Once your project is created, you must enable the Cloud Storage Service for your project.

Creating a Bucket

edit

Google Cloud Storage service uses the concept of Bucket as a container for all the data. Buckets are usually created using the Google Cloud Platform Console. The plugin will not automatically create buckets.

To create a new bucket:

  1. Connect to the Google Cloud Platform Console
  2. Select your project
  3. Got to the Storage Browser
  4. Click the "Create Bucket" button
  5. Enter a the name of the new bucket
  6. Select a storage class
  7. Select a location
  8. Click the "Create" button

The bucket should now be created.

Service Authentication

edit

The plugin supports two authentication modes:

Using Compute Engine

edit

When running on Compute Engine, the plugin use the Google’s built-in authentication mechanism to authenticate on the Storage service. Compute Engine virtual machines are usually associated to a default service account. This service account can be found in the VM instance details in the Compute Engine console.

To indicate that a repository should use the built-in authentication, the repository service_account setting must be set to _default_:

PUT _snapshot/my_gcs_repository_on_compute_engine
{
  "type": "gcs",
  "settings": {
    "bucket": "my_bucket",
    "service_account": "_default_"
  }
}

The Compute Engine VM must be allowed to use the Storage service. This can be done only at VM creation time, when "Storage" access can be configured to "Read/Write" permission. Check your instance details at the section "Cloud API access scopes".

Using a Service Account

edit

If your elasticsearch node is not running on Compute Engine, or if you don’t want to use Google built-in authentication mechanism, you can authenticate on the Storage service using a Service Account file.

To create a service account file: 1. Connect to the Google Cloud Platform Console 2. Select your project 3. Got to the Permission tab 4. Select the Service Accounts tab 5. Click on "Create service account" 6. Once created, select the new service account and download a JSON key file

A service account file looks like this:

{
  "type": "service_account",
  "project_id": "your-project-id",
  "private_key_id": "...",
  "private_key": "-----BEGIN PRIVATE KEY-----\n...\n-----END PRIVATE KEY-----\n",
  "client_email": "service-account-for-your-repository@your-project-id.iam.gserviceaccount.com",
  "client_id": "...",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://accounts.google.com/o/oauth2/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "..."
}

This file must be copied in the config directory of the elasticsearch installation and on every node of the cluster.

To indicate that a repository should use a service account file:

PUT _snapshot/my_gcs_repository
{
  "type": "gcs",
  "settings": {
    "bucket": "my_bucket",
    "service_account": "service_account.json"
  }
}

Set Bucket Permission

edit

The service account used to access the bucket must have the "Writer" access to the bucket:

  1. Connect to the Google Cloud Platform Console
  2. Select your project
  3. Got to the Storage Browser
  4. Select the bucket and "Edit bucket permission"
  5. The service account must be configured as a "User" with "Writer" access
« Google Cloud Storage Repository Plugin Create a Repository »