- Elastic Cloud Enterprise - Elastic Cloud on your Infrastructure: other versions:
- Introducing Elastic Cloud Enterprise
- Preparing your installation
- Installing Elastic Cloud Enterprise
- Identify the deployment scenario
- Install ECE on a public cloud
- Install ECE on your own premises
- Alternative: Install ECE with Ansible
- Log into the Cloud UI
- Install ECE on additional hosts
- Migrate ECE to Podman hosts
- Post-installation steps
- Configuring your installation
- System deployments configuration
- Configure deployment templates
- Tag your allocators
- Edit instance configurations
- Create instance configurations
- Create deployment templates
- Configure system deployment templates
- Configure index management for templates
- Updating custom templates to support
node_roles
and autoscaling - Updating custom templates to support Integrations Server
- Default instance configurations
- Include additional Kibana plugins
- Manage snapshot repositories
- Manage licenses
- Change the ECE API URL
- Change endpoint URLs
- Enable custom endpoint aliases
- Configure allocator affinity
- Change allocator disconnect timeout
- Migrate ECE on Podman hosts to SELinux in
enforcing
mode
- Securing your installation
- Monitoring your installation
- Administering your installation
- Working with deployments
- Create a deployment
- Access Kibana
- Adding data to Elasticsearch
- Migrating data
- Ingesting data from your application
- Ingest data with Node.js on Elastic Cloud Enterprise
- Ingest data with Python on Elastic Cloud Enterprise
- Ingest data from Beats to Elastic Cloud Enterprise with Logstash as a proxy
- Ingest data from a relational database into Elastic Cloud Enterprise
- Ingest logs from a Python application using Filebeat
- Ingest logs from a Node.js web application using Filebeat
- Manage data from the command line
- Administering deployments
- Change your deployment configuration
- Maintenance mode
- Terminate a deployment
- Restart a deployment
- Restore a deployment
- Delete a deployment
- Migrate to index lifecycle management
- Disable an Elasticsearch data tier
- Access the Elasticsearch API console
- Work with snapshots
- Restore a snapshot across clusters
- Upgrade versions
- Editing your user settings
- Deployment autoscaling
- Configure Beats and Logstash with Cloud ID
- Keep your clusters healthy
- Keep track of deployment activity
- Secure your clusters
- Deployment heap dumps
- Deployment thread dumps
- Traffic Filtering
- Connect to your cluster
- Manage your Kibana instance
- Manage your APM & Fleet Server (7.13+)
- Manage your APM Server (versions before 7.13)
- Manage your Integrations Server
- Switch from APM to Integrations Server payload
- Enable logging and monitoring
- Enable cross-cluster search and cross-cluster replication
- Access other deployments of the same Elastic Cloud Enterprise environment
- Access deployments of another Elastic Cloud Enterprise environment
- Access deployments of an Elasticsearch Service organization
- Access clusters of a self-managed environment
- Enabling CCS/R between Elastic Cloud Enterprise and ECK
- Edit or remove a trusted environment
- Migrate the cross-cluster search deployment template
- Enable App Search
- Enable Enterprise Search
- Enable Graph (versions before 5.0)
- Troubleshooting
- RESTful API
- Authentication
- API calls
- How to access the API
- API examples
- Setting up your environment
- A first API call: What deployments are there?
- Create a first Deployment: Elasticsearch and Kibana
- Applying a new plan: Resize and add high availability
- Updating a deployment: Checking on progress
- Applying a new deployment configuration: Upgrade
- Enable more stack features: Add Enterprise Search to a deployment
- Dipping a toe into platform automation: Generate a roles token
- Customize your deployment
- Remove unwanted deployment templates and instance configurations
- Secure your settings
- API reference
- Changes to index allocation and API
- Script reference
- Release notes
- Elastic Cloud Enterprise 3.7.3
- Elastic Cloud Enterprise 3.7.2
- Elastic Cloud Enterprise 3.7.1
- Elastic Cloud Enterprise 3.7.0
- Elastic Cloud Enterprise 3.6.2
- Elastic Cloud Enterprise 3.6.1
- Elastic Cloud Enterprise 3.6.0
- Elastic Cloud Enterprise 3.5.1
- Elastic Cloud Enterprise 3.5.0
- Elastic Cloud Enterprise 3.4.1
- Elastic Cloud Enterprise 3.4.0
- Elastic Cloud Enterprise 3.3.0
- Elastic Cloud Enterprise 3.2.1
- Elastic Cloud Enterprise 3.2.0
- Elastic Cloud Enterprise 3.1.1
- Elastic Cloud Enterprise 3.1.0
- Elastic Cloud Enterprise 3.0.0
- Elastic Cloud Enterprise 2.13.4
- Elastic Cloud Enterprise 2.13.3
- Elastic Cloud Enterprise 2.13.2
- Elastic Cloud Enterprise 2.13.1
- Elastic Cloud Enterprise 2.13.0
- Elastic Cloud Enterprise 2.12.4
- Elastic Cloud Enterprise 2.12.3
- Elastic Cloud Enterprise 2.12.2
- Elastic Cloud Enterprise 2.12.1
- Elastic Cloud Enterprise 2.12.0
- Elastic Cloud Enterprise 2.11.2
- Elastic Cloud Enterprise 2.11.1
- Elastic Cloud Enterprise 2.11.0
- Elastic Cloud Enterprise 2.10.1
- Elastic Cloud Enterprise 2.10.0
- Elastic Cloud Enterprise 2.9.2
- Elastic Cloud Enterprise 2.9.1
- Elastic Cloud Enterprise 2.9.0
- Elastic Cloud Enterprise 2.8.1
- Elastic Cloud Enterprise 2.8.0
- Elastic Cloud Enterprise 2.7.2
- Elastic Cloud Enterprise 2.7.1
- Elastic Cloud Enterprise 2.7.0
- Elastic Cloud Enterprise 2.6.2
- Elastic Cloud Enterprise 2.6.1
- Elastic Cloud Enterprise 2.6.0
- Elastic Cloud Enterprise 2.5.1
- Elastic Cloud Enterprise 2.5.0
- Elastic Cloud Enterprise 2.4.3
- Elastic Cloud Enterprise 2.4.2
- Elastic Cloud Enterprise 2.4.1
- Elastic Cloud Enterprise 2.4.0
- Elastic Cloud Enterprise 2.3.2
- Elastic Cloud Enterprise 2.3.1
- Elastic Cloud Enterprise 2.3.0
- Elastic Cloud Enterprise 2.2.3
- Elastic Cloud Enterprise 2.2.2
- Elastic Cloud Enterprise 2.2.1
- Elastic Cloud Enterprise 2.2.0
- Elastic Cloud Enterprise 2.1.1
- Elastic Cloud Enterprise 2.1.0
- Elastic Cloud Enterprise 2.0.1
- Elastic Cloud Enterprise 2.0.0
- Elastic Cloud Enterprise 1.1.5
- Elastic Cloud Enterprise 1.1.4
- Elastic Cloud Enterprise 1.1.3
- Elastic Cloud Enterprise 1.1.2
- Elastic Cloud Enterprise 1.1.1
- Elastic Cloud Enterprise 1.1.0
- Elastic Cloud Enterprise 1.0.2
- Elastic Cloud Enterprise 1.0.1
- Elastic Cloud Enterprise 1.0.0
- What’s new with the Elastic Stack
- About this product
Snapshotting to Minio on-premise storage
editSnapshotting to Minio on-premise storage
editMinio is a popular, open-source distributed object storage server compatible with the Amazon AWS S3 API. You can use it with Elastic Cloud Enterprise installations when you want to store your Elasticsearch snapshots locally.
Create a test environment
editWe recommend following the Minio Quickstart Guide Docker Container instructions to create a simple Minio standalone installation for your initial evaluation and development.
Be sure to use the docker -v
option to map persistent storage to the container.
Production environment prerequisites
editInstalling Minio for production requires a high-availability configuration where Minio is running in Distributed mode.
As mentioned in the Minio documentation, you will need to have 4-16 Minio drive mounts. There is no hard limit on the number of Minio nodes. It might be convenient to place the Minio node containers on your ECE hosts to ensure you have a suitable level of availability, but those can not be located on the same hosts as ECE proxies since they both listen on the same port.
The following illustration is a sample architecture for a large ECE installation. Note that there is at least one MinIO container in each availability zone.
There are a number of different ways of orchestrating the Minio deployment (Docker Compose, Kubernetes, and so on). We suggest you use the method most familiar to you.
We recommend:
- Using a single Minio endpoint with the Elastic Cloud Enterprise installation, to simplify repository management.
- Securing access to the Minio endpoint with TLS.
Create an offline installation
editIf you are installing MinIO offline, the process is very similar to the offline installation of Elastic Cloud Enterprise. There are two options:
- Use a private Docker repository and install the Minio images in the private repository.
-
Download the Minio images from an internet-connected machine, then use docker save to bundle the images into tar files. Copy the TAR files to the target hosts and use
docker load
to install.
Gather the following after your installation:
- Minio AccessKey
- Minio SecretKey
- Endpoint URL
Minio might report various Endpoint URLs, be sure to choose the one that will be routable from your Elasticsearch Docker containers.
Create the S3 bucket
editHow you create the AWS S3 bucket depends on what version of Elasticsearch you are using:
- For version 5.x, installing the S3 repository plugin creates an S3 bucket for you.
-
For versions 6.0–7.17:
- Using the Minio browser or an S3 client application, create an S3 bucket to store your snapshots. TIP: Don’t forget to make the bucket name DNS-friendly, e.g. no underscores or uppercase letters. For more details, read the bucket restrictions.
- Log into the Cloud UI and add the S3 repository plugin to your cluster.
- For versions 8.0 and later, Elasticsearch has built-in support for AWS S3 repositories; no repository plugin is needed. Use the Minio browser or an S3 client application to create an S3 bucket to store your snapshots. TIP: Don’t forget to make the bucket name DNS-friendly, for example no underscores or uppercase letters. For more details, read the bucket restrictions.
Elastic Cloud Enterprise configuration
editYou can configure existing deployments, or create new ones, with the following changes to use Minio storage.
Add the repository to Elastic Cloud Enterprise
editYou must add the new repository to Elastic Cloud Enterprise before it can be used with your Elasticsearch clusters.
- Log into the Cloud UI.
- From the Platform menu, select Repositories.
- Select Add Repository.
- From the Repository Type drop-down list, select Advanced.
-
In the Configuration text area, provide the repository JSON. You must specify the bucket, access_key, secret_key, endpoint, and protocol.
{ "type": "s3", "settings": { "bucket": "ece-backup", "access_key": "<your Minio AccessKey>", "secret_key": "<your Minio SecretKey>", "endpoint": "<your Minio endpoint URL>:9000", "path_style_access": "true", "protocol": "http" } }
- Select Save to submit your configuration.
The Minio repository is now available from the drop-down list of repositories when creating deployments.
Additional settings for 6.x clusters
editFor Elasticsearch versions 6.0 and later, after selecting the repository, you also need to set your User Settings YAML to specify the endpoint and protocol. For example:
s3.client.default.endpoint: "<your Minio endpoint>:9000" s3.client.default.protocol: http
Check the Elasticsearch S3 plugin details for more information.
Upgrade from 5.x to 6.x Elasticsearch clusters
editThe configuration options for the Elasticsearch S3 repository plugin have changed from 5.x to 6.x versions and you must copy the endpoint and protocol values from your repository configuration to your User Settings YAML before you upgrade.
Verify snapshots
editThe cluster should make a snapshot when the repository is set up. You can check that by going to the Elasticsearch and then the Snapshots page.
As an extra verification step, you can restore a cluster using the snapshots that have been taken.
- Log into the Cloud UI.
-
Get the plan from your test cluster.
-
From the Deployments page, select your deployment.
Narrow the list by name, ID, or choose from several other filters. To further define the list, use a combination of filters.
- From your deployment menu, go to the Edit page then go to the bottom of the page and select advanced Elasticsearch configuration.
- Copy the JSON format under the Deployment configuration heading.
-
- Create a new Elasticsearch cluster as your target.
-
On the new cluster, open the advanced cluster configuration editor. In the transient section, add the
restore_snapshot
settings to the plan.... "transient": { "restore_snapshot": { "repository_name": "<Minio repository name>", "snapshot_name": "latest_success" } }
- Select Save to restore from the snapshot. When the plan update is complete, you can check the restored indexes in your target cluster.
More details are available to work with snapshots.
On this page