Cluster management strategies such as adjusting availability zones and snapshot management can help you manage costs and ensure that your deployment configuration suits your use case.

Welcome to another blog in our series on cost management and optimisation in Elasticsearch Service. In previous installments, we looked at <a href="/blog/optimizing-costs-elastic-cloud-hot-warm-index-lifecycle-management">hot-warm architecture and index lifecycle management</a> as ways of managing the costs associated with data retention and at <a href="/blog/optimizing-costs-elastic-cloud-replica-shard-management">managing replicas</a> as a means of optimising the structure of your Elasticsearch Service deployment. Be sure to check out the other blogs in the series for additional tips to help you as you build out your deployment. And if you haven’t already, sign up to a <a href="/cloud/elasticsearch-service/signup">14-day free trial of Elasticsearch Service</a> on Elastic Cloud so you can follow along.
In this blog, we’ll look at cluster management strategies including adjusting availability zones and snapshot management.
Optimising the size and configuration of your deployment is crucial to ensuring your deployment suits your use case. For snapshot management in particular, being able to increase or decrease the frequency of snapshots depending on your use case can help you manage costs associated with snapshot storage and API requests. Furthermore, if you need to temporarily pause your deployment, knowing how to snapshot to an external repository is crucial.
All of these strategies can help you manage costs in Elastic Cloud. Balancing your architectural considerations between cost and resiliency is key to deploying these strategies. It is imperative to consider where and how they should be used to minimise the impact of any potential disruption.
Before we jump into snapshots, let’s take a brief look at adjusting availability zones for existing and new deployments.<h2>Adjusting availability zones</h2>Data centers can and do fail, so planning is crucial. In Elastic Cloud, you can always adjust the number of availability zones for your cluster, whether during a deployment or at the start of a deployment. When <a href="https://www.elastic.co/guide/en/cloud/current/ec-planning.html#ec-ha">planning for production</a> it is important to consider the number of availability zones you want. You can cut costs by reducing the number of availability zones, but you’ll also increase your risk of data loss. The best practice is to have at least two data centers for production and three for mission-critical systems. Single-zone architectures are suitable for development and test environments, but they are not recommended for production environments.
If you already have a deployment on Elastic Cloud, you can adjust the number of availability zones for all nodes (hot, warm, Kibana, machine learning, APM) by selecting the deployment from the <a href="https://cloud.elastic.co/home">Cloud Console</a> and clicking Edit in the menu on the left.
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltb77e303c0b135b76/5f2857e47f3b7908db030180/blog-cloud-costs-zones-edit.png" data-sys-asset-uid="bltb77e303c0b135b76" alt="" style="display: block; margin: auto;"> For brand new deployments, you can configure the number of availability zones at the bottom of the Create deployment screen by clicking Customize deployment at the bottom of the page. This will take you to a slider view similar to the image above where all nodes and availability zones can be adjusted as you wish.
 <h2>Managing snapshots</h2><a href="https://www.elastic.co/guide/en/cloud/current/ec-snapshot-restore.html">Snapshots</a> are important for backing up data stored in <a href="/cloud/">Elastic Cloud</a>. In addition to recovery from data center failure, snapshots can be used to bring up a new cluster for testing or recreate a deleted index.
For Elastic Stack versions 7.6 and later, you can use the&nbsp;<a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.7/snapshot-lifecycle-management.html">snapshot lifecycle management (SLM)</a> functionality in&nbsp;<a href="https://www.elastic.co/guide/en/kibana/7.7/snapshot-repositories.html">Kibana</a>. For older versions, you’ll need to use the&nbsp;<a href="https://www.elastic.co/guide/en/cloud/current/ec-restoring-snapshots.html">Elastic Cloud UI</a>.
The larger the volume of data stored in your deployment and the higher the frequency of snapshots, the more likely you are to incur costs related to snapshot storage. Regardless of your use case, you might find it beneficial to review our previous blogs on <a href="/blog/optimizing-costs-elastic-cloud-hot-warm-index-lifecycle-management">hot-warm and ILM</a> and <a href="/blog/optimizing-costs-elastic-cloud-replica-shard-management">replica management</a> while considering the following section.&nbsp;
If you’d like more information about costs as you look to adopt the best practices in this blog post, also check out our blog on <a href="/blog/elasticsearch-service-data-transfer-and-snapshot-storage-pricing">data transfer and snapshot storage pricing</a>.
<h2>Best practices for snapshotting</h2>Each deployment of the Elastic Stack is unique and has different requirements, so not all of the following recommendations may apply to you. However, the default settings are usually sufficient for most users. The settings can also be adjusted to fit your individual requirements.
The configuration options that can be adjusted are
<ul>
	<li>frequency by the schedule</li>
	<li>content by the indices included</li>
	<li>retention by age and count</li>
</ul>If the data set is relatively static, such as web pages about an organisation, snapshots can be&nbsp;<a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.7/trigger-schedule.html#schedule-cron">scheduled</a> to occur less frequently, such as once a day or every four hours. Even for time series data, such as logs, the frequency can be reduced as well. The logs may be available locally on application servers long enough to allow the logs to be ingested again to fill the gap between the last snapshot and now, so no data would be lost in the event of a failure.
Another option is to reduce the number of snapshots kept. With fewer snapshots kept, the storage space used is smaller, especially if the data changes frequently within the Elasticsearch cluster. If the snapshots are aged off more quickly and a number of indices are no longer contained in any snapshot, the disk space used by those indices will be released. If there is adequate monitoring and alerting in place to detect data problems quickly, it may not be necessary to restore more than a day or two back, allowing more of the older snapshots to be deleted.
Only a subset of the indices in a cluster may need to be backed up. For example, for observability, it’s common to snapshot very little or no metric data related to system health monitoring and retain the snapshots for a short period of time. The value of this data set significantly decreases as it gets older, so the cost of retaining it is often higher than the value it provides. Another example is to snapshot only the system indices storing Kibana dashboards in a development or non-production deployment but not the underlying business data, since the business data can be restored from another cluster such as the production cluster.
Finally, a combination of the three configuration options can be used in multiple&nbsp;<a href="https://www.elastic.co/guide/en/kibana/7.7/snapshot-repositories.html#kib-snapshot-policy">SLM policies</a> to snapshot different sets of indices at their own rate that are kept for a period of time. For example, a deployment used for order analytics may group indices into three different policies. Indices for small data sets that have a source of truth elsewhere and can be reloaded quickly may only be snapshotted once a week and retained for two weeks. A second group containing data that is not stored elsewhere, such as Kibana dashboards, saved queries, and machine learning models, is snapshotted every 30 minutes. And a third group of indices, including data that is also in an official system of record but can’t easily be ingested again due to the size of the data set, is snapshotted a few times a day.
The decision of how to best use the three control options requires careful analysis of acceptable risk and costs.
<h3>Snapshotting to a custom repository</h3>Another choice that can be made is where the Elasticsearch snapshots are stored. With Elasticsearch Service, there is the option to use a&nbsp;<a href="https://www.elastic.co/guide/en/cloud/current/ec-custom-repository.html">custom repository</a>. Depending on your existing cloud provider agreements, it may be more cost effective to use a repository from an existing account. The custom repository needs to be located in the same cloud provider and region as the Elasticsearch cluster to reduce data transfer costs between the cluster and the repository. A custom repository does not get around the limited set of Elasticsearch&nbsp;<a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.7/snapshot-restore.html#_version_compatibility">major versions</a> that each Elasticsearch version supports. It does, however, support longer-term storage of data that is common for security analytics and compliance use cases.When using a custom repository, the maintenance and monitoring is the responsibility of the user. Although the default Elastic-managed snapshot repository can be used in conjunction with a custom repository, you’ll need to ensure the two do not conflict with each other, as only&nbsp;<a href="https://www.elastic.co/guide/en/elasticsearch/reference/7.7/snapshots-monitor-snapshot-restore.html#_stop_snapshot_and_restore_operations">one snapshot</a> can be run at a time. Again, there are benefits, risks, and costs involved with where the snapshots are stored that will need to be analyzed for each deployment individually.
<h3>Review snapshots on Elastic Cloud</h3>Here’s how you can review your snapshots:
<ol>
	<li>Open your Elasticsearch Service deployment<ol>
		<li>Open the&nbsp;<a href="https://cloud.elastic.co/home">Elastic Cloud UI</a></li>
		<li>Click on Elasticsearch Service to view your&nbsp;<a href="https://cloud.elastic.co/deployments">deployments</a></li>
		<li>Click on the tile for the deployment</li>
	</ol></li>
	
	<li>On the left menu, click on Snapshots located under Deployments → {deployment name} → Elasticsearch<a href="https://github.com/elastic/consulting/blob/master/projects/blogs/images/cloud_7.7_deployment_overview.png"></a></li>
	<li>The display will differ slightly depending on the version of the Elasticsearch cluster. Somewhere around the middle of the top half of the page is a section with the status of the snapshots.<a href="https://github.com/elastic/consulting/blob/master/projects/blogs/images/cloud_7.7_snapshots_status.png"> </a></li>
</ol><img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltffcdba4d3daf7a1f/5f2858e6d9a24d7c67e4e2a9/blog-cloud-costs-zones-snapshots.png" data-sys-asset-uid="bltffcdba4d3daf7a1f" alt="" style="display: block; margin: auto;">The two key pieces of information here are the time of the last successful snapshot and the scheduled time for the next snapshot. The last successful snapshot time tells you whether the snapshots are working and how much data could be lost on failure. The next snapshot time indicates when the process is scheduled to run again to create a new snapshot.An active and successful snapshot process is important for protecting the data stored in an Elasticsearch cluster. You’ve indexed your data into Elasticsearch because it matters, so it’s critical to ensure that snapshots are properly set up and functioning.
<h3>Snapshot lifecycle management (SLM)</h3>If your deployment is version 7.6 or later, you’ll manage your snapshots using&nbsp;<a href="https://www.elastic.co/guide/en/kibana/7.7/snapshot-repositories.html">Kibana</a>.
To modify the frequency and retention of snapshots, click Snapshot and Restore, which will open Kibana in a new window. Kibana will open to the Snapshot and Restore page in&nbsp;<a href="https://www.elastic.co/guide/en/kibana/7.7/management.html">Management</a>. Click on Policies.Locate the cloud-snapshot-policy row and click on the pencil icon on the right to edit the policy.This will open a new page that will guide you through&nbsp;<a href="https://www.elastic.co/guide/en/kibana/7.7/snapshot-restore-tutorial.html#create-policy-example">a multi-step process</a> for adjusting the frequency at which snapshots are taken and the retention period of the snapshots. Both of these factors have an impact on the costs associated with snapshot storage. It is also possible to apply the settings to different indices, which gives you a finer level of granularity when it comes to managing these settings for an individual index or groups of indices.&nbsp;
The other option for modifying the frequency of snapshots from Elastic Cloud is to click Modify frequency, which will directly open the Policies tab of the Snapshot and Restore page.<a href="https://github.com/elastic/consulting/blob/master/projects/blogs/images/cloud_7.7_snapshots_modify-frequency.png"> </a>
If you need to restore data from a snapshot, you’ll do this on the Snapshots tab. Click the restore icon to the right of the snapshot name.<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltc2a4522a17a061f0/5f28597f7f3b7908db030186/blog-cloud-costs-zones-restore.png" data-sys-asset-uid="bltc2a4522a17a061f0" alt="" style="display: block; margin: auto;">This will open a new page that will guide you through&nbsp;<a href="https://www.elastic.co/guide/en/kibana/7.7/snapshot-restore-tutorial.html#restore-snapshot-example">a multi-step process</a> for restoring from a snapshot. If only a subset of indices needs to be restored, click the slider under the Indices section to select a subset of the indices from a snapshot.<a href="https://github.com/elastic/consulting/blob/master/projects/blogs/images/kibana_7.7_snapshot_restore-indices.png"> </a>
<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/bltc2b93ed288c1d1b2/5f2859c5d9a24d7c67e4e2ad/blog-cloud-costs-zones-subset.png" data-sys-asset-uid="bltc2b93ed288c1d1b2" alt="" style="display: block; margin: auto;"><h3>Elastic Cloud UI</h3>If your deployment is version 7.5 or older, you’ll manage your snapshots using the&nbsp;<a href="https://www.elastic.co/guide/en/cloud/current/ec-restoring-snapshots.html">Elasticsearch Service UI</a>.
To modify the frequency and retention of the snapshots, click Edit settings in the center of the page.<img src="https://images.contentstack.io/v3/assets/bltefdd0b53724fa2ce/blt591e8fefb6927b27/5f285a209af97004e7f61c44/blog-cloud-costs-zones-edit-settings.png" data-sys-asset-uid="blt591e8fefb6927b27" alt="" style="display: block; margin: auto;">A pop-up window will appear where you can view and modify the current snapshot interval and snapshot count to retain.<a href="https://github.com/elastic/consulting/blob/master/projects/blogs/images/cloud_6.8_snapshots_settings.png"> </a>
If you need to restore data from a snapshot, you can do so by clicking Restore to the right of the relevant snapshot name.On the fly-out window that appears on the right, there is an option to restore only a subset of indices from the snapshot.The other option for restoring from a snapshot is to click on the snapshot name, which will open a new page that lists the indices within the snapshot and presents the same form at the bottom of the page. <h2>Next steps</h2>There are a number of ways to manage and optimise your data within Elastic Cloud. An optimal setup will ensure that the costs associated with your deployment remain as low as possible. Be sure to check out the previous parts in this blog series for more on how <a href="/blog/optimizing-costs-elastic-cloud-hot-warm-index-lifecycle-management">hot-warm and ILM</a> and <a href="/blog/optimizing-costs-elastic-cloud-replica-shard-management">replica management</a> can help you optimise costs, as well as our previous blog on cost-saving strategies&nbsp;<a href="/blog/cost-saving-strategies-for-the-elasticsearch-service-data-storage-efficiency">in Elastic Cloud</a>.
If you’re interested in more advanced management techniques, we offer a variety of <a href="/training/">Elastic Training</a> courses, including <a href="/training/elasticsearch-engineer-1">Elasticsearch Engineer I</a>, which provides a strong foundation for getting started with Elasticsearch, and <a href="/training/elasticsearch-engineer-2">Elasticsearch Engineer II</a>, which digs into more advanced cluster management techniques and best practices for capacity planning and scaling. We also encourage you to check out our <a href="https://discuss.elastic.co/">discussion forums</a>, where you can connect with Elastic users from around the world.

blog-banner-cloud-midnight.png

blog-thumb-cloud-midnight.png

Optimizing costs in Elastic Cloud: Availability zones and snapshot management

Start free trial

Blog Footer CTA

Sign up for Elastic Cloud free trial

headshot-tom-lawler-300x300.jpg

Customer Success Manager

Tom Lawler

Explore similar demos

Overview

By submitting you acknowledge that you've read and agree to our <a href="/legal/elastic-cloud-account-terms" target="_blank">Terms of Service</a>, and that Elastic may <a href="/legal/privacy-statement#how-we-use-the-information" target="_blank">contact you</a> about our related products and services, using the details you provide above. See <a href="/legal/privacy-statement/" target="_blank">Elastic’s Privacy Statement</a> for more details or to opt-out at any time.

Newsletter GDPR Text

提交即表示您确认您已阅读并同意我们的<a href="/cn/legal/elastic-cloud-account-terms" target="_self">服务条款</a>，并且 Elastic 可以使用您提供的上述详细信息与您联系，以向您介绍我们相关的<a href="/cn/legal/privacy-statement#how-we-use-the-information" target="_self">产品和服务</a>。如需了解更多信息，或想随时取消订阅，请参阅 <a href="/cn/legal/privacy-statement/" target="_self">Elastic 的隐私声明</a>。

Speakers

Close

See more insights

The content on this page is not available in the selected language. As Elastic grows globally, we continue to support content in multiple languages.

The content on this page is not available in the selected language.

本页内容尚不支持所选语言。Elastic 正在不断努力，以实现对多种语言内容的支持。感谢您在此期间给予的耐心与陪伴！

Author

Articles by

Learn more

Watch now (no PT)

Watch now

See all top stories

All (no PT translation)

Contact information

Press Release

Share on Reddit

Share this story

Share by Email

Thank you for your interest!

Contact Info

Follow us on Youtube

Load More

Share on LinkedIn

Follow us on LinkedIn

Search Integrations

All Solutions

Video for

Follow us on Twitter

See when this webinar starts in my time zone

查看本次网络研讨会在我的时区的开始时间

Register to Watch

Register now

See All Posts

Can't make it? Register and we'll send you the recording. You'll also receive an email with related content

无法实时参加？注册后我们会给您发送录像。您还将收到相关内容的电子邮件。

Elasticsearch Platform

ELK Stack

Elastic Cloud

可观测性

安全性

搜索

按行业

按解决方案

客户聚焦

开发人员

保持联系

学习

帮助

作者

文章作者 Tom Lawler

Customer Success Manager, Elastic

Optimizing costs in Elastic Cloud: Availability zones and snapshot management

关注我们

关于我们

加入我们

合作伙伴

信任和安全性

投资者关系

卓越奖

关于我们

加入我们

合作伙伴

信任和安全性

投资者关系

卓越奖