It is time to say goodbye: This version of Elastic Cloud Enterprise has reached end-of-life (EOL) and is no longer supported.
The documentation for this version is no longer being maintained. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.
Hot-warm architecture template
editHot-warm architecture template
editA template that you typically use for time-series analytics and log aggregation workloads that benefit from tiered-storage automatic index curation. Includes features to manage resources efficiently when you need greater capacity, such as:
- A tiered architecture with two different types of data nodes, hot and warm.
- Time-based indices, with automatic index curation to move indices from hot to warm nodes over time by changing their shard allocation.
The two type of data nodes in a hot-warm architecture each have their own characteristics:
- Hot data node
- Handles all indexing of new data in the cluster and holds the most recent daily indices that tend to be queried most frequently. Indexing is an I/O intensive activity and the hardware these nodes run on needs to be more powerful and use SSD storage.
- Warm data node
- Handles a large amount of read-only indices that are not queried frequently. With read-only indices, warm nodes can use very large spindle drives instead of SSD storage. Reducing the overall cost of retaining data over time yet making it accessible for queries.
Index curation
editOne of the key features of a hot-warm architecture, time-based index curation automates the task of moving data from hot to warm nodes as it ages. When you deploy a hot-warm architecture, Elastic Cloud Enterprise performs regular index curation according to these rules:
- Index curation moves indices from one Elasticsearch node to another by changing their shard allocation, always from hot to warm.
- Index curation is always time-based and takes place when an index reaches the age specified, in days, weeks, or months.
- Index curation always targets indexes according to one or more matching patterns. If an index matches a pattern, Elastic Cloud Enterprise moves it from a hot to a warm node.
While you create your deployment, you can define which indices get curated and when. To know more about index curation, see Configure index management
To know more about how hot-warm architectures work with Elasticsearch, see “Hot-Warm” Architecture in Elasticsearch 5.x.
In this template
editThe following features are included with this template:
-
Elasticsearch:
-
Data nodes - hot: Starts at 4 GB memory x 1 availability zone. Uses the
data.defaultinstance configuration. -
Data nodes - warm: Starts at 4 GB memory x 1 availability zone. Uses the
data.highstorageinstance configuration. -
Master nodes:
Additional master-eligible node added when choosing 2 availability zones (to create a quorum of 3).
When 1 AZ or 3 AZ are selected, the data nodes act as master-eligible node and there is no requirement for an additional master-eligible node.
Configurations beyond 5 nodes per AZ can also spin up a dedicated master-eligible set of nodes (in 3 AZs always) to offload the data nodes. Uses the
masterinstance configuration.
-
Data nodes - hot: Starts at 4 GB memory x 1 availability zone. Uses the
-
Kibana: Starts at 1 GB memory x 1 availability zone. Uses the
kibanainstance configuration. -
Machine learning (ML): Disabled by default. The functionality is pre-wired into the template, but you must explicitly enable it in the UI. Uses the
mlinstance configuration. -
APM (application performance monitoring): Disabled by default. The functionality is pre-wired into the template, but you must explicitly enable it in the UI. Uses the
apminstance configuration.
To use this template effectively, you must tag your allocators and edit the default instance configurations, so that ECE knows where to host the Elastic Stack products that are part of your deployment.