It is time to say goodbye: This version of Elastic Cloud Enterprise has reached end-of-life (EOL) and is no longer supported.
The documentation for this version is no longer being maintained. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.
Create deployment templates
editCreate deployment templates
editElastic Cloud Enterprise comes with some deployment templates already built in, but you can create new deployment templates to address particular use cases that you might have.
For example: You might decide to create a new deployment template, if you have a specific search use case that requires Elasticsearch data nodes in a specific configuration that also includes machine learning for anomaly detection. If you need to create these deployments fairly frequently, you can create a deployment template once and deploy it as many times as you like. Or, create a single template for both your test and production deployments to ensure they are exactly the same.
Before you begin
editBefore you start creating your own deployment templates, you should have: tagged your allocators to tell ECE what kind of hardware you have available for Elastic Stack deployments. If the default instance configurations don’t provide what you need, you might also need to create your own instance configurations first.
Create deployment templates in the UI
edit- Log into the Cloud UI.
- From the Platform menu, select Templates.
- Click Create template.
-
Configure Elasticsearch by picking the node configurations that will be available in the template and by assigning their initial resources. You can:
-
Adjust the memory or storage assigned to instance configurations using our sliders. Increasing memory or storage also improves performance by increasing the CPU resources that get assigned relative to the size of the instance, meaning that a 32 GB instance gets twice as much CPU resources as a 16 GB one. These resources are just template defaults that can be adjusted further before you create actual deployments.
Note that to increase the number of nodes assigned to an instance configuration you must first scale up to the maximum RAM for that instance type. For example, with an Elasticsearch
data.defaultinstance you need to scale the RAM per node up to 64GB before you can add additional nodes. - Add fault tolerance (high availability) by using more than one availability zones.
- Add Elasticsearch user settings.
-
Add and configure additional features, such as:
- Dedicated master nodes
- Dedicated coordinating nodes
-
Machine learning nodes
For example: You can could create your template with a large Elasticsearch data node to handle data indexing, include a basic machine learning configuration to allow enabling anomaly detection later on, and then add Kibana to the template to enable data visualization.
To create templates that support a hot-warm architecture, include more than one instance configuration for data nodes by clicking Add data configuration. The option to edit index curation settings becomes available when at least two data configurations are part of the template.
The default instance configuration used for the data section of the template uses all three of the capabilities: data, master, and ingest. If you use a dedicated configuration instead, where only data is enabled, you must also enable a dedicated master section and set the threshold value to
1. If you do not enable dedicated coordinating nodes in that situation, your data nodes will assume coordination of incoming read and write requests to the deployment, but you will not be able to use any ingest pipelines. To use ingest pipelines, you need to have either adefaultor acoordinatinginstance configuration enabled in at least one section of the template.
-
- Click Next.
-
If you added multiple Elasticsearch data configurations on the previous page, such as
data.defaultanddata.highstorage: Configure index management. - Configure how other Elastic products, like Kibana instances or APM Servers, get included in your template. You can select if they are included at all, what high availability options and sizes are available, and what user settings should always be specified.
- Give your deployment template a name and include a description that reflects its intended use.
- Click Save and create template.
Create deployment templates through the RESTful API:
edit-
Obtain the existing deployment templates to see some examples of what the required JSON looks like. You can take the JSON for one of the existing templates and modify it to create a new template, similar to what gets shown in the next step.
curl -k -X GET -H "Authorization: ApiKey $ECE_API_KEY" https://COORDINATOR_HOST:12443/api/v1/deployments/templates?region=ece-region
-
Post the JSON for your new deployment template.
The following example creates a deployment template that defaults to a highly available Elasticsearch cluster with 4 GB per hot node, a 16 GB machine learning node, 3 dedicated master nodes of 1 GB each, a 1 GB Kibana instance, and a 1 GB dedicated coordinating node that is tasked with handling and coordinating all incoming requests for the cluster. Elasticsearch and Kibana use the default instance configurations, but the machine learning node is based on the custom instance configuration in our previous example.
curl -k -X POST -H "Authorization: ApiKey $ECE_API_KEY" https://$COORDINATOR_HOST:12443/api/v1/deployments/templates?region=ece-region -H 'content-type: application/json' -d '{ "name" : "Default", "description" : "Default deployment template for clusters", "deployment_template": { "resources": { "elasticsearch": [ { "ref_id": "es-ref-id", "region": "ece-region", "plan": { "cluster_topology": [ { "node_type": { "master": true, "data": true, "ingest": true }, "zone_count": 1, "instance_configuration_id": "data.default", "size": { "value": 4096, "resource": "memory" }, "node_roles": [ "master", "ingest", "data_hot", "data_content", "remote_cluster_client", "transform" ], "id": "hot_content", "elasticsearch": { "node_attributes": { "data": "hot" } }, "topology_element_control": { "min": { "value": 1024, "resource": "memory" } }, "autoscaling_max": { "value": 2097152, "resource": "memory" } }, { "node_type": { "data": true, "ingest": false, "master": false }, "instance_configuration_id": "data.highstorage", "zone_count": 1, "size": { "resource": "memory", "value": 0 }, "node_roles": [ "data_warm", "remote_cluster_client" ], "id": "warm", "elasticsearch": { "node_attributes": { "data": "warm" } }, "topology_element_control": { "min": { "value": 0, "resource": "memory" } }, "autoscaling_max": { "value": 2097152, "resource": "memory" } }, { "node_type": { "data": true, "ingest": false, "master": false }, "instance_configuration_id": "data.highstorage", "zone_count": 1, "size": { "resource": "memory", "value": 0 }, "node_roles": [ "data_cold", "remote_cluster_client" ], "id": "cold", "elasticsearch": { "node_attributes": { "data": "cold" } }, "topology_element_control": { "min": { "value": 0, "resource": "memory" } }, "autoscaling_max": { "value": 2097152, "resource": "memory" } }, { "node_type": { "data": true, "ingest": false, "master": false }, "instance_configuration_id": "data.frozen", "zone_count": 1, "size": { "resource": "memory", "value": 0 }, "node_roles": [ "data_frozen" ], "id": "frozen", "elasticsearch": { "node_attributes": { "data": "frozen" } }, "topology_element_control": { "min": { "value": 0, "resource": "memory" } }, "autoscaling_max": { "value": 2097152, "resource": "memory" } }, { "node_type": { "master": false, "data": false, "ingest": true }, "zone_count": 1, "instance_configuration_id": "coordinating", "size": { "value": 1024, "resource": "memory" }, "node_roles": [ "ingest", "remote_cluster_client" ], "id": "coordinating", "topology_element_control": { "min": { "value": 0, "resource": "memory" } } }, { "node_type": { "master": true, "data": false, "ingest": false }, "zone_count": 3, "instance_configuration_id": "master", "size": { "value": 1024, "resource": "memory" }, "node_roles": [ "master", "remote_cluster_client" ], "id": "master", "topology_element_control": { "min": { "value": 0, "resource": "memory" } } }, { "node_type": { "master": false, "data": false, "ingest": false, "ml": true }, "zone_count": 1, "instance_configuration_id": "ml", "size": { "value": 0, "resource": "memory" }, "node_roles": [ "ml", "remote_cluster_client" ], "id": "ml", "topology_element_control": { "min": { "value": 16384, "resource": "memory" } }, "autoscaling_min": { "resource": "memory", "value": 16384 }, "autoscaling_max": { "value": 2097152, "resource": "memory" } } ], "elasticsearch": {}, "autoscaling_enabled": false }, "settings": { "dedicated_masters_threshold": 3 } } ], "kibana": [ { "ref_id": "kibana-ref-id", "elasticsearch_cluster_ref_id": "es-ref-id", "region": "ece-region", "plan": { "zone_count": 1, "cluster_topology": [ { "instance_configuration_id": "kibana", "size": { "value": 1024, "resource": "memory" } } ], "kibana": {} } } ], "apm": [ { "ref_id": "apm-ref-id", "elasticsearch_cluster_ref_id": "es-ref-id", "region": "ece-region", "plan": { "cluster_topology": [ { "instance_configuration_id": "apm", "size": { "value": 0, "resource": "memory" }, "zone_count": 1 } ], "apm": {} } } ], "enterprise_search": [ { "ref_id": "enterprise_search-ref-id", "elasticsearch_cluster_ref_id": "es-ref-id", "region": "ece-region", "plan": { "cluster_topology": [ { "node_type": { "appserver": true, "connector": true, "worker": true }, "instance_configuration_id": "enterprise.search", "size": { "value": 0, "resource": "memory" }, "zone_count": 2 } ], "enterprise_search": {} } } ] } } }'
When specifying node_roles in the Elasticsearch plan of the deployment template, the template must contain all resource types and all Elasticsearch tiers.
The deployment template must contain exactly one entry for each resource type. It must have one Elasticsearch, one Kibana, one APM, and one Enterprise Search. On top of that, it must also include all supported Elasticsearch tiers in the Elasticsearch plan. The supported tiers are identified by the IDs hot_content, warm, cold, frozen, master, coordinating and ml.
After you have saved your new template, you can start creating new deployments with it.
To support deployment templates that are versioned due to a constraint on architecture that is only supported by newer versions of ECE, for example ARM instances, you must add additional configuration:
-
The
template_category_idfor both template versions must be identical. -
The
min_versionattribute must be set.
These attributes are set at the same level as name and description. The UI selects the template with the highest matching min_version that is returned by the API.