Universal Profiling index life cycle management
editUniversal Profiling index life cycle management
editIndex lifecycle policies allow you to automate the lifecycle of your profiling indices as they grow and age. A default policy is applied, but you can customize it based on your business needs.
Default policy
editThe default Universal Profiling index lifecycle policy includes the following rollover and delete definitions:
- Rollover: Rollover prevents a single index from growing too large and optimizes indexing and search performance. After an age or size metric threshold is met, a new index is created and all subsequent updates are written to the new index.
- Delete: The delete phase permanently removes the index after a time threshold is met.
The following table lists the default thresholds for rollover and delete:
Rollover | Warm tier | Delete |
---|---|---|
after 30 days or 50 GB |
after 30 days |
after 60 days |
The rollover condition blocks phase transitions which means that indices are kept 30 days after rollover on the hot tier.
To view the Universal Profiling index lifecycle policy in Kibana, navigate to Stack Management → Index Lifecycle Management, and search for profiling
.
Default ILM policies can change between minor versions. This is not considered a breaking change as index management should continually improve and adapt to new features.
Configure a custom index lifecycle policy
editComplete the following steps to configure a custom index lifecycle policy.
- Navigate to Stack Management → Index Lifecycle Policies.
- Click Create policy.
-
Name your new policy, for example
custom-profiling-policy
. - Customize the policy to your liking.
- Click Save policy.
See Manage the index lifecycle to learn more about ILM policies.
To apply your new index lifecycle policy for Universal Profiling, create a new component template named profiling-ilm@custom
.
To apply a custom ILM policy, you must name the component template profiling-ilm@custom
. Other names are not supported.
- From the Index Management page, select the Component Template tab and click Create component template.
-
Enter
profiling-ilm@custom
as the name and click Next. -
In Index settings, set the ILM policy name created in the previous step:
{ "lifecycle": { "name": "custom-profiling-policy" } }
- Continue to the Review step, and select the Request tab. Your request should look similar to the following image.
If it does, click Create component template.
+
To confirm that Universal Profiling is now using the new index template and ILM policy, navigate to Dev Tools and run the following:
The name of the custom ILM policy chosen in Step 1. |
If the custom policy is already applied, the result should include the following:
{ "in_use_by": { "indices": [ ... ], "data_streams": [ ... "profiling-events-all", ... ], "composable_templates": [ "profiling-stackframes", "profiling-symbols-global", "profiling-metrics", "profiling-stacktraces", "profiling-executables", "profiling-hosts", "profiling-events" ] } }
If the result is empty, the custom ILM policy is not yet in use. New ILM policies only take effect when new indices are created, so either wait for a rollover to occur (usually after 30 days or when the index size reaches 50 GB), or force a rollover using the Elasticsearch rollover API:
POST /profiling-events-5pow01/_rollover/ POST /profiling-events-5pow02/_rollover/ POST /profiling-events-5pow03/_rollover/ POST /profiling-events-5pow04/_rollover/ POST /profiling-events-5pow05/_rollover/ POST /profiling-events-5pow06/_rollover/ POST /profiling-events-5pow07/_rollover/ POST /profiling-events-5pow08/_rollover/ POST /profiling-events-5pow09/_rollover/ POST /profiling-events-5pow10/_rollover/ POST /profiling-events-5pow11/_rollover/ POST /profiling-events-all/_rollover/ POST /profiling-executables/_rollover/ POST /profiling-hosts/_rollover/ POST /profiling-metrics/_rollover/ POST /profiling-stackframes/_rollover/ POST /profiling-stacktraces/_rollover/ POST /profiling-symbols-global/_rollover/
After the rollover, the custom ILM policy will be applied to new indices and data streams.