Updates to executing policies

edit

This functionality is in beta and is subject to change. The design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features.

Indices preserve the phase definition from the latest policy version that existed at the time that it entered that phase. Changes to the currently-executing phase within policy updates will not be reflected during execution. This means that updates to the hot phase, for example, will not affect indices that are currently executing the corresponding hot phase.

Let’s say we have an index my_index managed by the below my_executing_policy definition.

PUT _ilm/policy/my_executing_policy
{
  "policy": {
    "phases": {
      "hot": {
        "actions": {
          "rollover": {
            "max_docs": 1
          }
        }
      },
      "delete": {
        "min_age": "10d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

The Explain API is useful to introspect managed indices to see which phase definition they are currently executing. Using this API, we can find out that my_index is currently checking if it is ready to be rolled over.

GET my_index/_ilm/explain
{
  "indices": {
    "my_index": {
      "index": "my_index",
      "managed": true,
      "policy": "my_executing_policy",
      "lifecycle_date_millis": 1538475653281,
      "phase": "hot",
      "phase_time_millis": 1538475653317,
      "action": "rollover",
      "action_time_millis": 1538475653317,
      "step": "check-rollover-ready",
      "step_time_millis": 1538475653317,
      "phase_execution": {
        "policy": "my_executing_policy",
        "modified_date_in_millis": 1538475653317,
        "version": 1,
        "phase_definition": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_docs": 1
            }
          }
        }
      }
    }
  }
}

Updating my_executing_policy to have no rollover action and, instead, go directly into a newly introduced warm phase.

PUT _ilm/policy/my_executing_policy
{
  "policy": {
    "phases": {
      "warm": {
        "min_age": "1d",
        "actions": {
          "forcemerge": {
            "max_num_segments": 1
          }
        }
      },
      "delete": {
        "min_age": "10d",
        "actions": {
          "delete": {}
        }
      }
    }
  }
}

Now, version 2 of this policy has no hot phase, but if we run the Explain API again, we will see that nothing has changed. The index my_index is still executing version 1 of the policy.

{
  "indices": {
    "my_index": {
      "index": "my_index",
      "managed": true,
      "policy": "my_executing_policy",
      "lifecycle_date_millis": 1538475653281,
      "phase": "hot",
      "phase_time_millis": 1538475653317,
      "action": "rollover",
      "action_time_millis": 1538475653317,
      "step": "check-rollover-ready",
      "step_time_millis": 1538475653317,
      "phase_execution": {
        "policy": "my_executing_policy",
        "modified_date_in_millis": 1538475653317,
        "version": 1,
        "phase_definition": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_docs": 1
            }
          }
        }
      }
    }
  }
}

After indexing one document into my_index so that rollover succeeds and moves onto the next phase, we will notice something new. The index will move into the next phase in the updated version 2 of its policy.

{
  "indices": {
    "my_index": {
      "index": "my_index",
      "managed": true,
      "policy": "my_executing_policy",
      "lifecycle_date_millis": 1538475653281,
      "phase": "warm",
      "phase_time_millis": 1538475653317,
      "action": "forcemerge",
      "action_time_millis": 1538475653317,
      "step": "forcemerge",
      "step_time_millis": 1538475653317,
      "phase_execution": {
        "policy": "my_executing_policy",
        "modified_date_in_millis": 1538475653317,
        "version": 2, 
        "phase_definition": {
          "min_age": "1d",
          "actions": {
            "forcemerge": {
              "max_num_segments": 1
            }
          }
        }
      }
    }
  }
}

The index has moved to using version 2 of the policy

my_index will move to the next phase in the latest policy definition, which is the newly added warm phase.