Create auto-follow pattern API
editCreate auto-follow pattern API
editCreates an auto-follow pattern.
Request
editPUT /_ccr/auto_follow/<auto_follow_pattern_name> { "remote_cluster" : "<remote_cluster>", "leader_index_patterns" : [ "<leader_index_pattern>" ], "leader_index_exclusion_patterns": [ "<leader_index_exclusion_pattern>" ], "follow_index_pattern" : "<follow_index_pattern>" }
Prerequisites
edit-
If the Elasticsearch security features are enabled, you must have
read
andmonitor
index privileges for the leader index patterns. You must also havemanage_ccr
cluster privileges on the cluster that contains the follower index. For more information, see Security privileges.
Description
editThis API creates a new named collection of auto-follow patterns against the remote cluster specified in the request body. Newly created indices on the remote cluster matching any of the specified patterns will be automatically configured as follower indices. Indices on the remote cluster that were created before the auto-follow pattern is created won’t be auto-followed even if they match the pattern.
This API can also be used to update existing auto-follow patterns. Note that follower indices that were configured automatically before updating an auto-follow pattern will remain unchanged even if they don’t match against the new patterns.
Path parameters
edit-
<auto_follow_pattern_name>
- (Required, string) The name of the collection of auto-follow patterns.
Request body
edit-
remote_cluster
- (Required, string) The remote cluster containing the leader indices to match against.
-
leader_index_patterns
-
(Optional, array) An array of simple index patterns to match against indices
in the remote cluster specified by the
remote_cluster
field. -
leader_index_exclusion_patterns
-
(Optional, array) An array of simple index patterns that can be used to exclude indices
from being auto-followed. Indices in the remote cluster whose names are matching one or
more
leader_index_patterns
and one or moreleader_index_exclusion_patterns
won’t be followed. -
follow_index_pattern
-
(Optional, string) The name of follower index. The template
{{leader_index}}
can be used to derive the name of the follower index from the name of the leader index. When following a data stream, thefollow_index_pattern
will be used for renaming not only the leader index, but also the data stream containing the leader index. For example, a data stream calledlogs-mysql-default
with a backing index of.ds-logs-mysql-default-2022-01-01-000001
and afollow_index_pattern
of{{leader_index}}_copy
will replicate the data stream aslogs-mysql-default_copy
and the backing index as.ds-logs-mysql-default_copy-2022-01-01-000001
. -
settings
-
(object) Settings to override from the leader index. Note that certain
settings can not be overrode (e.g.,
index.number_of_shards
).
-
max_read_request_operation_count
- (integer) The maximum number of operations to pull per read from the remote cluster.
-
max_outstanding_read_requests
- (long) The maximum number of outstanding reads requests from the remote cluster.
-
max_read_request_size
- (byte value) The maximum size in bytes of per read of a batch of operations pulled from the remote cluster.
-
max_write_request_operation_count
- (integer) The maximum number of operations per bulk write request executed on the follower.
-
max_write_request_size
- (byte value) The maximum total bytes of operations per bulk write request executed on the follower.
-
max_outstanding_write_requests
- (integer) The maximum number of outstanding write requests on the follower.
-
max_write_buffer_count
- (integer) The maximum number of operations that can be queued for writing. When this limit is reached, reads from the remote cluster will be deferred until the number of queued operations goes below the limit.
-
max_write_buffer_size
- (byte value) The maximum total bytes of operations that can be queued for writing. When this limit is reached, reads from the remote cluster will be deferred until the total bytes of queued operations goes below the limit.
-
max_retry_delay
- (time value) The maximum time to wait before retrying an operation that failed exceptionally. An exponential backoff strategy is employed when retrying.
-
read_poll_timeout
- (time value) The maximum time to wait for new operations on the remote cluster when the follower index is synchronized with the leader index. When the timeout has elapsed, the poll for operations will return to the follower so that it can update some statistics. Then the follower will immediately attempt to read from the leader again.
Default values
editThe following output from the follow info api describes all the default values for the above described index follow request parameters:
{ "follower_indices" : [ { "parameters" : { "max_read_request_operation_count" : 5120, "max_read_request_size" : "32mb", "max_outstanding_read_requests" : 12, "max_write_request_operation_count" : 5120, "max_write_request_size" : "9223372036854775807b", "max_outstanding_write_requests" : 9, "max_write_buffer_count" : 2147483647, "max_write_buffer_size" : "512mb", "max_retry_delay" : "500ms", "read_poll_timeout" : "1m" } } ] }
Examples
editThis example creates an auto-follow pattern named my_auto_follow_pattern
:
PUT /_ccr/auto_follow/my_auto_follow_pattern { "remote_cluster" : "remote_cluster", "leader_index_patterns" : [ "leader_index*" ], "follow_index_pattern" : "{{leader_index}}-follower", "settings": { "index.number_of_replicas": 0 }, "max_read_request_operation_count" : 1024, "max_outstanding_read_requests" : 16, "max_read_request_size" : "1024k", "max_write_request_operation_count" : 32768, "max_write_request_size" : "16k", "max_outstanding_write_requests" : 8, "max_write_buffer_count" : 512, "max_write_buffer_size" : "512k", "max_retry_delay" : "10s", "read_poll_timeout" : "30s" }
The API returns the following result:
{ "acknowledged" : true }