Configuration File

edit

The default location of the configuration file is ~/.curator/curator.yml, but another location can be specified using the --config flag on the command-line.

Starting in Curator 4.1, you can use environment variables in your configuration files.

The configuration file contains client connection and settings for logging. It looks like this:

---
# Remember, leave a key empty if there is no value.  None will be a string,
# not a Python "NoneType"
client:
  hosts:
    - 127.0.0.1
  port: 9200
  url_prefix:
  use_ssl: False
  certificate:
  client_cert:
  client_key:
  ssl_no_validate: False
  http_auth:
  timeout: 30
  master_only: False

logging:
  loglevel: INFO
  logfile:
  logformat: default
  blacklist: ['elasticsearch', 'urllib3']

It is a YAML configuration file. The two root keys must be client and logging. The subkeys of each of these will be described here.

hosts

edit

This can be a single value:

hosts: 127.0.0.1

Or multiple values in the 3 acceptable YAML ways to render sequences, or arrays:

Flow:

hosts: [ "10.0.0.1", "10.0.0.2" ]

Spanning:

hosts: [ "10.0.0.1",
    "10.0.0.2" ]

Block:

hosts:
  - 10.0.0.1
  - 10.0.0.2

You can also provide these hosts with optional ports, and bypass the port option:

hosts:
  - 10.0.0.1:9200
  - 10.0.0.2:9201

When adding a port to the end of a host or IP, the YAML Flow and Spanning styles require host:port to be single ' or double " quote encapsulated or you will receive an error. The Block style does not have this limitation.

port

edit

This should be a single value:

port: 9200

The default is 9200. This value will only be applied to hosts without a port affixed, e.g. localhost:9202.

url_prefix

edit

This should be a single value or left empty.

url_prefix:

In some cases you may be obliged to connect to your Elasticsearch cluster through a proxy of some kind. There may be a URL prefix before the API URI items, e.g. http://example.com/elasticsearch/ as opposed to http://localhost:9200. In such a case, set the url_prefix to the appropriate value, elasticsearch in this example.

The default is an empty string.

use_ssl

edit

This should be True, False or left empty.

use_ssl:

If access to your Elasticsearch instance is protected by SSL encryption, you must use set use_ssl to True.

The default is False

certificate

edit

This should be a file path to your CA certificate, or left empty.

certificate:

This setting allows the use of a specified CA certificate file to validate the SSL certificate used by Elasticsearch.

There is no default.

client_cert

edit

This should be a file path to a client certificate (public key), or left empty.

client_cert:

Allows the use of a specified SSL client cert file to authenticate to Elasticsearch. The file may contain both an SSL client certificate and an SSL key, in which case client_key is not used. If specifying client_cert, and the file specified does not also contain the key, use client_key to specify the file containing the SSL key. The file must be in PEM format, and the key part, if used, must be an unencrypted key in PEM format as well.

client_key

edit

This should be a file path to a client key (private key), or left empty.

client_key:

Allows the use of a specified SSL client key file to authenticate to Elasticsearch. If using client_cert and the file specified does not also contain the key, use client_key to specify the file containing the SSL key. The key file must be an unencrypted key in PEM format.

aws_key

edit

This feature allows connection to AWS using IAM credentials, but Curator 4 does not currently work with AWS.

This setting will not work unless the requests-aws4auth Python module has been manually installed first.

This should be an AWS IAM access key, or left empty.

aws_key:

You must set your hosts to the proper hostname with port. It may not work setting port and hosts to only a host name due to the different connection module used.

aws_secret_key

edit

This feature allows connection to AWS using IAM credentials, but Curator 4 does not currently work with AWS.

This setting will not work unless the requests-aws4auth Python module has been manually installed first.

This should be an AWS IAM secret access key, or left empty.

aws_secret_key:

You must set your hosts to the proper hostname with port. It may not work setting port and hosts to only a host name due to the different connection module used.

aws_region

edit

This feature allows connection to AWS using IAM credentials, but Curator 4 does not currently work with AWS.

This setting will not work unless the requests-aws4auth Python module has been manually installed first.

This should be an AWS region, or left empty.

aws_region:

You must set your hosts to the proper hostname with port. It may not work setting port and hosts to only a host name due to the different connection module used.

ssl_no_validate

edit

This should be True, False or left empty.

ssl_no_validate:

If access to your Elasticsearch instance is protected by SSL encryption, you may set ssl_no_validate to True to disable SSL certificate verification.

Valid use cases for doing so include the use of self-signed certificates that cannot be otherwise verified and would generate error messages.

Setting ssl_no_validate to True will likely result in a warning message that your SSL certificates are not trusted. This is expected behavior.

The default value is False.

http_auth

edit

This should be a authentication credentials (e.g. user:pass), or left empty.

http_auth:

This setting allows basic HTTP authentication to an Elasticsearch instance.

The default is empty.

timeout

edit

This should be an integer number of seconds, or left empty.

timeout:

You can change the default client connection timeout value with this setting.

The default value is 30 (seconds) should typically not be changed to be very large. If a longer timeout is necessary for a given action, such as snapshot, restore, or forcemerge, the client timeout can be overridden on per action basis by setting timeout_override in the action options. There are default override values for some of those longer running actions.

master_only

edit

This should be True, False or left empty.

master_only:

In some situations, primarily with automated deployments, it makes sense to install Curator on every node. But you wouldn’t want it to run on each node. By setting master_only to True, this is possible. It tests for, and will only continue running on the node that is the elected master.

If master_only is True, and hosts has more than one value, Curator will raise an Exception. This setting should only be used with a single host in hosts, as its utility centers around deploying to all nodes in the cluster.

The default value is False.

loglevel

edit

This should be CRITICAL, ERROR, WARNING, INFO, DEBUG, or left empty.

loglevel:

Set the minimum acceptable log severity to display.

  • CRITICAL will only display critical messages.
  • ERROR will only display error and critical messages.
  • WARN will display error, warning, and critical messages.
  • INFO will display informational, error, warning, and critical messages.
  • DEBUG will display debug messages, in addition to all of the above.

The default value is INFO.

logfile

edit

This should be a path to a log file, or left empty.

logfile:

The default value is empty, which will result in logging to STDOUT, or the console.

logformat

edit

This should default, json, logstash, or left empty.

logformat:

The default format looks like:

2016-04-22 11:53:09,972 INFO      Action #1: ACTIONNAME

The json or logstash formats look like:

{"@timestamp": "2016-04-22T11:54:29.033Z", "function": "cli", "linenum": 178,
"loglevel": "INFO", "message": "Action #1: ACTIONNAME", "name": "curator.cli"}

The default value is default.

blacklist

edit

This should be an empty array [], an array of log handler strings, or left empty.

blacklist: ['elasticsearch', 'urllib3']

The default value is ['elasticsearch', 'urllib3'], which will result in logs for the elasticsearch and urllib3 Python modules not being output. These can be quite verbose, so unless you need them to debug an issue, you should accept the default value.

If you do need to troubleshoot an issue, set blacklist to [], which is an empty array. Leaving it unset will result in the default behavior, which is to filter out elasticsearch and urllib3 log traffic.