Enable Watcher
editEnable Watcher
editIf you are looking for Kibana alerting, check Alerting and Actions in the Kibana Guide.
Watcher lets you take action based on changes in your data. It is designed around the principle that, if you can query something in Elasticsearch, you can alert on it. Simply define a query, condition, schedule, the actions to take, and Watcher will do the rest.
Watcher can be enabled when configuring your cluster. You can run Alerting on a separate cluster from the cluster whose data you are actually watching.
Before you begin
editSome restrictions apply when adding alerts. To learn more, check Restrictions for alerts (via Watcher).
To enable Watcher on a cluster, you may first need to perform one or several of the following steps. The options shown in the UI differ between stack versions; if an option is not available, you can skip it.
- To receive default Elasticsearch Watcher alerts (cluster status, nodes changed, version mismatch), you need to have monitoring enabled to send to the Admin email address specified in Kibana. To enable this, go to Advanced Settings > Admin email.
To learn more about Kibana alerting and how to use it, check Alerting and Actions.
Send alerts by email
editAlerting can send alerts by email. You can configure notifications similar to the operational emails that Elasticsearch Service sends automatically to alert you about performance issues in your clusters.
Watcher in Elastic Cloud is preconfigured with an email service and can be used without any additional configuration. Alternatively, a custom mail server can be configured as described in Configuring a custom mail server
You can optionally add HTML sanitization settings under Elasticsearch User settings in the Elasticsearch Service Console so that HTML elements are sanitized in the email notification.
For more information on sending alerts by email, check Email action.
Cloud email service limits
editThe following quotas apply when using the Elastic email service:
- Email sending quota: 500 emails per 15min period
- Maximum number of recipients per message: 30 recipients per email (To, CC and BCC all count as recipients).
- Maximum message size (including attachments): 10 MB per message (after base64 encoding).
-
The email-sender can’t be customized (Any custom
From:
header will be removed)
Advanced usage
editSlack and PagerDuty integration
editUnder the hood, Alerting is configured through elasticsearch.yml
. If you want to customize your Alerting settings, you can provide custom elasticsearch.yml
snippet which is appended to your configuration.
To provide the custom snippet, you can use the console Elasticsearch settings editor for your deployment.
For example if you want to use the Slack integration:
There are three steps to integrate Elasticsearch with Slack:
-
Generate a Webhook URL in Slack. It will look similar to
https://hooks.slack.com/services/..
- Add a Slack account name to your Elasticsearch User settings
- Associate the Slack account with the Slack Webhook in the Elasticsearch keystore
To add a webhook in Slack, select the settings icon, then choose Add an app and search for webhook
.
The following example shows a configuration with multiple Slack accounts (account1
, account2
, and account3
) specified in elasticsearch.yml
:
xpack.notification.slack: default_account: account1 account: account1: message_defaults: from: account1 to: channel1 account2: message_defaults: from: account2 to: channel2 account3: message_defaults: from: account3 to: channel3
Slack Webhook account settings
editThe Slack Webhook is set for each account in the Elasticsearch Keystore with the following settings:
- Setting name
-
xpack.notification.slack.account.ACCOUNT_NAME.secure_url
where ACCOUNT_NAME is the Slack account, such asaccount1
. - Type
- Single string
- Secret
- The Webhook URL you generated in Slack earlier.
To specify a Slack account to use for a Watcher Alert that isn’t set as default_account
, you must create an Advanced Watch and explicitly define which Slack account to use in the actions section.
If you have a Slack account that is not currently set as default_account, and you want to use this account for a Watcher Alert, you must create an Advanced Watch and explicitly define in the Actions section of the UI which Slack account to use.
PUT _watcher/watch/test-alarm { "metadata" : { ... }, "trigger" : { ... }, "input" : { ... }, "actions" : { "notify-slack" : { "throttle_period" : "10s", "slack" : { "account" : "account2", "message" : { "to" : [ "#testing-channel" ], "text" : "You Know, for Search" } } } } }
In Elasticsearch versions before 7.0:, you are not required to use the Elasticsearch keystore. Instead, you can use the console Elasticsearch settings editor for your deployment.
Configuring a custom mail server
editIt is possible to use a custom mail service instead of the one configured by default. It can be configured by following the Elasticsearch documentation for configuring email accounts.
An example on how to configure a new account from the Elastic cloud console:
- From your deployment menu, go to the Edit page.
- In the Elasticsearch section, select Manage user settings and extensions.
-
Add the settings for a new mail account.
xpack.notification.email: default_account: my_email_service account: my_email_service: smtp: auth: true starttls.enable: true starttls.required: true host: email-smtp.us-east-1.amazonaws.com port: 587 user: <username>
- Select Save changes.
-
To complete the configuration, the password for the email service has to be added to the keystore
- Follow the steps described in our security settings documentation to Add a secret value to the keystore
-
Set the Setting name as
xpack.notification.email.account.my_email_service.smtp.secure_password
(The account name must match the configuration in the user settings).
- The new email account is now set up. It will now be used by default for watcher email actions.
For a full reference of all available settings, see the Elasticsearch documentation.