- Elasticsearch Guide: other versions:
- Getting Started
- Set up Elasticsearch
- Installing Elasticsearch
- Configuring Elasticsearch
- Important Elasticsearch configuration
- Important System Configuration
- Bootstrap Checks
- Heap size check
- File descriptor check
- Memory lock check
- Maximum number of threads check
- Max file size check
- Maximum size virtual memory check
- Maximum map count check
- Client JVM check
- Use serial collector check
- System call filter check
- OnError and OnOutOfMemoryError checks
- Early-access check
- G1GC check
- All permission check
- Starting Elasticsearch
- Stopping Elasticsearch
- Adding nodes to your cluster
- Installing X-Pack
- Set up X-Pack
- Configuring X-Pack Java Clients
- X-Pack Settings
- Bootstrap Checks for X-Pack
- Upgrade Elasticsearch
- API Conventions
- Document APIs
- Search APIs
- Aggregations
- Metrics Aggregations
- Avg Aggregation
- Weighted Avg Aggregation
- Cardinality Aggregation
- Extended Stats Aggregation
- Geo Bounds Aggregation
- Geo Centroid Aggregation
- Max Aggregation
- Min Aggregation
- Percentiles Aggregation
- Percentile Ranks Aggregation
- Scripted Metric Aggregation
- Stats Aggregation
- Sum Aggregation
- Top Hits Aggregation
- Value Count Aggregation
- Bucket Aggregations
- Adjacency Matrix Aggregation
- Auto-interval Date Histogram Aggregation
- Intervals
- Children Aggregation
- Composite Aggregation
- Date Histogram Aggregation
- Date Range Aggregation
- Diversified Sampler Aggregation
- Filter Aggregation
- Filters Aggregation
- Geo Distance Aggregation
- GeoHash grid Aggregation
- Global Aggregation
- Histogram Aggregation
- IP Range Aggregation
- Missing Aggregation
- Nested Aggregation
- Range Aggregation
- Reverse nested Aggregation
- Sampler Aggregation
- Significant Terms Aggregation
- Significant Text Aggregation
- Terms Aggregation
- Pipeline Aggregations
- Avg Bucket Aggregation
- Derivative Aggregation
- Max Bucket Aggregation
- Min Bucket Aggregation
- Sum Bucket Aggregation
- Stats Bucket Aggregation
- Extended Stats Bucket Aggregation
- Percentiles Bucket Aggregation
- Moving Average Aggregation
- Moving Function Aggregation
- Cumulative Sum Aggregation
- Bucket Script Aggregation
- Bucket Selector Aggregation
- Bucket Sort Aggregation
- Serial Differencing Aggregation
- Matrix Aggregations
- Caching heavy aggregations
- Returning only aggregation results
- Aggregation Metadata
- Returning the type of the aggregation
- Metrics Aggregations
- Indices APIs
- Create Index
- Delete Index
- Get Index
- Indices Exists
- Open / Close Index API
- Shrink Index
- Split Index
- Rollover Index
- Put Mapping
- Get Mapping
- Get Field Mapping
- Types Exists
- Index Aliases
- Update Indices Settings
- Get Settings
- Analyze
- Index Templates
- Indices Stats
- Indices Segments
- Indices Recovery
- Indices Shard Stores
- Clear Cache
- Flush
- Refresh
- Force Merge
- cat APIs
- Cluster APIs
- Query DSL
- Mapping
- Analysis
- Anatomy of an analyzer
- Testing analyzers
- Analyzers
- Normalizers
- Tokenizers
- Standard Tokenizer
- Letter Tokenizer
- Lowercase Tokenizer
- Whitespace Tokenizer
- UAX URL Email Tokenizer
- Classic Tokenizer
- Thai Tokenizer
- NGram Tokenizer
- Edge NGram Tokenizer
- Keyword Tokenizer
- Pattern Tokenizer
- Char Group Tokenizer
- Simple Pattern Tokenizer
- Simple Pattern Split Tokenizer
- Path Hierarchy Tokenizer
- Path Hierarchy Tokenizer Examples
- Token Filters
- Standard Token Filter
- ASCII Folding Token Filter
- Flatten Graph Token Filter
- Length Token Filter
- Lowercase Token Filter
- Uppercase Token Filter
- NGram Token Filter
- Edge NGram Token Filter
- Porter Stem Token Filter
- Shingle Token Filter
- Stop Token Filter
- Word Delimiter Token Filter
- Word Delimiter Graph Token Filter
- Multiplexer Token Filter
- Conditional Token Filter
- Predicate Token Filter Script
- Stemmer Token Filter
- Stemmer Override Token Filter
- Keyword Marker Token Filter
- Keyword Repeat Token Filter
- KStem Token Filter
- Snowball Token Filter
- Phonetic Token Filter
- Synonym Token Filter
- Synonym Graph Token Filter
- Compound Word Token Filters
- Reverse Token Filter
- Elision Token Filter
- Truncate Token Filter
- Unique Token Filter
- Pattern Capture Token Filter
- Pattern Replace Token Filter
- Trim Token Filter
- Limit Token Count Token Filter
- Hunspell Token Filter
- Common Grams Token Filter
- Normalization Token Filter
- CJK Width Token Filter
- CJK Bigram Token Filter
- Delimited Payload Token Filter
- Keep Words Token Filter
- Keep Types Token Filter
- Exclude mode settings example
- Classic Token Filter
- Apostrophe Token Filter
- Decimal Digit Token Filter
- Fingerprint Token Filter
- Minhash Token Filter
- Remove Duplicates Token Filter
- Character Filters
- Modules
- Index Modules
- Ingest Node
- Pipeline Definition
- Ingest APIs
- Accessing Data in Pipelines
- Conditional Execution in Pipelines
- Handling Failures in Pipelines
- Processors
- Append Processor
- Bytes Processor
- Convert Processor
- Date Processor
- Date Index Name Processor
- Dissect Processor
- Drop Processor
- Dot Expander Processor
- Fail Processor
- Foreach Processor
- Grok Processor
- Gsub Processor
- Join Processor
- JSON Processor
- KV Processor
- Lowercase Processor
- Pipeline Processor
- Remove Processor
- Rename Processor
- Script Processor
- Set Processor
- Set Security User Processor
- Split Processor
- Sort Processor
- Trim Processor
- Uppercase Processor
- URL Decode Processor
- SQL Access
- Monitor a cluster
- Rolling up historical data
- Set up a cluster for high availability
- Secure a cluster
- Overview
- Configuring security
- Encrypting communications in Elasticsearch
- Encrypting communications in an Elasticsearch Docker Container
- Enabling cipher suites for stronger encryption
- Separating node-to-node and client traffic
- Configuring an Active Directory realm
- Configuring a file realm
- Configuring an LDAP realm
- Configuring a native realm
- Configuring a PKI realm
- Configuring a SAML realm
- Configuring a Kerberos realm
- FIPS 140-2
- Security settings
- Security files
- Auditing settings
- How security works
- User authentication
- Built-in users
- Internal users
- Realms
- Realm chains
- Active Directory user authentication
- File-based user authentication
- LDAP user authentication
- Native user authentication
- PKI user authentication
- SAML authentication
- Kerberos authentication
- Integrating with other authentication systems
- Enabling anonymous access
- Controlling the user cache
- Configuring SAML single-sign-on on the Elastic Stack
- User authorization
- Auditing security events
- Encrypting communications
- Restricting connections with IP filtering
- Cross cluster search, tribe, clients, and integrations
- Tutorial: Getting started with security
- Tutorial: Encrypting communications
- Troubleshooting
- Can’t log in after upgrading to 6.5.4
- Some settings are not returned via the nodes settings API
- Authorization exceptions
- Users command fails due to extra arguments
- Users are frequently locked out of Active Directory
- Certificate verification fails for curl on Mac
- SSLHandshakeException causes connections to fail
- Common SSL/TLS exceptions
- Common Kerberos exceptions
- Common SAML issues
- Internal Server Error in Kibana
- Setup-passwords command fails due to connection failure
- Failures due to relocation of the configuration files
- Limitations
- Alerting on Cluster and Index Events
- Command line tools
- How To
- Testing
- Glossary of terms
- X-Pack APIs
- Info API
- Cross-cluster replication APIs
- Explore API
- Licensing APIs
- Migration APIs
- Machine learning APIs
- Add events to calendar
- Add jobs to calendar
- Close jobs
- Create calendar
- Create datafeeds
- Create filter
- Create jobs
- Delete calendar
- Delete datafeeds
- Delete events from calendar
- Delete filter
- Delete forecast
- Delete jobs
- Delete jobs from calendar
- Delete model snapshots
- Find file structure
- Flush jobs
- Forecast jobs
- Get calendars
- Get buckets
- Get overall buckets
- Get categories
- Get datafeeds
- Get datafeed statistics
- Get influencers
- Get jobs
- Get job statistics
- Get machine learning info
- Get model snapshots
- Get scheduled events
- Get filters
- Get records
- Open jobs
- Post data to jobs
- Preview datafeeds
- Revert model snapshots
- Start datafeeds
- Stop datafeeds
- Update datafeeds
- Update filter
- Update jobs
- Update model snapshots
- Rollup APIs
- Security APIs
- Authenticate
- Change passwords
- Clear cache
- Clear roles cache
- Create or update application privileges
- Create or update role mappings
- Create or update roles
- Create or update users
- Delete application privileges
- Delete role mappings
- Delete roles
- Delete users
- Disable users
- Enable users
- Get application privileges
- Get role mappings
- Get roles
- Get token
- Get users
- Has privileges
- Invalidate token
- SSL certificate
- Watcher APIs
- Definitions
- Release Highlights
- Breaking changes
- Release Notes
- Elasticsearch version 6.5.4
- Elasticsearch version 6.5.3
- Elasticsearch version 6.5.2
- Elasticsearch version 6.5.1
- Elasticsearch version 6.5.0
- Elasticsearch version 6.4.3
- Elasticsearch version 6.4.2
- Elasticsearch version 6.4.1
- Elasticsearch version 6.4.0
- Elasticsearch version 6.3.2
- Elasticsearch version 6.3.1
- Elasticsearch version 6.3.0
- Elasticsearch version 6.2.4
- Elasticsearch version 6.2.3
- Elasticsearch version 6.2.2
- Elasticsearch version 6.2.1
- Elasticsearch version 6.2.0
- Elasticsearch version 6.1.4
- Elasticsearch version 6.1.3
- Elasticsearch version 6.1.2
- Elasticsearch version 6.1.1
- Elasticsearch version 6.1.0
- Elasticsearch version 6.0.1
- Elasticsearch version 6.0.0
- Elasticsearch version 6.0.0-rc2
- Elasticsearch version 6.0.0-rc1
- Elasticsearch version 6.0.0-beta2
- Elasticsearch version 6.0.0-beta1
- Elasticsearch version 6.0.0-alpha2
- Elasticsearch version 6.0.0-alpha1
- Elasticsearch version 6.0.0-alpha1 (Changes previously released in 5.x)
Email action
editEmail action
editUse the email
action to send email notifications. To send email, you must
configure at least one email account in
elasticsearch.yml
.
Email notifications can be plain text or styled using HTML. You can include information from the watch execution payload using templates and attach the entire watch payload to the message.
See Email action attributes for the supported attributes. Any attributes that
are missing from the email action definition are looked up in the email
account configuration. The required attributes must either be set in the email
action definition or the account’s email_defaults
.
Configuring email actions
editYou configure email actions in the actions
array. Action-specific attributes
are specified using the email
keyword.
For example, the following email action uses a template to include data from the watch payload in the email body:
"actions" : { "send_email" : { "email" : { "to" : "<username>@<domainname>", "subject" : "Watcher Notification", "body" : "{{ctx.payload.hits.total}} error logs found" } } }
The id of the action. |
|
The action type is set to |
|
One or more addresses to send the email to. Must be specified in the action definition or in the email account configuration. |
|
The subject of the email can contain static text and Mustache templates. |
|
The body of the email can contain static text and Mustache templates. Must be specified in the action definition or in the email account configuration. |
Configuring email attachments
editYou can attach the execution context payload or data from an any HTTP service to the email notification. There is no limit on the number of attachments you can configure.
To configure attachments, specify a name for the attached file and the type of
attachment: data
, http
or reporting
. The data
attachment type attaches the execution
context payload to the email message. The http
attachment type enables
you to issue an HTTP request and attach the response to the email message. When
configuring the http
attachment type, you must specify the request URL. The
reporting
attachment type is a special type to include PDF rendered dashboards
from kibana. This type is consistently polling the kibana app if the dashboard
rendering is done, preventing long running HTTP connections, that are potentially
killed by firewalls or load balancers in-between.
"actions" : { "email_admin" : { "email": { "to": "John Doe <john.doe@example.com>", "attachments" : { "my_image.png" : { "http" : { "content_type" : "image/png", "request" : { "url": "http://example.org/foo/my-image.png" } } }, "dashboard.pdf" : { "reporting" : { "url": "http://example.org:5601/api/reporting/generate/dashboard/Error-Monitoring" } }, "data.yml" : { "data" : { "format" : "yaml" } } } } } }
The ID of the attachment, which is used as the file name in the email attachment. |
|
The type of the attachment and its specific configuration. |
|
The URL from which to retrieve the attachment. |
|
Data attachments default to JSON if you don’t specify the format. |
Table 85. http
attachment type attributes
Name | Description |
---|---|
|
Sets the content type for the email attachment. By default, the content type is extracted from the response sent by the HTTP service. You can explicitly specify the content type to ensure that the type is set correctly in the email in case the response does not specify the content type or it’s specified incorrectly. Optional. |
|
Configures as an attachment to sent with disposition |
|
Contains the HTTP request attributes. At a minimum, you must
specify the |
Table 86. data
attachment type attributes
Name | Description |
---|---|
|
Attaches the watch data, equivalent to specifying |
Table 87. reporting
attachment type attributes
Name | Description |
---|---|
|
The URL to trigger the dashboard creation |
|
Configures as an attachment to sent with disposition |
|
The reporting attachment type tries to poll regularly to receive the
created PDF. This configures the number of retries. Defaults to |
|
The time to wait between two polling tries. Defaults to |
|
Additional auth configuration for the request |
|
Additional proxy configuration for the request |
Attaching reports to an email
editYou can use the reporting
attachment type in an email
action to automatically
generate a Kibana report and distribute it via email.
Email action attributes
editName | Required | Default | Description |
---|---|---|---|
|
no |
the default account |
The email account to use to send the email. |
|
no |
- |
The email address from which the email
will be sent. The |
|
yes |
- |
The email addresses of the |
|
no |
- |
The email addresses of the |
|
no |
- |
The email addresses of the |
|
no |
- |
The email addresses that will be set on the
message’s |
|
no |
- |
The subject of the email. The subject can be static text or contain Mustache templates. |
|
no |
- |
The body of the email. When this field holds a string, it will default to the text body of the email. Set as an object to specify either the text or the html body or both (using the fields below) |
|
yes |
- |
The plain text body of the email. The body can be static text
or contain Mustache templates. The email |
|
yes |
- |
The html body of the email. The body can be static text or
contain Mustache templates. This body will be
sanitized to remove dangerous content such as scripts. This
behavior can be disabled by setting
|
|
no |
- |
The priority of this email. Valid values are: |
|
no |
- |
Attaches the watch payload ( |
|
no |
false |
Indicates whether the watch execution data should be attached
to the email. You can specify a Boolean value or an object.
If |
|
no |
yaml |
When |
- Email Address
-
An email address can contain two possible parts—the address itself and an
optional personal name as described in RFC 822.
The address can be represented either as a string of the form
user@host.domain
orPersonal Name <user@host.domain>
. You can also specify an email address as an object that containsname
andaddress
fields.
Configuring email accounts
editWatcher can send email using any SMTP email service. Email messages can contain basic HTML tags. You can control which groups of tags are allowed by Configuring HTML Sanitization Options.
You configure the accounts Watcher can use to send email in the
xpack.notification.email
namespace in elasticsearch.yml
.
If your email account is configured to require two step verification, you need to generate and use a unique App Password to send email from Watcher. Authentication will fail if you use your primary password.
Currently, neither Watcher nor Shield provide a mechanism to encrypt
settings in elasticsearch.yml
. Because the email account credentials appear
in plain text, you should limit access to elasticsearch.yml
to the user that
you use to run Elasticsearch.
Watcher provides three email profiles that control how MIME messages are
structured: standard
(default), gmail
, and outlook
. These profiles
accommodate differences in how various email systems interpret the MIME
standard. If you are using Gmail or Outlook, we recommend using the
corresponding profile. Use the standard
profile if you are using another
email system.
For more information about configuring Watcher to work with different email systems, see:
If you configure multiple email accounts, you must either configure a default
account or specify which account the email should be sent with in the
email
action.
xpack.notification.email: default_account: team1 account: team1: ... team2: ...
Sending email from Gmail
editUse the following email account settings to send email from the Gmail SMTP service:
xpack.notification.email.account: gmail_account: profile: gmail smtp: auth: true starttls.enable: true host: smtp.gmail.com port: 587 user: <username>
In order to store the account SMTP password, use the keystore command (see secure settings)
bin/elasticsearch-keystore xpack.notification.email.account.gmail_account.smtp.secure_password
If you get an authentication error that indicates that you need to continue the sign-in process from a web browser when Watcher attempts to send email, you need to configure Gmail to Allow Less Secure Apps to access your account.
If two-step verification is enabled for your account, you must generate and use a unique App Password to send email from Watcher. See Sign in using App Passwords for more information.
Sending email from Outlook.com
editUse the following email account settings to send email action from the Outlook.com SMTP service:
xpack.notification.email.account: outlook_account: profile: outlook smtp: auth: true starttls.enable: true host: smtp-mail.outlook.com port: 587 user: <email.address>
In order to store the account SMTP password, use the keystore command (see secure settings)
bin/elasticsearch-keystore xpack.notification.email.account.outlook_account.smtp.secure_password
When sending emails, you have to provide a from address, either a default one in your account configuration or as part of the email action in the watch.
You need to use a unique App Password if two-step verification is enabled. See App passwords and two-step verification for more information.
Sending email from Amazon SES (Simple Email Service)
editUse the following email account settings to send email from the Amazon Simple Email Service (SES) SMTP service:
xpack.notification.email.account: ses_account: smtp: auth: true starttls.enable: true starttls.required: true host: email-smtp.us-east-1.amazonaws.com port: 587 user: <username>
In order to store the account SMTP password, use the keystore command (see secure settings)
bin/elasticsearch-keystore xpack.notification.email.account.ses_account.smtp.secure_password
You need to use your Amazon SES SMTP credentials to send email through Amazon SES. For more information, see Obtaining Your Amazon SES SMTP Credentials. You might also need to verify your email address or your whole domain at AWS.
Sending email from Microsoft Exchange
editUse the following email account settings to send email action from Microsoft Exchange:
xpack.notification.email.account: exchange_account: profile: outlook email_defaults: from: <email address of service account> smtp: auth: true starttls.enable: true host: <your exchange server> port: 587 user: <email address of service account>
Some organizations configure Exchange to validate that the |
|
Many organizations support use of your email address as your username, though it is a good idea to check with your system administrator if you receive authentication-related failures. |
In order to store the account SMTP password, use the keystore command (see secure settings)
bin/elasticsearch-keystore xpack.notification.email.account.exchange_account.smtp.secure_password
Configuring HTML sanitization options
editThe email
action supports sending messages with an HTML body. However, for
security reasons, Watcher sanitizes
the HTML.
You can control which HTML features are allowed or disallowed by configuring the
xpack.notification.email.html.sanitization.allow
and
xpack.notification.email.html.sanitization.disallow
settings in
elasticsearch.yml
. You can specify individual HTML elements and
HTML feature groups. By default, Watcher allows the following
features: body
, head
, _tables
, _links
, _blocks
, _formatting
and
img:embedded
.
For example, the following settings allow the HTML to contain tables and block
elements, but disallow <h4>
, <h5>
and <h6>
tags.
xpack.notification.email.html.sanitization: allow: _tables, _blocks disallow: h4, h5, h6
To disable sanitization entirely, add the following setting to
elasticsearch.yml
:
xpack.notification.email.html.sanitization.enabled: false
On this page
- Configuring email actions
- Configuring email attachments
- Attaching reports to an email
- Email action attributes
- Configuring email accounts
- Sending email from Gmail
- Sending email from Outlook.com
- Sending email from Amazon SES (Simple Email Service)
- Sending email from Microsoft Exchange
- Configuring HTML sanitization options