- Elasticsearch Guide: other versions:
- What is Elasticsearch?
- What’s new in 7.10
- Getting started with Elasticsearch
- Set up Elasticsearch
- Installing Elasticsearch
- Configuring Elasticsearch
- Setting JVM options
- Secure settings
- Auditing settings
- Circuit breaker settings
- Cluster-level shard allocation and routing settings
- Cross-cluster replication settings
- Discovery and cluster formation settings
- Field data cache settings
- HTTP
- Index lifecycle management settings
- Index management settings
- Index recovery settings
- Indexing buffer settings
- License settings
- Local gateway settings
- Logging
- Machine learning settings
- Monitoring settings
- Node
- Network settings
- Node query cache settings
- Search settings
- Security settings
- Shard request cache settings
- Snapshot lifecycle management settings
- Transforms settings
- Transport
- Thread pools
- Watcher settings
- Important Elasticsearch configuration
- Important System Configuration
- Bootstrap Checks
- Heap size check
- File descriptor check
- Memory lock check
- Maximum number of threads check
- Max file size check
- Maximum size virtual memory check
- Maximum map count check
- Client JVM check
- Use serial collector check
- System call filter check
- OnError and OnOutOfMemoryError checks
- Early-access check
- G1GC check
- All permission check
- Discovery configuration check
- Bootstrap Checks for X-Pack
- Starting Elasticsearch
- Stopping Elasticsearch
- Discovery and cluster formation
- Add and remove nodes in your cluster
- Full-cluster restart and rolling restart
- Remote clusters
- Set up X-Pack
- Configuring X-Pack Java Clients
- Plugins
- Upgrade Elasticsearch
- Index modules
- Mapping
- Text analysis
- Overview
- Concepts
- Configure text analysis
- Built-in analyzer reference
- Tokenizer reference
- Token filter reference
- Apostrophe
- ASCII folding
- CJK bigram
- CJK width
- Classic
- Common grams
- Conditional
- Decimal digit
- Delimited payload
- Dictionary decompounder
- Edge n-gram
- Elision
- Fingerprint
- Flatten graph
- Hunspell
- Hyphenation decompounder
- Keep types
- Keep words
- Keyword marker
- Keyword repeat
- KStem
- Length
- Limit token count
- Lowercase
- MinHash
- Multiplexer
- N-gram
- Normalization
- Pattern capture
- Pattern replace
- Phonetic
- Porter stem
- Predicate script
- Remove duplicates
- Reverse
- Shingle
- Snowball
- Stemmer
- Stemmer override
- Stop
- Synonym
- Synonym graph
- Trim
- Truncate
- Unique
- Uppercase
- Word delimiter
- Word delimiter graph
- Character filters reference
- Normalizers
- Index templates
- Data streams
- Ingest node
- Search your data
- Query DSL
- Aggregations
- Bucket aggregations
- Adjacency matrix
- Auto-interval date histogram
- Children
- Composite
- Date histogram
- Date range
- Diversified sampler
- Filter
- Filters
- Geo-distance
- Geohash grid
- Geotile grid
- Global
- Histogram
- IP range
- Missing
- Nested
- Parent
- Range
- Rare terms
- Reverse nested
- Sampler
- Significant terms
- Significant text
- Terms
- Variable width histogram
- Subtleties of bucketing range fields
- Metrics aggregations
- Pipeline aggregations
- Bucket aggregations
- EQL
- SQL access
- Overview
- Getting Started with SQL
- Conventions and Terminology
- Security
- SQL REST API
- SQL Translate API
- SQL CLI
- SQL JDBC
- SQL ODBC
- SQL Client Applications
- SQL Language
- Functions and Operators
- Comparison Operators
- Logical Operators
- Math Operators
- Cast Operators
- LIKE and RLIKE Operators
- Aggregate Functions
- Grouping Functions
- Date/Time and Interval Functions and Operators
- Full-Text Search Functions
- Mathematical Functions
- String Functions
- Type Conversion Functions
- Geo Functions
- Conditional Functions And Expressions
- System Functions
- Reserved keywords
- SQL Limitations
- Scripting
- Data management
- ILM: Manage the index lifecycle
- Overview
- Concepts
- Automate rollover
- Manage Filebeat time-based indices
- Index lifecycle actions
- Configure a lifecycle policy
- Migrate index allocation filters to node roles
- Resolve lifecycle policy execution errors
- Start and stop index lifecycle management
- Manage existing indices
- Skip rollover
- Restore a managed data stream or index
- Monitor a cluster
- Frozen indices
- Roll up or transform your data
- Set up a cluster for high availability
- Snapshot and restore
- Secure a cluster
- Overview
- Configuring security
- User authentication
- Built-in users
- Internal users
- Token-based authentication services
- Realms
- Realm chains
- Active Directory user authentication
- File-based user authentication
- LDAP user authentication
- Native user authentication
- OpenID Connect authentication
- PKI user authentication
- SAML authentication
- Kerberos authentication
- Integrating with other authentication systems
- Enabling anonymous access
- Controlling the user cache
- Configuring SAML single-sign-on on the Elastic Stack
- Configuring single sign-on to the Elastic Stack using OpenID Connect
- User authorization
- Built-in roles
- Defining roles
- Granting access to Stack Management features
- Security privileges
- Document level security
- Field level security
- Granting privileges for data streams and index aliases
- Mapping users and groups to roles
- Setting up field and document level security
- Submitting requests on behalf of other users
- Configuring authorization delegation
- Customizing roles and authorization
- Enabling audit logging
- Encrypting communications
- Restricting connections with IP filtering
- Cross cluster search, clients, and integrations
- Tutorial: Getting started with security
- Tutorial: Encrypting communications
- Troubleshooting
- Some settings are not returned via the nodes settings API
- Authorization exceptions
- Users command fails due to extra arguments
- Users are frequently locked out of Active Directory
- Certificate verification fails for curl on Mac
- SSLHandshakeException causes connections to fail
- Common SSL/TLS exceptions
- Common Kerberos exceptions
- Common SAML issues
- Internal Server Error in Kibana
- Setup-passwords command fails due to connection failure
- Failures due to relocation of the configuration files
- Limitations
- Watch for cluster and index events
- Command line tools
- How To
- Glossary of terms
- REST APIs
- API conventions
- Compact and aligned text (CAT) APIs
- cat aliases
- cat allocation
- cat anomaly detectors
- cat count
- cat data frame analytics
- cat datafeeds
- cat fielddata
- cat health
- cat indices
- cat master
- cat nodeattrs
- cat nodes
- cat pending tasks
- cat plugins
- cat recovery
- cat repositories
- cat segments
- cat shards
- cat snapshots
- cat task management
- cat templates
- cat thread pool
- cat trained model
- cat transforms
- Cluster APIs
- Cluster allocation explain
- Cluster get settings
- Cluster health
- Cluster reroute
- Cluster state
- Cluster stats
- Cluster update settings
- Nodes feature usage
- Nodes hot threads
- Nodes info
- Nodes reload secure settings
- Nodes stats
- Pending cluster tasks
- Remote cluster info
- Task management
- Voting configuration exclusions
- Cross-cluster replication APIs
- Data stream APIs
- Document APIs
- Enrich APIs
- Graph explore API
- Index APIs
- Add index alias
- Analyze
- Clear cache
- Clone index
- Close index
- Create index
- Delete index
- Delete index alias
- Delete component template
- Delete index template
- Delete index template (legacy)
- Flush
- Force merge
- Freeze index
- Get component template
- Get field mapping
- Get index
- Get index alias
- Get index settings
- Get index template
- Get index template (legacy)
- Get mapping
- Index alias exists
- Index exists
- Index recovery
- Index segments
- Index shard stores
- Index stats
- Index template exists (legacy)
- Open index
- Put index template
- Put index template (legacy)
- Put component template
- Put mapping
- Refresh
- Rollover index
- Shrink index
- Simulate index
- Simulate template
- Split index
- Synced flush
- Type exists
- Unfreeze index
- Update index alias
- Update index settings
- Resolve index
- List dangling indices
- Import dangling index
- Delete dangling index
- Index lifecycle management APIs
- Ingest APIs
- Info API
- Licensing APIs
- Machine learning anomaly detection APIs
- Add events to calendar
- Add jobs to calendar
- Close jobs
- Create jobs
- Create calendars
- Create datafeeds
- Create filters
- Delete calendars
- Delete datafeeds
- Delete events from calendar
- Delete filters
- Delete forecasts
- Delete jobs
- Delete jobs from calendar
- Delete model snapshots
- Delete expired data
- Estimate model memory
- Find file structure
- Flush jobs
- Forecast jobs
- Get buckets
- Get calendars
- Get categories
- Get datafeeds
- Get datafeed statistics
- Get influencers
- Get jobs
- Get job statistics
- Get machine learning info
- Get model snapshots
- Get overall buckets
- Get scheduled events
- Get filters
- Get records
- Open jobs
- Post data to jobs
- Preview datafeeds
- Revert model snapshots
- Set upgrade mode
- Start datafeeds
- Stop datafeeds
- Update datafeeds
- Update filters
- Update jobs
- Update model snapshots
- Machine learning data frame analytics APIs
- Create data frame analytics jobs
- Create trained models
- Update data frame analytics jobs
- Delete data frame analytics jobs
- Delete trained models
- Evaluate data frame analytics
- Explain data frame analytics
- Get data frame analytics jobs
- Get data frame analytics jobs stats
- Get trained models
- Get trained models stats
- Start data frame analytics jobs
- Stop data frame analytics jobs
- Migration APIs
- Reload search analyzers API
- Repositories metering APIs
- Rollup APIs
- Search APIs
- Searchable snapshots APIs
- Security APIs
- Authenticate
- Change passwords
- Clear cache
- Clear roles cache
- Clear privileges cache
- Clear API key cache
- Create API keys
- Create or update application privileges
- Create or update role mappings
- Create or update roles
- Create or update users
- Delegate PKI authentication
- Delete application privileges
- Delete role mappings
- Delete roles
- Delete users
- Disable users
- Enable users
- Get API key information
- Get application privileges
- Get builtin privileges
- Get role mappings
- Get roles
- Get token
- Get users
- Grant API keys
- Has privileges
- Invalidate API key
- Invalidate token
- OpenID Connect prepare authentication
- OpenID Connect authenticate
- OpenID Connect logout
- SAML prepare authentication
- SAML authenticate
- SAML logout
- SAML invalidate
- SSL certificate
- Snapshot and restore APIs
- Snapshot lifecycle management APIs
- Transform APIs
- Usage API
- Watcher APIs
- Definitions
- Migration guide
- Release notes
- Elasticsearch version 7.10.2
- Elasticsearch version 7.10.1
- Elasticsearch version 7.10.0
- Elasticsearch version 7.9.3
- Elasticsearch version 7.9.2
- Elasticsearch version 7.9.1
- Elasticsearch version 7.9.0
- Elasticsearch version 7.8.1
- Elasticsearch version 7.8.0
- Elasticsearch version 7.7.1
- Elasticsearch version 7.7.0
- Elasticsearch version 7.6.2
- Elasticsearch version 7.6.1
- Elasticsearch version 7.6.0
- Elasticsearch version 7.5.2
- Elasticsearch version 7.5.1
- Elasticsearch version 7.5.0
- Elasticsearch version 7.4.2
- Elasticsearch version 7.4.1
- Elasticsearch version 7.4.0
- Elasticsearch version 7.3.2
- Elasticsearch version 7.3.1
- Elasticsearch version 7.3.0
- Elasticsearch version 7.2.1
- Elasticsearch version 7.2.0
- Elasticsearch version 7.1.1
- Elasticsearch version 7.1.0
- Elasticsearch version 7.0.0
- Elasticsearch version 7.0.0-rc2
- Elasticsearch version 7.0.0-rc1
- Elasticsearch version 7.0.0-beta1
- Elasticsearch version 7.0.0-alpha2
- Elasticsearch version 7.0.0-alpha1
- Dependencies and versions
Watcher email action
editWatcher email action
editUse the email
action to send email notifications. To send email, you must
configure at least one email account in
elasticsearch.yml
.
Email notifications can be plain text or styled using HTML. You can include information from the watch execution payload using templates and attach the entire watch payload to the message.
See Email action attributes for the supported attributes. Any attributes that
are missing from the email action definition are looked up in the email
account configuration. The required attributes must either be set in the email
action definition or the account’s email_defaults
.
Configuring email actions
editYou configure email actions in the actions
array. Action-specific attributes
are specified using the email
keyword.
For example, the following email action uses a template to include data from the watch payload in the email body:
"actions" : { "send_email" : { "email" : { "to" : "username@example.org", "subject" : "Watcher Notification", "body" : "{{ctx.payload.hits.total}} error logs found" } } }
The id of the action. |
|
The action type is set to |
|
One or more addresses to send the email to. Must be specified in the action definition or in the email account configuration. |
|
The subject of the email can contain static text and Mustache templates. |
|
The body of the email can contain static text and Mustache templates. Must be specified in the action definition or in the email account configuration. |
Configuring email attachments
editYou can attach the execution context payload or data from an any HTTP service to the email notification. There is no limit on the number of attachments you can configure.
To configure attachments, specify a name for the attached file and the type of
attachment: data
, http
or reporting
. The data
attachment type attaches the execution
context payload to the email message. The http
attachment type enables
you to issue an HTTP request and attach the response to the email message. When
configuring the http
attachment type, you must specify the request URL. The
reporting
attachment type is a special type to include PDF rendered dashboards
from kibana. This type is consistently polling the kibana app if the dashboard
rendering is done, preventing long running HTTP connections, that are potentially
killed by firewalls or load balancers in-between.
"actions" : { "email_admin" : { "email": { "to": "John Doe <john.doe@example.com>", "attachments" : { "my_image.png" : { "http" : { "content_type" : "image/png", "request" : { "url": "http://example.org/foo/my-image.png" } } }, "dashboard.pdf" : { "reporting" : { "url": "http://example.org:5601/api/reporting/generate/dashboard/Error-Monitoring" } }, "data.yml" : { "data" : { "format" : "yaml" } } } } } }
The ID of the attachment, which is used as the file name in the email attachment. |
|
The type of the attachment and its specific configuration. |
|
The URL from which to retrieve the attachment. |
|
Data attachments default to JSON if you don’t specify the format. |
Table 91. http
attachment type attributes
Name | Description |
---|---|
|
Sets the content type for the email attachment. By default, the content type is extracted from the response sent by the HTTP service. You can explicitly specify the content type to ensure that the type is set correctly in the email in case the response does not specify the content type or it’s specified incorrectly. Optional. |
|
Configures as an attachment to sent with disposition |
|
Contains the HTTP request attributes. At a minimum, you must
specify the |
Table 92. data
attachment type attributes
Name | Description |
---|---|
|
Attaches the watch data, equivalent to specifying |
Table 93. reporting
attachment type attributes
Name | Description |
---|---|
|
The URL to trigger the dashboard creation |
|
Configures as an attachment to sent with disposition |
|
The reporting attachment type tries to poll regularly to receive the
created PDF. This configures the number of retries. Defaults to |
|
The time to wait between two polling tries. Defaults to |
|
Additional auth configuration for the request |
|
Additional proxy configuration for the request |
Attaching reports to an email
editYou can use the reporting
attachment type in an email
action to automatically
generate a Kibana report and distribute it via email.
Email action attributes
editName | Required | Default | Description |
---|---|---|---|
|
no |
the default account |
The email account to use to send the email. |
|
no |
- |
The email address from which the email
will be sent. The |
|
yes |
- |
The email addresses of the |
|
no |
- |
The email addresses of the |
|
no |
- |
The email addresses of the |
|
no |
- |
The email addresses that will be set on the
message’s |
|
no |
- |
The subject of the email. The subject can be static text or contain Mustache templates. |
|
no |
- |
The body of the email. When this field holds a string, it will default to the text body of the email. Set as an object to specify either the text or the html body or both (using the fields below) |
|
no |
- |
The plain text body of the email. The body can be static text or contain Mustache templates. |
|
no |
- |
The html body of the email. The body can be static text or
contain Mustache templates. This body will be
sanitized to remove dangerous content such as scripts. This
behavior can be disabled by setting
|
|
no |
- |
The priority of this email. Valid values are: |
|
no |
- |
Attaches the watch payload ( |
|
no |
false |
Indicates whether the watch execution data should be attached
to the email. You can specify a Boolean value or an object.
If |
|
no |
yaml |
When |
- Email Address
-
An email address can contain two possible parts—the address itself and an
optional personal name as described in RFC 822.
The address can be represented either as a string of the form
user@host.domain
orPersonal Name <user@host.domain>
. You can also specify an email address as an object that containsname
andaddress
fields.
Configuring email accounts
editWatcher can send email using any SMTP email service. Email messages can contain basic HTML tags. You can control which groups of tags are allowed by Configuring HTML Sanitization Options.
You configure the accounts Watcher can use to send email in the
xpack.notification.email
namespace in elasticsearch.yml
.
The password for the specified SMTP user is stored securely in the
Elasticsearch keystore.
If your email account is configured to require two step verification, you need to generate and use a unique App Password to send email from Watcher. Authentication will fail if you use your primary password.
Watcher provides three email profiles that control how MIME messages are
structured: standard
(default), gmail
, and outlook
. These profiles
accommodate differences in how various email systems interpret the MIME
standard. If you are using Gmail or Outlook, we recommend using the
corresponding profile. Use the standard
profile if you are using another
email system.
For more information about configuring Watcher to work with different email systems, see:
If you configure multiple email accounts, you must either configure a default
account or specify which account the email should be sent with in the
email
action.
xpack.notification.email: default_account: team1 account: team1: ... team2: ...
Sending email from Gmail
editUse the following email account settings to send email from the Gmail SMTP service:
xpack.notification.email.account: gmail_account: profile: gmail smtp: auth: true starttls.enable: true host: smtp.gmail.com port: 587 user: <username>
To store the account SMTP password, use the keystore command (see secure settings)
bin/elasticsearch-keystore add xpack.notification.email.account.gmail_account.smtp.secure_password
If you get an authentication error that indicates that you need to continue the sign-in process from a web browser when Watcher attempts to send email, you need to configure Gmail to Allow Less Secure Apps to access your account.
If two-step verification is enabled for your account, you must generate and use a unique App Password to send email from Watcher. See Sign in using App Passwords for more information.
Sending email from Outlook.com
editUse the following email account settings to send email action from the Outlook.com SMTP service:
xpack.notification.email.account: outlook_account: profile: outlook smtp: auth: true starttls.enable: true host: smtp-mail.outlook.com port: 587 user: <email.address>
To store the account SMTP password, use the keystore command (see secure settings)
bin/elasticsearch-keystore add xpack.notification.email.account.outlook_account.smtp.secure_password
When sending emails, you have to provide a from address, either a default one in your account configuration or as part of the email action in the watch.
You need to use a unique App Password if two-step verification is enabled. See App passwords and two-step verification for more information.
Sending email from Amazon SES (Simple Email Service)
editUse the following email account settings to send email from the Amazon Simple Email Service (SES) SMTP service:
xpack.notification.email.account: ses_account: smtp: auth: true starttls.enable: true starttls.required: true host: email-smtp.us-east-1.amazonaws.com port: 587 user: <username>
To store the account SMTP password, use the keystore command (see secure settings)
bin/elasticsearch-keystore add xpack.notification.email.account.ses_account.smtp.secure_password
You need to use your Amazon SES SMTP credentials to send email through Amazon SES. For more information, see Obtaining Your Amazon SES SMTP Credentials. You might also need to verify your email address or your whole domain at AWS.
Sending email from Microsoft Exchange
editUse the following email account settings to send email action from Microsoft Exchange:
xpack.notification.email.account: exchange_account: profile: outlook email_defaults: from: <email address of service account> smtp: auth: true starttls.enable: true host: <your exchange server> port: 587 user: <email address of service account>
Some organizations configure Exchange to validate that the |
|
Many organizations support use of your email address as your username, though it is a good idea to check with your system administrator if you receive authentication-related failures. |
To store the account SMTP password, use the keystore command (see secure settings)
bin/elasticsearch-keystore add xpack.notification.email.account.exchange_account.smtp.secure_password
Configuring HTML sanitization options
editThe email
action supports sending messages with an HTML body. However, for
security reasons, Watcher sanitizes
the HTML.
You can control which HTML features are allowed or disallowed by configuring the
xpack.notification.email.html.sanitization.allow
and
xpack.notification.email.html.sanitization.disallow
settings in
elasticsearch.yml
. You can specify individual HTML elements and
HTML feature groups. By default, Watcher allows the following
features: body
, head
, _tables
, _links
, _blocks
, _formatting
and
img:embedded
.
For example, the following settings allow the HTML to contain tables and block
elements, but disallow <h4>
, <h5>
and <h6>
tags.
xpack.notification.email.html.sanitization: allow: _tables, _blocks disallow: h4, h5, h6
To disable sanitization entirely, add the following setting to
elasticsearch.yml
:
xpack.notification.email.html.sanitization.enabled: false
On this page
- Configuring email actions
- Configuring email attachments
- Attaching reports to an email
- Email action attributes
- Configuring email accounts
- Sending email from Gmail
- Sending email from Outlook.com
- Sending email from Amazon SES (Simple Email Service)
- Sending email from Microsoft Exchange
- Configuring HTML sanitization options