WARNING: Version 5.6 of the Elastic Stack has passed its EOL date.
This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.
Change List
editChange List
edit5.6.16
editThere are no user facing changes in this release.
5.6.15
edit- Security
-
- Fixes an improper permission issue when a new name is attached to an index. (CVE-2019-7611). See Security issues.
5.6.14
editThere are no user facing changes in this release.
5.6.13
editThere are no user facing changes in this release.
5.6.12
editThere are no user facing changes in this release.
5.6.11
editThere are no user facing changes in this release.
5.6.10
editBug fixes
edit- Machine Learning
-
-
Fixed problem that could cause the machine learning
autodetect
process to crash when persisting state or upon hitting the soft memory limit, and hence cause the machine learning job to fail.
-
Fixed problem that could cause the machine learning
- Monitoring
-
-
Requests made to
_xpack/monitoring/_bulk
are now run on the generic thread pool rather than the HTTP worker that handles the request. - Ignores data collection when no cluster UUID exists. This situation occurs when the cluster is initializing and should be ignored.
-
Requests made to
- Security
-
- Reduces the number of object allocations made by X-Pack security when resolving the indices and aliases for a request (#30180).
5.6.9
editEnhancements
edit- Machine Learning
-
- Adds checks for web URLs prior to adding to the links menu in the anomalies table.
Bug fixes
edit- Machine Learning
-
- Fixes the Estimate bucket span button in the multi-metric wizard in Firefox.
- Fixes the formatting of HTML characters in Kibana so that text is rendered correctly.
- Fixes the formatting of partition titles in multi-metric and population job wizards in Kibana.
- Formats the field name in the tooltip for the swimlane labels in Kibana.
- Monitoring
-
- Fixes CPU stats such that it uses "max" as expected (instead of average).
- Security
-
- Fixes the PKI realm bootstrap check such that it works with secure settings.
- Upgrade Assistant
-
-
Fixes the deprecation check against
match_mapping_type
such that it recognizes * (wildcard) as a valid matching type. -
Fixes the check for the
cluster.name
inpath.data
such that it returns a critical message. - Updates logic that converts string fields to text or keyword to account for doc_values.
-
Fixes the deprecation check against
5.6.8
editBug fixes
edit- Security
-
- If the realm uses native role mappings and the security index health changes, the realm caches are cleared. For example, they are cleared when the index recovers from a red state, when the index is deleted, or when the index becomes outdated.
- Fixed a bug that could prevent auditing to a remote index if the remote cluster was re-started at the same time as the audited cluster.
-
Removed AuthorityKeyIdentifier’s Issuer and Serial number from certificates
generated by
certgen
andcertutil
. This improves compatibility with certificate verification in Kibana.
5.6.7
editBug fixes
edit- Other
-
-
Fixed a deprecation check that was erroneously failing when the
path.data
setting contained a trailing/
.
-
Fixed a deprecation check that was erroneously failing when the
- Watcher
-
- Ensured collections obtained from scripts are protected from self-referencing. See https://github.com/elastic/elasticsearch/pull/28335.
5.6.6
editJanuary 16, 2018
Enhancements
edit- Watcher
-
- Ensured the watcher thread pool size is reasonably bound. In particular, the watcher thread pool size is now five times the number of processors until 50 threads are reached. If more than 50 cores exist and 50 threads exist, the watch thread pool size grows to match the number of processors.
Bug fixes
edit- Monitoring
-
- Fixed monitoring delays that could occur when Kibana started and prevent timely cluster alerts.
- Security
-
- Fixed exception that occurred when using auditing and transport clients. In particular, the problem occurred when the number of processors on the transport client did not match the number of processors on the server.
- Fixed an issue in the Active Directory realm when following referrals that resulted in an increase in the number of connections made to Active Directory.
- Watcher
-
- Fixed the pagerduty action to send context data. For more information, see PagerDuty Action.
5.6.5
editDecember 6, 2017
Enhancements
edit- Machine Learning
-
-
Added machine learning deprecation checks in the Upgrade Assistant in Kibana. For example, it checks whether the datafeed uses deprecated query options.
If you have used machine learning on your node at any point in the past, re-enable machine learning before you run the Upgrade Assistant. Datafeeds that contain deprecated search options can cause the upgrade to fail even when machine learning is disabled.
-
- Security
-
- Added debug logging during role resolution, which helps explain how each role was resolved.
Bug fixes
edit- Dev Tools
-
- Added enforcement for the use of a single type after Kibana index migration. For more information, see Removal of mapping types.
- Machine Learning
-
- Fixed problems that resulted in incorrect results after reverting a model snapshot.
- Fixed scenario where the presence of any read only indices was incorrectly preventing machine learning job deletion on other indices.
- Fixed problems that caused datafeeds to fail with "Cannot parse scroll ID" errors.
- Fixed an intermittent problem that prevented jobs from opening when a node had almost reached the maximum number of open jobs.
- Fixed exception handling such that when a job fails, the datafeed also stops.
- Fixed representation of non-ASCII characters in log messages that are forwarded from machine learning native processes to Elasticsearch.
- Monitoring
-
-
Fixed the
xpack.monitoring.node_resolver
setting such that it works properly whenname
ortransport_address
values are used. This setting was deprecated in 5.5.0 and will be removed in 7.0. For more information about this setting, see Monitor Settings in Kibana - Added checks for clusters that report before the cluster state is loaded.
-
Fixed the
- Security
-
- Fixed an IllegalArgumentException that could occur when retrieving cluster settings if a realm had been configured to use secure settings from the Elasticsearch keystore.
-
Enabled read-only access to the index audit log by the
_xpack
internal user. For more information, see Internal Users.
5.6.4
editNovember 7, 2017
Enhancements
edit- Watcher
-
- Added verification that the required templates exist before Watcher starts. For more information, see Scripts and Templates.
-
Added the
xpack.watcher.history.cleaner_service.enabled
setting. You can use this setting to enable or disable the cleaner service, which removes previous versions of Watcher indices (for example, .watcher-history*) when it determines that they are old. For more information, see Watcher Settings in Elasticsearch.
Bug fixes
edit- Machine Learning
-
- Fixed problems when model plot is enabled and there are sparse metrics.
- Security
-
-
Addressed problems caused by unknown mappings in the
.security
index. Unknown mappings are now ignored. - Improved auditing such that it automatically restarts its failed services. Also increased the default queue size to accommodate high traffic clusters.
- Fixed authorization issues for bulk requests that use index aliases.
-
Enabled PKI realms to obtain the password for the truststore from either the
truststore.secure_password
or thetruststore.password
setting. For more information, see PKI Realm Settings. - Fixed an issue that caused LDAP authentication requests to be slow and to require multiple binds when authenticating in user search mode.
-
Addressed problems caused by unknown mappings in the
- Watcher
-
- Fixed failure to update the watch status when the master node changes during the execution of the watch. In certain cases, the newly elected master node was not able to update the watch status after the execution.
5.6.3
editOctober 10, 2017
Bug fixes
edit- Machine Learning
-
- Fixed the Get Buckets API such that if you specify a timestamp in the URL, it is now possible to supply a body that contains other parameters.
-
Removed the hard-coded
query_delay
value from the simple job creation wizards in Kibana. By default, that property contains a random value. For more information about thequery_delay
property, see Datafeed Resources.
- Monitoring
-
-
Logstash: Removed requirement for
xpack.monitoring.elasticsearch.username
andxpack.monitoring.elasticsearch.password
settings inlogstash.yml
when X-Pack security is disabled.
-
Logstash: Removed requirement for
- Security
-
- Fixed loading of role mappings after upgrading indices.
- Watcher
-
-
Changed the watch history index pattern from
.watcher-history-3-*
to.watcher-history-*
. This addresses a problem where watches were not appearing in the Management > Elasticsearch > Watcher > Watches > Watch History in Kibana.
-
Changed the watch history index pattern from
5.6.2
editSeptember 26, 2017
Bug fixes
edit- Machine Learning
-
- Fixed memory consumption and machine learning initialization on Windows machines that use non-English locales.
- Monitoring
-
- Fixed handling of cluster alerts for X-Pack license expiration during a rolling upgrade to 6.x.
- Security
-
-
Added backwards compatibility for the internal
_xpack_security
user. For more information about internal users, see Setting Up User Authentication. - Fixed ability to "run as" (impersonate) another user in an Active Directory realm. For more information see Submitting Requests on Behalf of Other Users.
-
Added backwards compatibility for the internal
- Watcher
-
- Fixed link to the JSON syntax documentation in the Kibana New Watch page.
- Fixed the state of an executed watch to respect the high level state of the watch response.
5.6.1
editSeptember 18, 2017
Enhancements
edit- Machine Learning
-
- Added random offset of 1-2 hours to the daily maintenance task execution time. This means the daily maintenance task now starts between 00:30 and 02:30 local time. This offset staggers the load on a single physical machine that runs many Elasticsearch nodes, as is common in cloud environments.
-
Randomized the default datafeed query delay to avoid having multiple jobs starting
their searches at the same time. For more information about the
query_delay
property, see Datafeed Resources.
Bug fixes
edit- Machine Learning
-
- Fixed calculation of the query delay for real-time datafeeds.
-
Improved adherence to the
model_memory_limit
when many entities are created in the same bucket.
- Monitoring
-
-
Updated the Cleaner Service to remove previous versions of X-Pack monitoring
indices (for example,
.monitoring-es-2-*
,.monitoring-data-2
,.monitoring-alerts-2
, and.marvel-*
) when it determines that they are old.
-
Updated the Cleaner Service to remove previous versions of X-Pack monitoring
indices (for example,
- Security
-
-
Registered the legacy
truststore.password
setting for the PKI realm. -
Added a
_xpack_security
internal user, which is responsible for the X-Pack security operations that take place inside an Elasticsearch cluster. For more information, see Internal Users.
-
Registered the legacy
5.6.0
editSeptember 11, 2017
Deprecations
edit-
The
user_search.attribute
setting is deprecated and is replaced byuser_search.filter
. See LDAP Realm Settings. -
The use of the
system_key
file for encrypting sensitive values is deprecated. It will be removed in 6.0. In order to continue using watches with encrypted data, use theelasticsearch-keystore
tool to store the key in the secure settings keystore. See Secure Settings. -
The
time
field in the Profile API, which shows human readable timing output, has been deprecated. It will be replaced by thetime_in_nanos
field, which displays the elapsed time in nanoseconds. The Profiler now usestime_in_nanos
. For more information about the Profiler, see Profiling your Queries and Aggregations.
New features
edit- Added Migration Assistance, Migration Upgrade, and Deprecation Info APIs, which simplify upgrading X-Pack indices from one version to another. For more information, see Migration APIs.
- The new Upgrade Assistant in Kibana helps you migrate your indices to a version 6 compatible format. After using the assistant you can more easily upgrade the Elastic Stack to version 6. The assistant provides a deprecations checker (the "Cluster Checkup" tool), a Reindex Helper, and lets you toggle Deprecation Logging. For more information, see Upgrade Assistant.
Enhancements
edit- Dev Tools
-
- Improved backwards compatibility with version 5.3 and earlier in the Profiler.
- Machine Learning
-
- Added a bucket span estimator to multi-metric job creation in Kibana. It runs bucket span tests for each detector and suggests a bucket span value for the job.
-
Added
xpack.ml.max_open_jobs
as a node attribute. See Machine Learning Settings in Elasticsearch. -
The
model_memory_limit
property can be either a number or a string with units. See Analysis Limits.
- Monitoring
-
-
Added read-only access to the
.monitoring-*
indices for thekibana_system
role. See Built-in Roles. - If there is an error while fetching data in Kibana, better messages are provided.
-
Added read-only access to the
- Security
-
- User and role names can be longer with more characters (1024 characters instead of 30 characters). For more information, see Defining Roles and User Management APIs.
-
Removed write privileges from the built-in
reporting_user
. -
Added the
user_search.filter
setting for the LDAP user search filter. See LDAP Realm Settings. -
Improved error reporting when an I/O problem occurs in the
users
tool. - Enabled the Active Directory UPN authenticator to work with suffixes. For more information, see Active Directory User Authentication.
- Added active directory bind user and user lookup support. For more information, see Active Directory User Authentication.
- Added secure versions of SSL passphrases.
- More concise logging for reserved realm authentication failures.
- Added scroll support for cross cluster search.
- Removed local security checks and restrictions on remote cluster search. For more information, see Cross Cluster Search and Security.
Bug fixes
edit- Machine Learning
-
- Fixed error in Kibana when saving an advanced job that does not have a datafeed.
- Fixed anomaly charts when the datafeed has an empty types array.
- Fixed calculation of bucket count and empty bucket count statistics.
- Added verification that metric value is finite before writing JSON records.
- Fixed failures when using saved search for single and multi-metric job creation.
- Fixed error in the console log when using Kibana 5.6.0 with Elasticsearch 6.0.0 and creating a single metric job.
- Include datafeed retry in the event of a broken scroll.
-
Single Metric View now checks for use of
terms
field inmodel_plot_config
. - Fixed attempts to match terms against a non-existent by field for model plot.
- Management
-
- Fixed the index patterns drop-down on the Roles page in Kibana. It was displaying Kibana document IDs instead of index patterns.
- Monitoring
-
- Fixed the nodes API such that it no longer sorts nodes that are offline above nodes that are online.
- Fixed blank machine learning Jobs label in Kibana.
- Added support for future time ranges in charts.
- Removed extended bounds from lists, which now display the latest value instead of a blank future value.
- Fixed occasional initial spike in derivative metric charts.
-
Added warning when using the deprecated
xpack.monitoring.loggingTag
setting. - Clarified the error message when a cluster is not found within the selected time range.
-
By default, the old cluster alerts index (
.monitoring-alerts-2
) is no longer used. If you are still monitoring 5.4 clusters, add that index to thexpack.monitoring.cluster_alerts.index
setting. - Enabled Kibana 5.6 to work with the next major version of Elasticsearch.
-
Removed the
dedicated indices_stats
collector by combining it with theindex_stats collector
.
- Reporting
-
- Resolved issues with X-Pack reporting polling not waiting for a call to finish before queueing up a new call. Also added a delay before continuing when polling encounters an error.
- Fixed error that occurred when you clicked the Reporting button in Kibana after you created but did not save a visualization.
- Security
-
- Fixed X-Pack Security permission issue (ESA-2017-18).
-
Improved the safety of file updates in the
x-pack/users
tool. - Bootstrap checks no longer fail when checking secure settings.
- Fixed validation of the input parameters in the create token API.
- Updated the bootstrap check to read the token passphrase earlier.
- Clear security caches when the security index recovers from being temporarily unavailable.
- Fixed invalid character errors by validating filenames both before and after resolving the path.
- Fixed login failure messages in Kibana when Elasticsearch is not available.
- Watcher
-
- Fixed problems in the Create Watch pages in Kibana on FireFox.
- Ensured that no new watches can be put in memory when Watcher is stopped.
5.5.3
editSeptember 11, 2017
Bug fixes
edit- Machine Learning
-
- Reverted validation checks for empty strings in analysis config.
-
Avoided repeated updates of results index mappings for
_meta
field. - Fixed handling of simultaneous calls to forcefully stop and delete datafeeds.
- Security
-
- Fixed X-Pack Security permission issue (ESA-2017-18).
- Watcher
-
-
Fixed the
emit_stacktraces
parameter in the stats API.
-
Fixed the
5.5.2
editAugust 17, 2017
Breaking Changes
edit- Reporting
-
-
When X-Pack security is enabled, user authorization for creating reports is
controlled by the
xpack.reporting.roles.allow
setting in thekibana.yml
, which defaults to['reporting_user']
. When they are assigned to the built-inreporting_user
role, users no longer have write access to the underlying X-Pack reporting indices in Elasticsearch. If you use custom reporting roles, the write privilege to the indices must be removed and the role must be added to thexpack.reporting.roles.allow
setting. See Reporting Settings in Kibana.
-
When X-Pack security is enabled, user authorization for creating reports is
controlled by the
Bug fixes
edit- Machine Learning
-
- Fixed mapping problems in the results index after an upgrade from 5.4 to 5.5.
- Fixed error handling when you try to create a job and the results or state index is closed.
- Closed InputStream after its use, to avoid potential memory issues.
- Improved handling of errors when the native controller dies.
- Added closing jobs to the node capacity check, which determines whether enough threads exist to start a new process.
- Added ability to force close and force delete jobs in the "closing" state.
- Fixed job creation in Kibana for saved searches with a blank query string.
- Added missing validations for the analysis configuration, such as handling empty strings.
- Monitoring
-
- Fixed the sort order of the top cluster alerts in Kibana.
- Security
-
- Fixed validation of the username and password in the CreateTokenRequest.
- Updated the bootstrap check such that the token passphrase is read earlier. This avoids issues where the node startup might have failed because the secure settings keystore was already closed.
Enhancements
edit- Machine Learning
-
- Added Elasticsearch version as meta data in machine learning indices.
- Security
-
-
Added
xpack.security.useDefaultEsCredentials
setting, which you can use to disable the default Elasticsearch credentials in Kibana. See Security Settings in Kibana.
-
Added
5.5.1
editJuly 25, 2017
Bug Fixes
edit- Watcher
-
- Resetting the acknowledgement state of an action with a condition, if the watch wide condition was true, has been fixed
5.5.0
editJune 29, 2017
Breaking Changes
edit- Machine learning
-
- If you have jobs that were created in the beta, you cannot open or run those jobs from nodes that have been upgraded to Version 5.5. You can, however, view the results of the old jobs. You can also use Kibana to clone your old jobs.
- Monitoring
-
-
All index patterns were upgraded to a new 6.0-compatible format that leverage single mapping
_type
(doc
). Instead, a new field namedtype
was added.-
Any manually written queries need only change
_type
to usetype
.
-
Any manually written queries need only change
-
The
cluster_state
type was merged into thecluster_stats
type.- Pre-existing data will still be detected.
-
The
cluster_info
type was merged into thecluster_stats
type.- Pre-existing data will not be detected.
-
Monitoring no longer uses the
.monitoring-data-2
index.-
All data now exists exclusively in the time-based indices (
.monitoring-*
). - Clusters reporting to 5.5 from 5.0 - 5.4 should not be impacted if the ingest pipeline is enabled (default).
- Any monitored cluster that is no longer reported to the monitoring cluster once the cluster is upgraded to 5.5, will not appear in the UI.
-
Old clusters (5.0 - 5.4) still expect the old templates to exist, so they are created by the HTTP exporter unless that feature is disabled.
-
You can set
xpack.monitoring.exporters.<exporter-name>.index.template.create_legacy_templates: false
to disable the specification of old templates by the HTTP exporter. - This will be disabled by default in 6.0.
-
You can set
-
All data now exists exclusively in the time-based indices (
-
New Features
edit- Machine learning
-
- Added option to create a basic alert in Watcher, available when using the Single or Multi Metric job creation wizard.
- Added bucket span estimator which calculates the minimum viable bucket span, available when using the Single or Multi Metric job creation wizards. This is experimental.
- Monitoring
-
- All indices are now exclusively time-based, which makes curation much simpler.
- The Monitoring UI has become keyboard accessible.
- Security
-
- A new Role Mapping API has been introduced to supplement the existing file based mappings.
Enhancements
edit- Machine learning
-
-
Added
Sparse data
check box forsum
andcount
aggregations when using the Single or Multi Metric job creation wizards. This will configure job to use thenon_null_sum
andnon_zero_count
functions respectively. - Single Metric Viewer now allows selection of detector and entity.
- Added low and high median detection functions.
- Improved handling for closing and deleting jobs and for stopping and deleting datafeeds. Especially relevant for larger jobs which take longer to close. Includes a force delete API option.
- Improved handling for stopping native processes.
- Automatic detection in datafeed for field storage locations.
-
New aliases
.ml-anomalies-.write-*
now used for writing results. -
For 6.0 compatability,
types
is no longer required in the datafeed config. -
Job configuration now returns
detector_index
andjob_version
. - Analytics refactor to create a single feature model.
-
Added
- Monitoring
-
-
http
exporters can be configured to not create legacy templates by settingindex.template.create_legacy_templates: false
as part of the exporter’s configuration.- This feature is new in 5.5 to support 5.0 - 5.4 clusters. If you only monitor clusters running 5.5 or newer, then this setting should be used.
-
The
local
exporter does not support this setting because all nodes should be upgraded to the same version when it is used.
- Machine Learning Jobs are now listed alongside nodes and indices.
- Cluster Alerts now show when they were initially triggered.
- Numerous background requests have been combined, which should speed up the UI.
-
Templates, pipelines, and cluster alerts are replaced based on their
version
.
-
- Security
-
-
Hide sensitive realm configuration in the
_nodes
API (ESA-2017-10).
-
Hide sensitive realm configuration in the
- Watcher
-
-
The
reporting
email attachment now has support for HTTP proxies
-
The
Bug Fixes
edit- Machine learning
-
- Fix to include basePath in UI and support for using a proxy.
- Fix styling issues in IE11.
- Fix to respect start time when resuming a datafeed.
- Fix to include quantiles and model stats when reverting a model snapshot.
- Fix to prevent job from being permanently stuck in a closing state.
- Monitoring
-
-
An issue existed that prevented the UI from working when indices existed with
the name
nodes
has been resolved.
-
An issue existed that prevented the UI from working when indices existed with
the name
- Watcher
-
-
The search input now works with an empty
body
field
-
The search input now works with an empty
5.4.3
editJune 27, 2017
Bug Fixes
edit- Security
-
-
Remove redundant
name
attribute from password and username fields. -
Provide
username
as credentials property during cookie-based authentication.
-
Remove redundant
- Monitoring
-
- Shard activity caused by restoring snapshots will now properly show the snapshot repository and snapshot names.
5.4.2
editJune 20, 2017
Bug Fixes
edit- Monitoring
-
-
Removed
_field_stats
calls from the Elasticsearch queries, which was causing deprecation warnings to be logged.
-
Removed
- Watcher
-
-
Start
croneval
command line tool correctly, when used in packaged environments
-
Start
5.4.1
editJune 1, 2017
Bug Fixes
edit- Machine Learning
-
- Added automatic disabling of machine learning on tribe nodes to avoid conflicts.
- Removed the automatic creation of Kibana index patterns.
- Fixed escaping values in Explorer dashboard for metric queries.
- Fixed job state in error messages when a job deletion fails.
- Improved error messages when doc values are unavailable.
- Fixed an issue where attempts were made to get the native code version even though machine learning was not enabled.
- Improved error reporting by validating job configurations earlier.
- Fixed handling of field names that contain round brackets.
- Added basePath to all machine learning API requests in the user interface.
- Disallowed normal closure of failed jobs; you must now force closure of failed jobs.
- Improved handling of errors when indexing machine learning results; the job no longer deadlocks in this scenario.
- Added fix to avoid the need to restart master nodes twice during rolling upgrades to Version 5.4.
- Improved shutdown such that it only waits for the native controller to shut down on platforms that machine learning native programs are available on.
- Monitoring
-
- Added "Nodes" breadcrumb on the Logstash node detail page.
- Fixed an issue where brushing over the charts would cause an error to print out in the JavaScript console.
- Fixed an issue where sending a SIGHUP signal to the Kibana server would cause Kibana monitoring to stop sending metrics.
- Fixed a page rendering issue on the cluster listing page when the cluster alerts feature is disabled.
- Security
-
- Fixed an issue where disabling security and starting Kibana with Elasticsearch unreachable would cause an HTTP Status 500 Server Error.
- Tile Map
-
- Users with a Basic license now have access to 18 zoom levels on the tile map. The behavior now correctly matches the documentation.
- Watcher UI
-
- Added error handling for action acknowledgement.
5.4.0
editMay 4, 2017
New Features
edit- Machine learning
-
- Introduces machine learning beta for the Elastic Stack. See Machine Learning in the Elastic Stack.
- Monitoring
-
- Top Cluster Alerts are now displayed in the Monitoring UI.
- Security
-
- X-Pack extensions can implement custom roles providers that define new roles without adding them to the existing existing native or file stores. This feature requires a Platinum license.
- Watcher
-
- Added UI for managing Alerts.
Enhancements
edit- Graph
-
- Improved “add links to existing terms” behavior. Previously, adding links to existing vertices sometimes failed to discover links that were present in the search index. Now, all the relevant data is taken into account. Due to memory limitations, we consider a max of 100 vertices at a time for link discovery. If more than 100 vertices are selected in the UI, we pick a random sub-selection of 100 vertices.
- Monitoring
-
-
Monitoring no longer requires the
kibana_user
role to have thecluster:monitor
privilege. Thecluster:monitor
privilege has been removed from thekibana_user
role in X-Pack Security. -
The X-Pack banner is hidden for all users of the Kibana instance when
initially dismissed. It can be shown again by changing the setting
xPackMonitoring:showBanner
in Kibana’s Advanced Settings page. Previously this setting only persisted per-browser in local storage. - The Elasticsearch clusters page no longer indicates the "primary" cluster with an asterisk (the cluster that the instance of Kibana is connected to). The check for which cluster is "primary" is to determine, when there are multiple Basic-license clusters, which can be supported by the Monitoring UI app for the Single Cluster Monitoring feature that comes with the free Basic license. Now in 5.4.0, that check only happens if there are multiple Basic-license clusters found in the monitoring data. Supporting a single Basic-license cluster still works, only the asterisk icon is gone from the cluster listing. As always, paid licenses grant support for multi-cluster monitoring.
-
Monitoring no longer requires the
- Security
-
- A new configuration setting is available to disable support for the default password ("changeme").
-
Added security privileges for machine learning:
manage_ml
andmonitor_ml
. -
Added
machine_learning_admin
andmachine_learning_user
roles. -
Added
watcher_user
andwatcher_admin
roles. -
Users that are authenticated using the
ldap
oractive_directory
realms now have their DN and list of groups included in their user metadata under the keysldap_dn
andldap_groups
. This metadata is available in DLS query templates. -
Added
has_privileges
API to determine which index privileges a user has. -
Added a new
ignore_referral_errors
setting to theldap
andactive_directory
realms. Setting this tofalse
means that LDAP errors that occur while following an LDAP referral are treated as fatal errors and cause authentication to fail. Defaults totrue
.
- Watcher
-
- Comma-separated email addresses are now supported in a watch’s to/cc/bcc fields.
- X-Pack
-
-
Dismissing the X-Pack Welcome Banner now dismisses the banner for all
users and browsers. The banner setting is stored as a Kibana Advanced
Setting in the
.kibana
index along with the phone home setting. You can modify these settings from Advanced Settings in Kibana.
-
Dismissing the X-Pack Welcome Banner now dismisses the banner for all
users and browsers. The banner setting is stored as a Kibana Advanced
Setting in the
Bug Fixes
edit- Security
-
- Index names are now correctly resolved if the index name pattern contains both date math and wildcards.
- LDAP error handling now detects additional failure cases, particularly around search timeouts. This resolves some situations where the LDAP realm would silently determine that a user had no groups, when it should have reported an error.
- Watcher
-
- Watch ack status is correctly reset, when the condition is not met again for the first time.
- X-Pack
-
- Basic license now correctly enables 18 levels of zoom in the Tilemap visuablization. Previously, Basic wasn’t being passed as a valid license type and it defaulted to 10 zoom levels.
5.3.3
editJune 1, 2017
Bug Fixes
edit- Monitoring
-
- Limited the indices stats being collected to just the ones that are needed to reduce the overhead of the lookup.
5.3.2
editApril 27, 2017
Bug Fixes
edit- Watcher
-
- Fixed resetting of acknowledgement status on unmet conditions.
5.3.1
editApril 20, 2017
Enhancements
edit- Watcher
-
-
Parsing of comma separated email addresses, so you can use the
{{#join}}
tag to join an array of email addresses.
-
Parsing of comma separated email addresses, so you can use the
Bug Fixes
edit- Security
-
- Fixed a problem in field level security that could result in users not seeing all the documents that they should.
- Watcher
-
- HTTP JSON parsing has been fixed when a response contains only a JSON array.
- When watcher is disabled, the start up check on automatic index creation is now disabled as well.
5.3.0
editMarch 28, 2017
Breaking Changes
edit- Security
-
-
Unauthorized bulk request is rejected on individual bulk item basis instead of being rejected as an entire operation.
In case of insufficient privileges, individual bulk item responses return security exception with
403
error code instead of top level bulk response returning security exception with403
error code. The top level bulk response always returns with200
status code. -
create
,index
anddelete
index privileges can execute bulk action. -
Usage of Netty 3 for transport (
transport.type=security3
) or HTTP (http.type=security3
) is deprecated and will be removed in X-Pack 6.0.0. Instead, rely on the default implementations which are based on Netty 4.
-
Unauthorized bulk request is rejected on individual bulk item basis instead of being rejected as an entire operation.
In case of insufficient privileges, individual bulk item responses return security exception with
Enhancements
edit- Watcher
-
-
A new dedicated action status called
ACKNOWLEDGED
has been introduced to easily find watches, that have been acknowledged by the user -
Index Actions now support the ability to index using a dynamic
_id
field in the payload or constantdoc_id
parameter
-
A new dedicated action status called
5.2.2
editFebruary 28, 2017
Bug Fixes
edit- Monitoring
-
- Auto-Refresh can be set to Off without adverse effects.
5.2.1
editFebruary 14, 2017
Enhancements
edit- Monitoring
-
-
You can now specify multiple Elasticsearch hosts as an array when
configuring the
xpack.monitoring.elasticsearch.url
. - The Clusters table now includes a column for Logstash.
-
You can now specify multiple Elasticsearch hosts as an array when
configuring the
- Security
-
- Added TRACE logging for LDAP traffic.
Bug Fixes
edit- Monitoring
-
- The list of Logstash nodes can now display more than 10 nodes.
- Fixed the cluster summary rollups for Kibana and Logstash that appear in the Cluster Overview and Cluster Summary.
- Security
-
-
You no longer get a
StatusLogger
error when you run thecertgen
tool. - If you upgrade after reverting to a Basic license, you no longer get an error indicating that the realm cache could not be cleared.
-
You no longer get a
- Watcher
-
- Ensure that no NPE is thrown on startup if a required index for Watcher has been manually closed.
5.2.0
editJanuary 31, 2017
Breaking Changes
edit- Security
-
- The validation of security related settings has been tightened. Elasticsearch will refuse to start if it detects incorrect configuration of security realms or SSL/TLS.
New Features
edit- Monitoring
-
- You can now monitor Logstash nodes.
- If you’re running Elasticsearch in containers, you can now monitor a container’s utilization metrics that are reported from each Elasticsearch instance.
Enhancements
edit- Monitoring
-
- You can once again monitor tribe nodes.
- Security
-
-
Renamed the
kibana
role tokibana_system
. A backwards compatibility layer is provided so that kibana access still works properly during rolling upgrades.
-
Renamed the
- Watcher
-
- Watches can now be deleted even if the license is expired.
-
A deprecation warning is logged if an executed watch contains unencoded URLs.
Starting in 6.0, all URLs in
http
andwebhook
actions must be properly encoded.
Bug Fixes
edit- Reporting
-
- Now correctly uses the dashboard state when rendering visualizations. Previously, if you saved a change to a dashboard and then generated a report, the report would not reflect the change.
- Security
-
-
LDAP and AD realms now obey the
ssl.verification_mode setting
. Thehostname_verification
setting is deprecated. - When using a Gold license, the role management UI now identifies any roles that are disabled because they rely on the document or field level security features available during the trial period and with a Platinum license. You cannot modify disabled roles, but you can view and delete them.
-
LDAP and AD realms now obey the
- Watcher
-
-
The
search
input now correctly handles theextract
parameter.
-
The
5.1.2
editBug Fixes
edit- Security
-
-
Fixed a problem where
realm_authentication_failed
audit events were being recorded asauthentication_failed
when auditing was logged to an index. - If a destructive operations check fails, stop execution of the index operation.
- Disable roles with document and field level security if the trial period is over and a Gold license is installed. (DLS/FLS are only available with a Platinum license.)
- Looking up the groups an LDAP user belongs to now returns the correct information. This was preventing users from being correctly mapped to roles in some cases.
-
Fixed a problem where
- Watcher
-
-
Index action: Ensure that a failed index request, which is part of a bulk
request via the
_doc
array is logged correctly as success or (partial) failure.
-
Index action: Ensure that a failed index request, which is part of a bulk
request via the
5.1.1
editDecember 8, 2016
Enhancements
edit- Watcher
-
- You can now specify a proxy in HipChat, PagerDuty, and Slack actions.
- Monitoring
-
- Combined advnanced node charts.
New Features
edit- Watcher
-
-
Added a new
jira
action that allows to create Jira issues using Watcher.
-
Added a new
Bug Fixes
edit- Monitoring
-
- Improved behavior of the monitoring charts' crosshair.
- Temporarily removed monitoring chart tooltip.
- Fixed monitoring chart legend labels.
- Properly handle 404 responses from Elasticsearch.
- Reporting
-
- Fixed an unhandled error when extracting the PhantomJS archive that was causing the Kibana server to stall on start up.
-
Updated the
phantomjs.exe
path for Windows.
- Security
-
- Fixed a problem where the search template endpoint threw an error when used against all indices or a wildcard expression.
- Kibana no longer crashses when a basic license is installed.
- Watcher
-
- An invalid HTTP response is now correctly marked as a failure.
- Ensure that watcher history does not contain secrets
5.1.0 (skipped)
editVersion 5.1.0 doesn’t exist because, for a short period of time, the Elastic Yum and Apt repositories included unreleased binaries labeled 5.1.0. To avoid confusion and upgrade issues for the people that have installed these without realizing, we decided to skip the 5.1.0 version and release 5.1.1 instead.
5.0.2
editNovember 29, 2016
Bug Fixes
edit- Monitoring
-
- Add support for custom headers in the monitoring connection and make phone home always return 200.
- Security
-
- Allow reads of native users and roles when the template version hasn’t been updated to match the current version. This prevents failures from occurring during rolling upgrades.
- Retain all user information for run as requests.
- Prevent unknown run as users from executing any APIs. Previously, if an authenticated user with run as permission attempted to run as an unknown user, the unknown user was assigned the default and anonymous roles if they were enabled.
-
If an exception is thrown when resolving the index in an index request, it
is now recorded as
accessDenied
in the audit-trail. Previously, no entry was recorded in the audit trail.
5.0.1
editNovember 15, 2016
Bug Fixes
edit- Graph
-
-
Fixed the license check so Graph doesn’t throw an
undefined
error when Security is disabled and you try to load a workspace URL.
-
Fixed the license check so Graph doesn’t throw an
- Monitoring
-
- Show Replica Count not Replication Factor in Overview.
- A non-aliased Monitoring index can now be always be created for the current day when upgrading from Marvel.
- Duplicate shards no longer appear in the shard allocation table.
- The Kibana Cluster Summary now always shows the last-known status.
- Kibana now makes sure Monitoring is enabled before attempting to send stats.
- Security
-
- Security can no longer pollute the thread context with incorrect users, which could cause failures during the discovery process.
-
Security now honors the
action.destructive_requires_name
setting and prevents users from deleting indices with wildcards if it is set totrue
. - Made changes to preserve the context when performing internal actions. This ensures subsequent actions are performed as the correct user.
-
Files generated by the
certgen
tool now have permissions set to 600 so they aren’t world-readable. - The Security UI no longer hangs when you configure field-level security when adding a role.
- When running with a Basic License, the login dialog is no longer displayed and no Security elements are visible in Kibana.
- The last sub URL of each Kibana app is no longer cached between sessions. This means that when a different user logs in, they are longer redirected to the URLs the previous user viewed last.
- Watcher
-
- Chain input: An exception is now thrown if the inputs in the chain are specified with a data structure that does not preserve the input order. The inputs in a chain must be specified as array elements to guarantee the order in which the inputs are processed. (JSON does not guarantee the order of arbitrary objects.)
- Watch history template: Removed the unused Watcher plugin version.
- Email output: Fixed an error that prevented emails from being sennt when localhost could not be resolved.
5.0.0
editOctober 26, 2016
Breaking Changes
edit- X-Pack
-
-
All settings have been updated to use the
xpack
prefix. For more information, see Migrating to X-Pack.
-
All settings have been updated to use the
- Licensing
-
-
Licensing endpoint has been renamed from
/_license
to/_xpack/license
.
-
Licensing endpoint has been renamed from
- Monitoring
-
-
http
exporters no longer honor thekeep_alive
setting as this is handled by the low-level REST Client. -
All
monitoring.agent.*
settings have been changed to more closely match other monitoring collection settings:xpack.monitoring.collection.*
andxpack.monitoring.exporters.*
. - The Index page’s Lucene Memory chart was replaced with an Index Memory chart, which includes a superset of the information. Fielddata, which has become a significantly less common issue, has been rolled into the Index Memory chart.
- To use an external monitoring cluster to monitor an Elasticsearch 5.0 cluster, you must run Elasticsearch 5.0 on the monitoring cluster. For more information about external monitoring clusters, see Setting up a Separate Monitoring Cluster.
-
All settings have been updated to use the
xpack.monitoring
prefix. For more information, see Migrating to X-Pack.
-
- Reporting
-
-
Reporting encryption keys configured in
kibana.yml
must now be at least 32 characters.
-
Reporting encryption keys configured in
- Security
-
-
Security encryption keys configured in
kibana.yml
must now be at least 32 characters. - The SSL configuration settings have been changed to use an easier to use format that also supports PEM files.
-
Removed the
files.users
andfiles.users_roles
settings from thefile
realm. -
Removed the setting that allowed for a custom
roles.yml
file location to be specified. Theroles.yml
file must always be in theCONF_DIR/x-pack
directory. -
Removed the setting that allowed for a custom system key location to be
defined. The
system_key
file must always be in theCONF_DIR/x-pack
directory. -
The
logfile
output for auditing no longer uses the log level to determine which events to log. The events are now controlled in the same way as theindex
output. - Changed the syntax for field-level-security. Roles stored in the old format in native or file based realm will continue to work but new roles must use the new format.
-
The
esusers
realm has been renamed tofile
and theesusers
command line tool has been renamed tousers
. Note that the User and Role APIs are the preferred way to manage internal users. -
Elasticsearch enables HTTP compression by default now. To mitigate potential
security risks like the BREACH attack, X-Pack security disables compression if HTTPS
is enabled. If Elasticsearch should compress HTTPS traffic, please explicitly
set
http.compression
totrue
in ‘elasticsearch.yml’. -
You must specify all required values to override the global SSL configuration
in a profile. If any values are omitted, the entire configuration falls back to
the global settings,
xpack.security.ssl.*
. -
The
skipSslCheck
anduseUnsafeSessions
for Kibana have been replaced byxpack.security.secureCookies
inkibana.yml
. SSL is now disabled by default. You can start Kibana without making any changes tokibana.yml
after you install X-Pack. Do not deploy to production without enabling SSL/TLS encryption! - A default role is now applied to all users, including anonymous users. The default role enables users to access the authenticate endpoint, change their own passwords, and get information about themselves.
-
All settings have been updated to use the
xpack.security
prefix. For more information, see Migrating to X-Pack.
-
Security encryption keys configured in
- Watcher
-
-
The
force
parameter of the Delete Watch Action has been removed. -
The use of the
_timestamp
field for the execution time has been removed. The user now needs to set this explicitly in theindex
action. -
The
_xpack/watcher/_start
,_xpack/watcher/_restart
, and_xpack/watcher/_stop
REST endpoints requirePOST
actions instead ofPUT
actions. The deprecated_watcher/_start
,_watcher/_restart
, and_watcher/_stop
endpoints still allowPUT
. -
Watch history now uses a versioned template. The index names also changed
and contain this version. So instead of
.watch_history_2016.02.03
the new index name is.watcher-history-1-2016.02.03
, where1
is the current version. If you are using X-Pack security, this might require you to change roles/permissions because of the different index names! The old index template namedwatch_history
can safely be deleted. However, it does not interfere with the new index template. -
The setting that enables scripting only for Watcher has been renamed from
script.engine.groovy.inline.elasticsearch-watcher_watch
toscript.engine.groovy.inline.xpack_watch
. -
Elasticsearch has several breaking changes in the query DSL, including that
search_type=count
is no longer supported. Check to see if your watches use this search type and upgrade them to usesize: 0
in the request body as needed. For more information about breaking changes including search changes, see breaking changes section in Elasticsearch. -
All account SMTP timeouts (
smtp.timeout
,smtp.connection_timeout
andsmtp.write_timeout
) now require a time value instead of a number in milliseconds. -
The notification settings for PagerDuty, Slack, HipChat, and email have been
moved from
watcher.actions
toxpack.notification
. You need to update your Elasticsearch configuration accordingly. -
All watcher endpoints have been renamed from
/_watcher/XYZ
to/_xpack/watcher/XYZ
. You might need to fix this in external scripts as well as in your watches. -
The notification settings have been stripped of their
service
part. Sowatcher.actions.slack.service.default_account
becomesxpack.notification.slack.default_account
-
The setting
watcher.shield.encrypt_sensitive_data
has been renamed toxpack.watcher.encrypt_sensitive_data
-
The
New Features
edit- Monitoring
-
-
Added new node resolver,
uuid
, to the Monitoring UI configuration and made it the default. Starting with Elasticsearch 5.0, instances of Elasticsearch create a persistent UUID that remains the same across restarts unless the data directory is deleted. If the data directory is deleted, the instance a new UUID on start up. - Latencies calculated against totals use derivatives to get the rate of change. If any derivative is negative, then that time bucket is ignored and left blank on the latency chart. Values that are negative indicate that the underlying total shrank, which means that the data is skewed and showing the result is misleading (for example, due to nodes restarting).
- Added Segment Count memory chart to the Index page.
-
Added new node resolver,
- Security
-
- Support for forest wide authentication in the Active Directory Realm.
-
The default LDAP group search filter now includes
posixGroup
groups. - LDAP user search can now use un-pooled connections.
- Watcher
-
-
Added support for accessing the HTTP status code of a response in the HTTP
input through
ctx.payload._status_code
. -
The new REST endpoint for acknowledging certain actions of a watch is
_xpack/watcher/watch/{watch_id}/_ack/{action_id}
. The old notation waswatcher/watch/{watch_id}/{action_id}/_ack
, which will be removed in future releases.
-
Added support for accessing the HTTP status code of a response in the HTTP
input through
Enhancements
edit- Graph
-
- Added ability to save Graph workspaces
- Added ability to drill-down on Graph selections using other Kibana visualizations
-
In the Graph UI, you can now use an index pattern such as
logstash-*
to select multiple time-based indices instead of a single index.
- Monitoring
-
- Added dots for all points on charts.
- Added the ability to highlight points by hovering close to them. The highlighted point, and those from other series at the X-position, are what are displayed in the legend.
-
Added a monitoring ingest pipeline so that future releases will be compatible
even if backward incompatible changes are made. This is enabled by default, but
can be disabled by setting
use_ingest
tofalse
at the exporter level (for example,xpack.monitoring.exporters.my_exporter.use_ingest: false
). -
Added the ability for HTTP exporters to send arbitrary HTTP headers along
with requests. This allows the HTTP exporter to be used with proxies to route
monitoring data more dynamically, if necessary. This can be used by supplying
name-value pairs at the exporter level (for example,
xpack.monitoring.exporters.my_exporter.headers.X-My-Header: abc123
). - Rewrote the HTTP exporter to use the low-level REST Client and better pool connections. This reduces the resources used for both networking and parsing.
- Added Kibana instance monitoring as part of the same Elastic Cluster.
- Added experimental charts to be used while monitoring Kibana instances.
- Added breadcrumbs to allow simpler navigation between monitoring pages.
- Simplified the Indices tab to remove charts that already appeared on the Overview page so that indices are more accessible.
- Simplified overall status handling so that it is clearer what the status of the current item is (e.g., index view gives index status).
- Added index memory graph to the Node page so that the cost of open indices can be determined more accurately.
- Added the total indexing rate alongside the primary indexing rate. Total includes both primaries and replicas.
- Added color to all charts.
- Added units to all chart titles.
- Added the internals to support monitoring Kibana instances.
- Improved the display of values in the legend.
- Shortened the welcome message.
- Security
-
- Native users and roles can now be used on tribe nodes.
- Added the ability to disable native and reserved users.
- Added ability to define exclusions for fields in field level security.
- Added built-in roles for reporting users, monitoring users, remote monitoring agents, and users of the Kibana ingest feature.
-
Auditing supports an
authentication_success
event that is output after authentication. This event can output the body of the request, so in combination with theauthentication_failed
event all request bodies can be audited. -
Added a X-Pack specific transport client,
PreBuiltXPackTransportClient
, that provides an easy way to use the transport client with X-Pack and other modules of Elasticsearch such as reindex. - Auditing now de-duplicates the names of indices when logging.
- Document and Field Level Security can be used with realtime requests.
-
The
certgen
tool no longer generates file names that would result in hidden files and now offers an option to specify the validity time of the generated certificates. -
Added an
ingest_admin
role that grants the permissions requried to use the ingest feature in Kibana. -
New
elastic
andkibana
built-in users. -
New
superuser
andtransport_client
built-in roles. - Added a Reset Password API to enable administrators and users to reset and change passwords.
-
Added a built-in
kibana_user
role that grants the minimum set of privileges needed to use Kibana. -
Default anonymous username changed to
_anonymous
(used to be_es_anonymous_user
)
- Watcher
-
-
Allow use of
inline
attachments in emails, so that desktop clients can display attachments like images embedded in emails. -
The HTTP headers of a response are now part of the payload and can be
accessed via
ctx.payload._headers
- Individual actions now support conditions. This is useful when a single watch contains multiple actions—specific actions can fire based on the current context.
- Watches can now be modified or deleted while they are running, which is especially useful for long running watches
-
Allow use of
Bug Fixes
edit- Security
-
-
Updated document level security to support preventing requests that use
scripts or
now()
from being cached.
-
Updated document level security to support preventing requests that use
scripts or
- Watcher
-
- The watch version is now ignored when deleting a watch.