Elastic Cloud Enterprise 3.5.0

The following changes are included in this release.

Show name for External trusts. Added read-only name field to External trusts in deployments API that show the name of the referenced TrustRelationship.

Nodestats output to Elasticsearch and Zookeeper storage analysis Kibana objects. New command in zkstat: zkstat node-stats-elasticsearch which is equivalent to node-stats with the difference that its output is replaced by sending the collected data to an Elasticsearch instance by indexing each entry accordingly, and optionally creating the required Kibana objects to easily analyze this data.

Force social login if the organization mandates it. Adds a feature where organizations can mandate all the members to use social login.

Validate cert UID consistency. Validates that within the same organization a fingerprint is only used with one trust relationship UID and that a trust relationship always has all known valid certificates for the environment.

Disable Proxy RouteStatus loading for Kubernetes. Disable loading of status files in proxy for Kubernetes.

Update other deployments when updating certificates in DirectTrustRelationships. Simplified certificate rotation and revocation for remote cluster connections to clusters running outside of the current ECE environment. When multiple clusters in ECE trust the same external environment defined with a DirectTrustRelationship, then the certificates update for this environment for one deployment will automatically be applied to all other deployments trusting this environment.

Organizations mandate MFA. Adds a feature where organizations can mandate the usage of MFA for new members.

[Deployment Monitoring] Improved health experience. This PR adds uses the newly created es health api in installations >=8.4 in order to give the user a better way of troubleshooting their issues. If using deployments < 8.4, users are prompted to upgrade their deployments to get better troubleshooting suggestions.

Allow ECE upgrade if the version is already up to date. Allows ECE upgrade to proceed even if ECE is already at the desired version. This allows the upgrader to perform upgrades of the system clusters if a newer stack version is available.

Use the Elasticsearch scoring when querying deployments. Searching deployments will now score and sort results based on the search query.

Add a name field to accounts. There is now a name field for account trust relationships.

Remove storageMultiplier from the list of immutable fields. Allow changes to the storage_multiplier field on existing instance configurations.

Add info to the Edit page when tier addition fails but instances still exist. If the addition of a new tier fails, but some instances are added, the user is now notified through the Edit page.

Add usage and creation timestamps to Usage Records. The usage_timestamp and creation_timestamp fields are now added to Usage Records generated by metering CTS tasks. The @timestamp field is now deprecated.

Populate deployment level autoscaling in deployment templates on upgrade. Copies the autoscaling_enabled field from the Elasticsearch plan into the deployment settings for all defined deployment templates.

Add the upgrade button to the portal for each deployment. When a new version is available, you are notified and you can upgrade your deployment from the portal.

Add trusted relationship dropdown. Adding a dropdown of existing Trust Relationships so that users can select those as templates when creating new ones.

Ingest allocator GC logs. This change will ingest allocator GC logs into Elasticsearch logging clusters via filebeat for longer retention and better search capabilities. Disk space used will increase in the Elasticsearch cluster in which allocator logs are ingested. This is about 70 megabytes per allocator per day.

Reconfigure monitors. We have improved the monitoring runner to reconfigure monitors using configurations optionally stored in the Usage Cluster’s billing-configuration index.

Conditionally set datastore clients. We have improved the monitoring runner to conditionally set PG and/or Usage Cluster clients on monitors, depending on whether they need them.

Pricing on template migration. Price details are now visible when you migrate from one template to another.

Enrich platform API health checks. Enriched the platform API to expose more information about the checks used to determine the health status of an allocator/runner.

Show a full architecture vizualisation in the restore modal. Show a full architecture vizualisation, including stateless components when restoring a terminated deployment. Previously, this modal would simply sum the single zone Elasticsearch capacity which was confusing when multiple data tiers were in use, or a deployment spanned multiple availability zones.

Flush response without buffering if X-Accel-Buffering: no header exists in backend response. Proxy will flush response without buffering if X-Accel-Buffering: no header exists in backend response.

Handle the security realm warnings on upgrade similarly to other warnings. Fixes a bug preventing users from updating their security realm settings during an upgrade.

ECE upgrader refactoring. Fixes a bug where ECE upgrade from versions older than 2.13 to versions greater or equal to 3.0 would be possible even if an ongoing upgrade was detected.

Disabled Enterprise Search not rendered in the UI. Fixes a bug in the UI that was preventing Enterprise Search to be re-enabled from the deployment Edit page.

Compare plans against the previously successful plan, not just the previous plan. The plan activity now correctly compares plans to the previous successful plan, rather than just the previous plan.

Set the idle timeout on Elasticsearch connections. Prevents connection leaks from the Container Task Service (CTS).

Correctly remove incompatible extensions on upgrade. Upgrades for deployments that use custom extensions should no longer fail if an extension is incompatible with the target version.

Added reset to waitLoop/recursion. With this fix, the step Waiting on Elastic Search to upgrade will wait until Elasticsearch completes (or fails) the upgrade. The time elapsed during this step won’t be counted towards the overall plan timeout.

Append 6 first digits of deployment ID to cluster.metadata.display_name. Update the default ES cluster.metadata.display_name to contain <deployment_name> (<first_6_chars_of_deployment_ID>), instead of simply containing the deployment name.