This documentation contains work-in-progress information for future Elastic Stack and Cloud releases. Use the version selector to view supported release docs. It also contains some Elastic Cloud serverless information. Check out our serverless docs for more details.
« 8.12.1 release notes 8.11.4 release notes »
Elastic Docs › ›Release notes

8.12.0 release notes

edit

8.12.0 release notes

edit

Upgrading to Enterprise Search 8.12.0? See Upgrading and migrating.

Breaking changes

edit

  • In 8.12 access control indices (used for document level security) are being migrated to a new naming pattern. The new naming pattern will now include the search- prefix for the original index name. Before 8.12, an index named search-foo would generate an ACL index name of .search-acl-filter-foo. From 8.12 onwards, an index named search-foo will generate an ACL index name of .search-acl-filter-search-foo.

    Any client code that is querying from the access control indices will need to be updated to reference the new index name(s).

    If you are using a self-managed connector client without an Enterprise Search node, this migration will not run automatically. As a result document level security (DLS) will not work after upgrading. There are two workaround options:

    1. Start up an Enterprise Search node for the purposes of carrying out this migration.

    2. If you don’t want to spin up an Enterprise Search node, use the following manual steps:

      • Manually delete all access control indices. Using the Dev Tools Console in Kibana, run the following command.

        DELETE .search-acl-filter-*
      • Run access control syncs for each connector affected.
      • Re-issue API keys for these connectors and add them to the config.yml.

New features

edit

  • The following connectors are now available as native connectors (managed service) on Elastic Cloud:

    • Amazon S3
    • Google Cloud Storage
    • Oracle
    • Salesforce

  • The following connectors are now generally available:

    • Amazon S3
    • Azure Blob Storage
    • Google Cloud Storage
    • Microsoft SQL
    • MongoDB
    • MySQL

    • PostgreSQL

      Check out our new internal knowledge search example app in the elasticsearch-labs repo. This app ingests data using our MongoDB and MySQL connectors.

  • Document Level Security (DLS) is now available for the following connectors:

    • Dropbox
    • Github

  • We added advanced sync rule support for the following connectors:

    • Amazon S3
    • Salesforce
  • The self-managed Network Drive connector client now supports Linux network drives, in addition to Windows network drives.
  • Use our new Connector APIs (technical preview) to manage your connectors and sync jobs programmatically, without needing to rely solely on the Kibana UI.

  • We have made it easier to deploy self-managed Elastic connector clients with official Docker images.

    • We’ve added scripts and example configurations to quickly spin up Elasticsearch, Kibana, and the connectors services, with Docker Compose. Refer to the README for more information.
  • Docker images are also now available for the local data extraction service.

  • We added a new Connectors CLI (technical preview), designed to streamline and automate the management of Elastic Connectors. The CLI can connect to any Elasticsearch instance, so you can use it for scripting and automation.

    Known limitations:

    • Does not support API keys for authentication
    • Does not support sync job scheduling configuration

  • Self-managed connector clients can now ingest to any existing Elasticsearch index, removing the need to work with search- prefixed indices created by Enterprise Search.

    • NOTE: Indices created in the Kibana UI still require the search- prefix.
    • You can now configure your connector client using the Connectors CLI and use the --from_index flag to ingest into an existing Elasticsearch index. Note that the target index cannot already be used by another connector.

Bug fixes

edit
  • Fixed a bug where Elasticsearch health issues might be ignored by the connectors service. This will prevent the unexpected deletion of the connector and backing index.
  • The connector service now avoids deleting indices for orphaned jobs, because in some edge cases this could lead to data loss.
  • Fixed bugs where some native connectors did not have the same features available as their connector client versions.
  • Fixed a bug where some sync jobs might fail with an empty error message.
  • Fixed a bug where the Sharepoint Online connector might sometimes fail syncs when DLS was enabled due to a batching error.

  • Addressed a bug where full syncs would not fully-resync data as expected, which caused friction when upgrading, making customized code changes, or modifying ingest pipelines.

    The fix may result in increased full sync times, as more documents may now be downloaded and indexed. If the performance impacts outweigh the value of fixing the bug, you can revert this change on your fork. See the pull request that introduced this change.

  • The Sharepoint Online connector now ensures that all Site Collection Administrators are present in Site, Site Page, Site List, Site List Item, and Drive Item access control lists for document level security.
  • Fixed a bug where the Sharepoint Online connector did not add access control list entries for Azure Security Groups.
  • Updated the rubyzip dependency, which was necessary for some security scans that flagged encrypted zipfile test data from the old dependency.

Enhancements

edit
  • The Salesforce connector no longer requires Global Admin permissions.

  • We’ve changed the the Salesforce connector’s Elasticsearch mapping to ensure a 1:1 representation of Salesforce objects in Elasticsearch. Previously, the connector used a custom mapping. The new document structure looks like this:

    {
  "_id:: <Id>,
  "_timestamp": <LastModifiedDate>,
  "type": <type>,
  "owner": <ownerName>,
  "owner_url": <OwnerURL>,
  "title": <objectName>,
  "description": <description>
  "CreatedDate": <CreatedDate>,
  "Name": <objectName>,
  "Description": <description>,
  ...
}
    • You can now ingest all custom objects and their associated fields
    • You can also use advanced sync rules to ingest specific objects and fields

Known issues

edit
  • The MongoDB connector will fail to sync content to Mongo Atlas URLs (mongo+srv) unless SSL/TLS is enabled.

  • A bug introduced in 8.12.0 causes the Connectors docker image to error out if run using MongoDB as its source. The command line will output the error cannot import name 'coroutine' from 'asyncio'.

    • This issue is fixed in versions 8.12.2 and 8.13.0.
    • This bug doesn’t affect native connectors.
« 8.12.1 release notes 8.11.4 release notes »