Connectors UI in Kibana

This document describes operations available to managed connectors and self-managed connectors, using the UI.

In the Kibana UI, go to Search > Content > Connectors to view a summary of all your connectors and sync jobs, and to create new connectors.

You connector writes data to an Elasticsearch index.

To create managed connectors or self-managed self-managed connectors, use the buttons under Search > Content > Connectors. Once you’ve chosen the data source type you’d like to sync, you’ll be prompted to create an Elasticsearch index.

View and manage all Elasticsearch indices managed by connectors.

In the Kibana UI, navigate to Search > Content > Connectors to view a list of connector indices and their attributes, including connector type health and ingestion status.

Within this interface, you can choose to view the details for each existing index or delete an index. Or, you can create a new connector index.

These operations require access to Kibana and additional index privileges.

Elasticsearch stores your data as documents in an index. Each index is made up of a set of fields and each field has a type (such as keyword, boolean, or date).

Mapping is the process of defining how a document, and the fields it contains, are stored and indexed. Connectors use dynamic mapping to automatically create mappings based on the data fetched from the source.

Index settings are configurations that can be adjusted on a per-index basis. They control things like the index’s performance, the resources it uses, and how it should handle operations.

When you create an index with a connector, the index is created with default search-optimized field template mappings and index settings. Mappings for specific fields are then dynamically created based on the data fetched from the source.

You can inspect your index mappings in the following ways:

You can manually edit the mappings and settings via the Elasticsearch APIs:

It’s important to note that these updates are more complex when the index already contains data.

Refer to the following sections for more information.

Updating mappings and settings is simpler when your index has no data. If you create and attach a new index while setting up a connector, you can customize the mappings and settings before syncing data, using the APIs mentioned earlier.

Once data has been added to Elasticsearch using dynamic mappings, you can’t directly update existing field mappings. If you’ve already synced data into an index and want to change the mappings, you’ll need to reindex your data.

The workflow for these updates is as follows:

After creating an index to be managed by a connector, you can configure automatic, recurring syncs.

In the Kibana UI, navigate to Search > Content > Connectors.

Choose the index to configure, and then choose the Scheduling tab.

Within this interface, you can enable or disable scheduled:

When enabled, you can additionally manage the sync schedule.

This operation requires access to Kibana and the write indices privilege for the .elastic-connectors index.

Alternatively, you can sync once.

After you enable recurring syncs or sync once, the first sync will begin. (There may be a short delay before the connector service begins the first sync.) You may want to view the index details to see the status or errors, or view the synced documents.

After creating the index to be managed by a connector, you can request a single sync at any time.

In the Kibana UI, navigate to Search > Content > Elasticsearch indices.

Then choose the index to sync.

Regardless of which tab is active, the Sync button is always visible in the top right. Choose this button to reveal sync options:

Choose one of the options to request a sync. (There may be a short delay before the connector service begins the sync.)

This operation requires access to Kibana and the write indices privilege for the .elastic-connectors index.

After a sync has started, you can cancel the sync before it completes.

In the Kibana UI, navigate to Search > Content > Elasticsearch indices.

Then choose the index with the running sync.

Regardless of which tab is active, the Sync button is always visible in the top right. Choose this button to reveal sync options, and choose Cancel Syncs to cancel active syncs. This will cancel the running job, and marks all pending and suspended jobs as canceled as well. (There may be a short delay before the connector service cancels the syncs.)

This operation requires access to Kibana and the write indices privilege for the .elastic-connectors and .elastic-connectors-sync-jobs index.

View the index details to see a variety of information that communicate the status of the index and connector.

In the Kibana UI, navigate to Search > Content > Elasticsearch indices.

Then choose the index to view.

The Overview tab presents a variety of information, including:

Possible values of ingestion status:

This tab also displays the recent sync history, including sync status (see below for possible values).

Possible values of sync status:

For each sync, choose the view button to display the job details, including:

This operation requires access to Kibana and the read indices privilege for the .elastic-connectors index.

View the documents the connector has synced from the data. Additionally view the index mappings to determine the current document schema.

In the Kibana UI, navigate to Search > Content > Elasticsearch indices.

Then choose the index to view.

Choose the Documents tab to view the synced documents. Choose the Index Mappings tab to view the index mappings that were created by the connector.

When setting up a new connector, ensure you are getting the documents and fields you were expecting from the data source. If not, see Troubleshooting for help.

These operations require access to Kibana and the read and manage indices privileges for the index containing the documents.

See Security for security details.

Use sync rules to limit which documents are fetched from the data source, or limit which fetched documents are stored in Elastic.

In the Kibana UI, navigate to Search > Content > Elasticsearch indices.

Then choose the index to manage and choose the Sync rules tab.

Use ingest pipelines to transform fetched data before it is stored in Elastic.

In the Kibana UI, navigate to Search > Content > Elasticsearch indices.

Then choose the index to manage and choose the Pipelines tab.