« Running from the source code Docker Compose quickstart »
Elastic Docs › ›Elastic connectors ›Connector clients (self-managed)

Running from a Docker container

edit

Running from a Docker container

edit

Use our Docker Compose quickstart to quickly get started with a full Elastic Stack deployment using Connectors.

Instead of running the Connectors Service from source, you can use the official Docker image to run the service in a container.

As a prerequisite, you need to have an Elasticsearch and Kibana instance running. From inside your Kibana UI, You will need to follow the initial setup in the same manner as if you are running the service from source.

When you are ready to run Connectors:

Step 1: Download sample configuration file

Download the sample configuration file. You can either download it manually or run the following command:

curl https://raw.githubusercontent.com/elastic/connectors/main/config.yml.example --output </absolute/path/to>/connectors-config/config.yml

Don’t forget to change the --output argument value to the path where you want to save the config.yml file on your local system. But keep note of where you wrote this file, as it is required in the docker run step below.

Step 2: Update the configuration file for your self-managed connector

Update the configuration file with the following settings to match your environment:

  • elasticsearch.host
  • elasticsearch.api_key
  • connectors

If you’re running the connector service against a Dockerized version of Elasticsearch and Kibana, your config file will look like this:

# When connecting to your cloud deployment you should edit the host value
elasticsearch.host: http://host.docker.internal:9200
elasticsearch.api_key: <ELASTICSEARCH_API_KEY>

connectors:
  -
    connector_id: <CONNECTOR_ID_FROM_KIBANA>
    service_type: {service-name} # sharepoint_online (example)
    api_key: <CONNECTOR_API_KEY_FROM_KIBANA> # Optional. If not provided, the connector will use the elasticsearch.api_key instead

Step 3: Run the Docker image

Run the Docker image with the Connector Service using the following command:

docker run \
-v "</absolute/path/to>/connectors-config:/config" \ # NOTE: you must change this path to match where the config.yml is located
--rm \
--tty -i \
--network host \
docker.elastic.co/enterprise-search/elastic-connectors:8.15.3.0 \
/app/bin/elastic-ingest \
-c /config/config.yml

For unreleased versions, append the -SNAPSHOT suffix to the version number. For example, docker.elastic.co/enterprise-search/elastic-connectors:8.14.0.0-SNAPSHOT.

Find all available Docker images in the official registry.

Each connector client reference in these docs contains specific instructions for deploying your connector using Docker.

When running the connector service Docker container against a local Elasticsearch cluster that has security and SSL enabled (like the docker compose example in the Elasticsearch docs), it’s important to handle the self-signed certificate correctly:

  1. Ensure the Docker container running the connector service has the volume attached that contains the generated certificate. When using Docker Compose, Docker automatically adds a project-specific prefix to volume names based on the directory where your docker-compose.yml is located.

    When starting the connector service with docker run, use -v <your_project>_certs:/usr/share/connectors/config/certs to reference the certificate volume. Replace <your_project> with the actual prefix, such as elastic_docker_certs if your docker-compose.yml that starts Elasticsearch stack is in a directory named elastic_docker.

  2. Make sure the connector service’s config.yml correctly references the certificate: elasticsearch.ca_certs: /usr/share/connectors/config/certs/ca/ca.crt.

Enter data source details in Kibana

edit

Once the connector service is running, it’s time to head back to the Kibana UI to finalize the connector configuration. In this step, you need to add the specific connection details about your data source instance, like URL, authorization credentials, etc. As mentioned above, these details will vary based on the third-party data source you’re connecting to.

For example, the PostgreSQL connector requires the following details:

  • Host
  • Port
  • Username
  • Password
  • Database
  • Comma-separated list of tables

You’re now ready to run a sync. Select the Full sync button in the Kibana UI to start ingesting documents into Elasticsearch.

« Running from the source code Docker Compose quickstart »