« Installation layout Using a proxy server with Elastic Agent and Fleet »
Elastic Docs ›Fleet and Elastic Agent Guide [8.16] ›Install Elastic Agents

Air-gapped environments

edit

Air-gapped environments

edit

When running Elastic Agents in a restricted or closed network, you need to take extra steps to make sure:

  • Kibana is able to reach the Elastic Package Registry to download package metadata and content.
  • Elastic Agents are able to download binaries during upgrades from the Elastic Artifact Registry.

The Elastic Package Registry must therefore be accessible from Kibana via an HTTP Proxy and/or self-hosted.

The Elastic Artifact Registry must therefore be accessible from Kibana via an HTTP Proxy and/or self-hosted.

See the Elastic Security Solution documentation for air-gapped offline endpoints.

When upgrading all the components in an air-gapped environment, it is recommended that you upgrade in the following order:

  1. Upgrade the Elastic Package Registry.
  2. Upgrade the Elastic Stack including Kibana.
  3. Upgrade the Elastic Artifact Registry and ensure the latest Elastic Agent binaries are available.
  4. Upgrade the on-premise Fleet Server.
  5. In Fleet, issue an upgrade for all the Elastic Agents.

Enable air-gapped mode for Fleet

edit

Set the following property in Kibana to enable air-gapped mode in Fleet. This allows Fleet to intelligently skip certain requests or operations that shouldn’t be attempted in air-gapped environments.

xpack.fleet.isAirGapped: true

Use a proxy server to access the Elastic Package Registry

edit

By default Kibana downloads package metadata and content from the public Elastic Package Registry at epr.elastic.co.

If you can route traffic to the public endpoint of the Elastic Package Registry through a network gateway, set the following property in Kibana to use a proxy server:

xpack.fleet.registryProxyUrl: your-nat-gateway.corp.net

For more information, refer to Using a proxy server with Elastic Agent and Fleet.

Host your own Elastic Package Registry

edit

The Elastic Package Registry packages include signatures used in package verification. By default, Fleet uses the Elastic public GPG key to verify package signatures. If you ever need to change this GPG key, use the xpack.fleet.packageVerification.gpgKeyPath setting in kibana.yml. For more information, refer to Fleet settings.

If routing traffic through a proxy server is not an option, you can host your own Elastic Package Registry.

The Elastic Package Registry can be deployed and hosted onsite using one of the available Docker images. These docker images include the Elastic Package Registry and a selection of packages.

There are different distributions available:

  • 8.16.0 (recommended): docker.elastic.co/package-registry/distribution:8.16.0 - Selection of packages from the production repository released with Elastic Stack 8.16.0.
  • lite-8.16.0: docker.elastic.co/package-registry/distribution:lite-8.16.0 - Subset of the most commonly used packages from the production repository released with Elastic Stack 8.16.0. This image is a good candidate to start using Fleet in air-gapped environments.
  • production: docker.elastic.co/package-registry/distribution:production - Packages available in the production registry (https://epr.elastic.co). Please note that this image is updated every time a new version of a package gets published.
  • lite: docker.elastic.co/package-registry/distribution:lite - Subset of the most commonly used packages available in the production registry (https://epr.elastic.co). Please note that this image is updated every time a new version of a package gets published.

To update the distribution image, re-pull the image and then restart the docker container.

Every distribution contains packages that can be used by different versions of the Elastic Stack. The Elastic Package Registry API exposes a Kibana version constraint that allows for filtering packages that are compatible with a particular version.

These steps use the standard Docker CLI, but you can create a Kubernetes manifest based on this information. These images can also be used with other container runtimes compatible with Docker images.

  1. Pull the Docker image from the public Docker registry:

    docker pull docker.elastic.co/package-registry/distribution:8.16.0

  2. Save the Docker image locally:

    docker save -o package-registry-8.16.0.tar docker.elastic.co/package-registry/distribution:8.16.0

    Check the image size to ensure that you have enough disk space.

  3. Transfer the image to the air-gapped environment and load it:

    docker load -i package-registry-8.16.0.tar

  4. Run the Elastic Package Registry:

    docker run -it -p 8080:8080 docker.elastic.co/package-registry/distribution:8.16.0

  5. (Optional) You can monitor the health of your Elastic Package Registry with requests to the root path:

    docker run -it -p 8080:8080 \
    --health-cmd "curl -f -L http://127.0.0.1:8080/health" \
    docker.elastic.co/package-registry/distribution:8.16.0
Connect Kibana to your hosted Elastic Package Registry
edit

Use the xpack.fleet.registryUrl property in the Kibana config to set the URL of your hosted package registry. For example:

xpack.fleet.registryUrl: "http://package-registry.corp.net:8080"
TLS configuration of the Elastic Package Registry
edit

You can configure the Elastic Package Registry to listen on a secure HTTPS port using TLS.

For example, given a key and a certificate pair available in /etc/ssl, you can start the Elastic Package Registry listening on the 443 port using the following command:

docker run -it -p 443:443 \
  -v /etc/ssl/package-registry.key:/etc/ssl/package-registry.key:ro \
  -v /etc/ssl/package-registry.crt:/etc/ssl/package-registry.crt:ro \
  -e EPR_ADDRESS=0.0.0.0:443 \
  -e EPR_TLS_KEY=/etc/ssl/package-registry.key \
  -e EPR_TLS_CERT=/etc/ssl/package-registry.crt \
  docker.elastic.co/package-registry/distribution:8.16.0

The Elastic Package Registry supports TLS versions from 1.0 to 1.3. The minimum version accepted can be configured with EPR_TLS_MIN_VERSION, it defaults to 1.0. If you want to restrict the supported versions from 1.2 to 1.3, you can use EPR_TLS_MIN_VERSION=1.2.

Using custom CA certificates
edit

If you are using self-signed certificates or certificates issued by a custom Certificate Authority (CA), you need to set the file path to your CA in the NODE_EXTRA_CA_CERTS environment variable in the Kibana startup files.

NODE_EXTRA_CA_CERTS="/etc/kibana/certs/ca-cert.pem"

Host your own artifact registry for binary downloads

edit

Elastic Agents must be able to access the Elastic Artifact Registry to download binaries during upgrades. By default Elastic Agents download artifacts from https://artifacts.elastic.co/downloads/.

To make binaries available in an air-gapped environment, you can host your own custom artifact registry, and then configure Elastic Agents to download binaries from it.

  1. Create a custom artifact registry in a location accessible to your Elastic Agents:

    1. Download the latest release artifacts from the public Elastic Artifact Registry at https://artifacts.elastic.co/downloads/. For example, the following cURL commands download all the artifacts that may be needed to upgrade Elastic Agents running on Linux x86_64. You may replace x86_64 with arm64 for the ARM64 version. The exact list depends on which integrations you’re using. Make sure to also download the corresponding sha512, and PGP Signature (.asc) files for each binary. These are used for file integrity validations during installations and upgrades.

      curl -O https://artifacts.elastic.co/downloads/apm-server/apm-server-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/apm-server/apm-server-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/apm-server/apm-server-8.16.0-linux-x86_64.tar.gz.asc
curl -O https://artifacts.elastic.co/downloads/beats/auditbeat/auditbeat-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/beats/auditbeat/auditbeat-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/beats/auditbeat/auditbeat-8.16.0-linux-x86_64.tar.gz.asc
curl -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.16.0-linux-x86_64.tar.gz.asc
curl -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.16.0-linux-x86_64.tar.gz.asc
curl -O https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/beats/heartbeat/heartbeat-8.16.0-linux-x86_64.tar.gz.asc
curl -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/beats/metricbeat/metricbeat-8.16.0-linux-x86_64.tar.gz.asc
curl -O https://artifacts.elastic.co/downloads/beats/osquerybeat/osquerybeat-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/beats/osquerybeat/osquerybeat-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/beats/osquerybeat/osquerybeat-8.16.0-linux-x86_64.tar.gz.asc
curl -O https://artifacts.elastic.co/downloads/beats/packetbeat/packetbeat-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/beats/packetbeat/packetbeat-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/beats/packetbeat/packetbeat-8.16.0-linux-x86_64.tar.gz.asc
curl -O https://artifacts.elastic.co/downloads/cloudbeat/cloudbeat-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/cloudbeat/cloudbeat-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/cloudbeat/cloudbeat-8.16.0-linux-x86_64.tar.gz.asc
curl -O https://artifacts.elastic.co/downloads/endpoint-dev/endpoint-security-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/endpoint-dev/endpoint-security-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/endpoint-dev/endpoint-security-8.16.0-linux-x86_64.tar.gz.asc
curl -O https://artifacts.elastic.co/downloads/fleet-server/fleet-server-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/fleet-server/fleet-server-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/fleet-server/fleet-server-8.16.0-linux-x86_64.tar.gz.asc
curl -O https://artifacts.elastic.co/downloads/prodfiler/pf-host-agent-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/prodfiler/pf-host-agent-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/prodfiler/pf-host-agent-8.16.0-linux-x86_64.tar.gz.asc
curl -O https://artifacts.elastic.co/downloads/prodfiler/pf-elastic-collector-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/prodfiler/pf-elastic-collector-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/prodfiler/pf-elastic-collector-8.16.0-linux-x86_64.tar.gz.asc
curl -O https://artifacts.elastic.co/downloads/prodfiler/pf-elastic-symbolizer-8.16.0-linux-x86_64.tar.gz
curl -O https://artifacts.elastic.co/downloads/prodfiler/pf-elastic-symbolizer-8.16.0-linux-x86_64.tar.gz.sha512
curl -O https://artifacts.elastic.co/downloads/prodfiler/pf-elastic-symbolizer-8.16.0-linux-x86_64.tar.gz.asc

    2. On your HTTP file server, group the artifacts into directories and sub-directories that follow the same convention used by the Elastic Artifact Registry:

      <source_uri>/<artifact_type>/<artifact_name>-<version>-<arch>-<package_type>

      Where:

      • <artifact_type> is in the format beats/elastic-agent, fleet-server, endpoint-dev, and so on.
      • <artifict_name> is om the format elastic-agent, endpoint-security, or fleet-server and so on.
      • arch-package-type is in the format linux-x86_64, linux-arm64, windows_x86_64, darwin_x86_64, or darwin_aarch64`.

      • If you’re using the DEB package manager:

        • The 64bit variant has the format <artifact_name>-<version>-amd64.deb.
        • The aarch64 variant has the format <artifact_name>-<version>-arm64.deb.

      • If you’re using the RPM package manager:

        • The 64bit variant has a format <artifact_name>-<version>-x86_64.rpm.
        • The aarch64 variant has a format <artifact_name>-<version>-aarch64.rpm.
    • If you’re ever in doubt, visit the Elastic Agent download page to see what URL the various binaries are downloaded from.
    • Make sure you have a plan or automation in place to update your artifact registry when new versions of Elastic Agent are available.

  2. Add the agent binary download location to Fleet settings:

    1. Open Fleet → Settings.
    2. Under Agent Binary Download, click Add agent binary source to add the location of your artifact registry. For more detail about these settings, refer to Agent binary download settings. If you want all Elastic Agents to download binaries from this location, set it as the default.

  3. If your artifact registry is not the default, edit your agent policies to override the default:

    1. Go to Fleet → Agent policies and click the policy name to edit it.
    2. Click Settings.

    3. Under Agent Binary Download, select your artifact registry.

      When you trigger an upgrade for any Elastic Agents enrolled in the policy, the binaries are downloaded from your artifact registry instead of the public repository.

Not using Fleet? For standalone Elastic Agents, you can set the binary download location under agent.download.sourceURI in the elastic-agent.yml file, or run the elastic-agent upgrade command with the --source-uri flag specified.

« Installation layout Using a proxy server with Elastic Agent and Fleet »