Red Hat Enterprise Linux (RHEL) 7 and CentOS 7

edit

Red Hat Enterprise Linux (RHEL) 7 and CentOS 7

edit

The following instructions show you how to prepare your hosts on Red Hat Enterprise Linux (RHEL) 7 or later and on CentOS 7 or later.

Install a supported Linux kernel

edit

Elastic Cloud Enterprise requires 3.10 or higher. Before you proceed, update the OS, and reboot the system.

Kernel-LT has a regression on 4.4.156. From the archive repository, install 4.4.155.

  1. Add the repository required to obtain the kernel:

    yum update
    reboot
  2. Add the required options to the kernel boot arguments:

    sudo /sbin/grubby --update-kernel=ALL --args='cgroup_enable=memory cgroup.memory=nokmem swapaccount=1'
  3. Enable the overlay kernel module:

    echo "overlay" | sudo tee -a /etc/modules-load.d/overlay.conf
  4. Refresh the dynamically generated grub2 configuration and configure grub to boot the newly installed kernel 3.10 or higher:

    sudo grub2-set-default 0
    sudo grub2-mkconfig -o /etc/grub2.cfg

Install Docker

edit

Which version of Docker you install depends on whether you use Red Hat Enterprise Linux (RHEL) or CentOS.

RHEL

edit

If you are using RHEL 7.5 or later, we recommend that you use Docker 1.13 included with your RHEL distribution:

If the docker daemon does not start after correctly install, continue with the rest of the configuration steps below.

  1. Follow the same Docker configuration steps as those for CentOS.

CentOS

edit

If you are using CentOS, install Docker LTS version 18.09.2:

  1. Add the Docker repository:

    sudo tee /etc/yum.repos.d/docker.repo <<-'EOF'
    [dockerrepo]
    name=Docker Repository
    baseurl=https://download.docker.com/linux/centos/7/x86_64/stable
    enabled=1
    gpgcheck=1
    gpgkey=https://download.docker.com/linux/centos/gpg
    EOF
    
    sudo yum makecache fast
  2. Install the latest version of docker-ce:

    sudo yum install docker-ce-18.09.2* containerd.io-1.4.3-3.1*

Recommended: Set Up XFS quotas

edit

Elastic Cloud Enterprise can run without XFS quotas, but they are required to support disk space quotas for Elasticsearch data directories. Some Linux distributions such as RHEL and CentOS already provide XFS as the default file system; however, quotas might be disabled.

Disk space quotas set a limit on the amount of disk space a cluster node can use. Currently, quotas are calculated by a static ratio of 1:32, which means that for every 1 GB of RAM a cluster is given, a cluster node is allowed to consume 32 GB of disk space.

Using LVM, mdadm, or a combination of the two for block device management is possible, but the configuration is not covered here, nor is it provided as part of supporting Elastic Cloud Enterprise.

If you use XFS, you must use XFS on all allocators.

To set up XFS with quotas on a single, pre-partitioned block device named /dev/xvdg1:

  1. Format the partition:

    sudo mkfs.xfs /dev/xvdg1
  2. Create the /mnt/data/ directory as a mount point:

    sudo install -o $USER -g $USER -d -m 700 /mnt/data
  3. Modify the entry for the XFS volume in the /etc/fstab file to add pquota,prjquota. The default filesystem path used by Elastic Cloud Enterprise is /mnt/data.

    /dev/xvdg1	/mnt/data	xfs	defaults,nofail,x-systemd.automount,prjquota,pquota  0 2
  4. Regenerate the mount files:

    sudo systemctl daemon-reload
    sudo systemctl restart local-fs.target

Update the configurations settings

edit
  1. Stop the Docker service:

    sudo systemctl stop docker
  2. Configure kernel parameters

    cat <<EOF | sudo tee -a /etc/sysctl.conf
    # Required by Elasticsearch 5.0 and later
    vm.max_map_count = 262144
    # enable the setting in order for Docker remove the containers cleanly
    fs.may_detach_mounts = 1
    # enable forwarding so the Docker networking works as expected
    net.ipv4.ip_forward = 1
    # Make sure the host doesn't swap too early
    vm.swappiness=1
    EOF
    1. Apply the settings:

      sudo sysctl -p
      sudo service network restart
  3. Adjust the system limits.

    Add the following configuration values to the /etc/security/limits.conf file. These values are derived from our experience with the Elastic Cloud hosted offering and should be used for Elastic Cloud Enterprise as well.

    If you are using a user name other than elastic, adjust the configuration values accordingly.

    *                soft    nofile         1024000
    *                hard    nofile         1024000
    *                soft    memlock        unlimited
    *                hard    memlock        unlimited
    elastic          soft    nofile         1024000
    elastic          hard    nofile         1024000
    elastic          soft    memlock        unlimited
    elastic          hard    memlock        unlimited
    root             soft    nofile         1024000
    root             hard    nofile         1024000
    root             soft    memlock        unlimited

The default limit for number of processes is too low. Remove it and rely on the kernel limit instead.

sudo rm /etc/security/limits.d/20-nproc.conf
  1. If you did not create the mount point earlier (if you did not set up XFS), create the /mnt/data/ directory as a mount point:

    sudo install -o $USER -g $USER -d -m 700 /mnt/data
  2. If you set up a new device with XFS earlier:

    1. Mount the block device (change the device name if you use a different device than /dev/xvdg1):

      sudo mount /dev/xvdg1
    2. Set the permissions on the newly mounted device:

      sudo chown $USER:$USER /mnt/data
  3. Create the /mnt/data/docker directory for the Docker service storage:

    sudo install -o $USER -g $USER -d -m 700 /mnt/data/docker
  4. Disable the firewalld service. The service is not compatible with Docker and interferes with the installation of ECE. You must disable firewalld before installing or reinstalling ECE.

    sudo systemctl disable firewalld

Configure the Docker daemon options

edit

Docker creates a bridge IP address that can conflict with IP addresses on your internal network. To avoid an IP address conflict, change the --bip=172.17.42.1/16 parameter in our examples to something that you know will work. If there is no conflict, you can omit the --bip parameter. The --bip parameter is internal to the host and can be set to the same IP for each host in the cluster. More information on Docker daemon options can be found in the dockerd command line reference.

CentOS

edit
  1. Update the /etc/systemd/system/docker.service.d/docker.conf file. If the file path and file do not exist, create them.

    [Unit]
    Description=Docker Service
    After=multi-user.target
    
    [Service]
    ExecStart=
    ExecStart=/usr/bin/dockerd --data-root /mnt/data/docker --storage-driver=overlay --bip=172.17.42.1/16

With Docker version 1.13.x, Docker will not start with the --data-root option. If this is the case, try using the --graph option instead.

  1. Apply the updated Docker daemon configuration:

    1. Reload the Docker daemon configuration:

      sudo systemctl daemon-reload
    2. Restart the Docker service:

      sudo systemctl restart docker
    3. Enable Docker to start on boot:

      sudo systemctl enable docker
  2. Enable your user to communicate with the Docker subsystem by adding it to the docker group:

    sudo usermod -aG docker $USER
  3. Pin the Docker version to ensure that the docker-ce package does not get upgraded:

    echo "exclude=docker-ce" | sudo tee -a /etc/yum.conf

RHEL

edit
  1. Update the system docker configuration:

    1. Edit the /etc/sysconfig/docker file and add the following to the OPTIONS variable:

      OPTIONS="... -g /mnt/data/docker --log-driver json-file"
    2. Edit the /etc/sysconfig/docker-network file to set the docker network options:

      DOCKER_NETWORK_OPTIONS="--bip=172.17.42.1/16"
    3. Edit the /etc/sysconfig/docker-storage-setup file to set the STORAGE_DRIVER option:

      STORAGE_DRIVER=overlay
  2. Apply the updated Docker daemon configuration:

    1. Reload the Docker daemon configuration:

      sudo systemctl daemon-reload
    2. Restart the Docker service:

      sudo systemctl restart docker
    3. Enable Docker to start on boot:

      sudo systemctl enable docker
  1. Enable your user to communicate with the Docker subsystem by adding it to the docker group:

    sudo groupadd -f docker
    sudo usermod -aG dockerroot docker $USER
    sudo chown root:docker /var/run/docker.sock
  2. Pin the Docker version to ensure that the docker package does not get upgraded:

    echo "exclude=docker" | sudo tee -a /etc/yum.conf
  3. Recommended: Tune your network settings.

    Create a 70-cloudenterprise.conf file in the /etc/sysctl.d/ file path that includes these network settings:

    cat << SETTINGS | sudo tee /etc/sysctl.d/70-cloudenterprise.conf
    net.ipv4.tcp_max_syn_backlog=65536
    net.core.somaxconn=32768
    net.core.netdev_max_backlog=32768
    SETTINGS
  4. Reboot your system to ensure that all configuration changes take effect:

    sudo reboot
  5. After rebooting, verify that your Docker settings persist as expected:

    sudo docker info | grep Root

    If the command returns Docker Root Dir: /mnt/data/docker, then your changes were applied successfully and persist as expected.

    If the command returns Docker Root Dir: /var/lib/docker, then you need to troubleshoot the previous configuration steps until the Docker settings are applied successfully before continuing with the installation process. For more information, see Custom Docker daemon options in the Docker documentation.

  6. Repeat these steps on other hosts that you want to use with Elastic Cloud Enterprise or follow the steps in the next section to start installing Elastic Cloud Enterprise.