Customers

Deploying Elasticsearch 2.0.0 with Chef

By
Martin Smith, Rackspace

chef-logo.pngWith the recent release bonanza at Elastic, Elasticsearch 2.0.0 has arrived, along with new versions of Kibana and Logstash, and many Elasticsearch plugins too. A couple of weeks later, a completely new, refactored and rewritten Elasticsearch cookbook for Chef has been released as well.

In this post, Martin Smith, the principal author of the rewrite, will show you how to install and configure an Elasticsearch node using the current version of the cookbook and Hosted Chef. If you want to follow along this tutorial, you can download the complete wrapper cookbook from Github.


Chef

Chef is a configuration tool written in Ruby and Erlang. It uses a pure-Ruby, domain-specific language (DSL) for writing system configuration “recipes”. Chef is used to streamline the task of configuring and maintaining a company’s servers, and can integrate with cloud-based platforms such as Rackspace, Internap, Amazon EC2, Google Cloud Platform, OpenStack, SoftLayer, and Microsoft Azure to automatically provision and configure new machines.

As mentioned above, the Elasticsearch cookbook for Chef can help you move from a demo environment to a full-fledged production ready cluster. With the newest 2.0.0 release, the cookbook can help you automate and maintain the following:

Features of the Chef cookbook for Elasticsearch:

  • Install, remove, and configure a system user and group for Elasticsearch
  • Add Yum and Apt package repositories for Elasticsearch
  • Download and unpack binary (.tar.gz) distributions of Elasticsearch
  • Install Elasticsearch to a system from package manager or binary distribution
  • Create and maintain the Elasticsearch configuration files, using centralized data from Chef
  • Configure a system service and init script for controlling the Elasticsearch service
  • Install and remove Elasticsearch plugins, optionally restarting Elasticsearch when needed
  • Configure cluster and discovery options using Chef search to connect cluster nodes together

Pre-requisites

Before we begin, there a few things you should have already for this tutorial. Namely

  1. A workstation with the Chef Development Kit installed, as well as git and common tools like a text editor and zip/unzip utility. In the examples, we are using ChefDK 0.10.0 for Ubuntu 14.04.
  2. An account on Hosted Chef. Visit manage.chef.io/signup to sign up and create a default Chef organization for your user. At the time of this writing, Hosted Chef is free for up to 5 servers.
  3. ChefDK should be configured to use the account you setup in the previous step. This involves downloading the starter kit, and extracting the .chef directory it contains, and moving it to ~/.chef.
  4. A fresh server capable of being used for Elasticsearch. In our examples, we’ll use one that is at least 1gb of RAM and enough disk space to hold Elasticsearch 2.0.0 itself, plus any data we plan to load into it.
  5. Java 7+ must be installed on the fresh server. This article does not cover how to install Java with Chef, though this is also certainly possible.

With those out of the way, let’s start by showing how to build a wrapper cookbook that can install, configure, and start the Elasticsearch software on a new server.

Build your wrapper cookbook

Now we’re ready to build our new cookbook. We’re using the wrapper cook book here, which means we’re making a new cookbook that we run directly on our new ES server; it will be the only cookbook we run on our ES server, and it will encapsulate all of the logic and dependencies for the new server as well.

We will be calling our wrapper cookbook elasticsearch-chef-blog, and we’ll use ChefDK’s chef command to create it:

root@chef-workstation01:~# chef generate cookbook elasticsearch-chef-blog-demo
Compiling Cookbooks...
Recipe: code_generator::cookbook
...
root@chef-workstation01:~# cd elasticsearch-chef-blog-demo/
root@chef-workstation01:~/elasticsearch-chef-blog-demo# ls -la
total 48
drwxr-xr-x 6 root root 4096 Nov 22 20:39 .
drwx------ 8 root root 4096 Nov 22 20:39 ..
-rw-r--r-- 1 root root 47 Nov 22 20:39 Berksfile
-rw-r--r-- 1 root root 1029 Nov 22 20:39 chefignore
drwxr-xr-x 7 root root 4096 Nov 22 20:39 .git
-rw-r--r-- 1 root root 126 Nov 22 20:39 .gitignore
-rw-r--r-- 1 root root 355 Nov 22 20:39 .kitchen.yml
-rw-r--r-- 1 root root 267 Nov 22 20:39 metadata.rb
-rw-r--r-- 1 root root 77 Nov 22 20:39 README.md
drwxr-xr-x 2 root root 4096 Nov 22 20:39 recipes
drwxr-xr-x 3 root root 4096 Nov 22 20:39 spec
drwxr-xr-x 3 root root 4096 Nov 22 20:39 test


Next, we’re going to add the Elasticsearch cookbook as a dependency in metadata.rb in our new cookbook. We’ll add the line depends 'elasticsearch', '>= 2.0.0' at the bottom of metadata.rb, so that our full file now looks like so:

name 'elasticsearch-chef-blog-demo'
maintainer 'The Authors'
maintainer_email 'you@example.com'
license 'all_rights'
description 'Installs/Configures elasticsearch-chef-blog-demo'
long_description 'Installs/Configures elasticsearch-chef-blog-demo'
version '0.1.0'
depends 'elasticsearch', '>= 2.0.0'


Note: The elasticsearch dependency here is for the Chef cookbook, not the Elasticsearch software itself.

Write your recipe

Next, we need to write the simple recipe that utilizes Chef resources to install a simple Elasticsearch node. We will edit the file recipes/default.rb, and we’ll place the following snippets there:

A note about resource names

Many of the resources provided in this cookbook need to share configuration values. For example, the elasticsearch_service resource needs to know the path to the configuration file(s) generated by

elastcisearch_configure and the path to the actual ES binary installed by elasticsearch_install. And they both need to know the appropriate system user and group defined by elasticsearch_user.

Search order: In order to make this easy, all resources in this cookbook use the following search order to locate resources that apply to the same overall Elasticsearch setup: 

  1. Resources that share the same resource name 
  2. Resources that share the same value for instance_name 
  3. Resources named default or resources named elasticsearch
    Note: This fails if both default and elasticsearch resources exist

Create a user

Whether we decide to install from a .tar.gz archive or using the provided OS packages, we use the elasticsearch_user resource here to be sure that a user and group are created for Elasticsearch on our system. The later resources will use this resource to determine what the appropriate user should be for Elasticsearch. By default, we name this user elasticsearch.

elasticsearch_user 'elasticsearch'


Alternately, if you’re installing by package, and would rather let the package create the user, you can set the action to :nothing on this resource, but it must still exist for the other resources to look up and retrieve the desired user. Here’s what that would look like:

elasticsearch_user 'elasticsearch' do
action :nothing
end


An even more complete example, with many of the defaults explicitly set, would be:

elasticsearch_user 'elasticsearch' do
username 'elasticsearch'
groupname 'elasticsearch'
shell '/bin/bash'
comment 'Elasticsearch User'
action :create
end


Install Elasticsearch

Next, with a user and group in place, it’s time to install elasticsearch. By default, the cookbook chooses the package installation, so the most minimal installation can simply be elasticsearch_install 'elasticsearch', however for our case, we want to remind ourselves of the default installation method, so we explicitly list it:

elasticsearch_install 'elasticsearch' do
type :package
end


This resource will download the correct package (using a default version from the elasticsearch chef cookbook, currently 2.0.0), and install it using the package manager. If you’d like to override the version, here’s a more elaborate example that specifies exactly what version to downloading and install:

elasticsearch_install 'elasticsearch' do
type :package
version "1.7.2"
action :install
end


If you’re wondering how the elasticsearch chef cookbook knows about specific versions, or figures out where to download them from, check out the attributes/default.rb file. That file has a complete list of known versions. If a newer version isn’t listed yet, we might be behind and would appreciate your opening an issue to let us know that we need an update! If you want even more control over the downloaded package, you can even specify a full URL using the url and checksum parameters to the resource:

elasticsearch_install 'elasticsearch' do
type :package
version "1.7.2"
download_url "https://download.elasticsearch.org/elasticsearch/elasticsearch/elasticsearch-1.7.2.deb"
download_checksum "791fb9f2131be2cf8c1f86ca35e0b912d7155a53f89c2df67467ca2105e77ec2" # sha256 checksum
action :install
end


It’s important to remember that Chef is idempotent here. It will query the package manager on each run, verifying that the specific package name at the given version is already installed. If a server is updated to a newer package, or is on an older package version, Chef will try to downgrade or upgrade as appropriate. It’s always critical that any upgrade or downgrade attempted via chef is tested before being applied to a production system. If you need to force a particular package manager operation, there is an attribute provided for that as well.

Configure Elasticsearch

Now that we’ve written our install resource into the recipe, it’s time to configure an Elasticsearch instance. In our pre-requisites, we specified that you should have a server with at least a 1GB of memory, so we’re going to define an instance that uses 512M of memory. We’ve also given a cluster and node name for our instance. It’s important to note that all Elasticsearch configuration items should be given as a single string, unlike in previous versions of the cookbook where you could nest these hashes.

elasticsearch_configure 'elasticsearch' do
allocated_memory '256m'
configuration ({
'cluster.name' => 'mycluster',
'node.name' => 'node01'
})
end


While this is a trivial example, this resource can get complex quickly. If you take a look at the resource file, you can see there are a ton of default values that you might want to override. Most importantly, take note that most configuration parameters for the resource are a hash with defaults both for the packaged installation and the tarball/binary installation.

elasticsearch_configure 'elasticsearch' do
allocated_memory '256m'
path_data package: "/mnt/high_performance_data_disk"
configuration ({
'cluster.name' => 'mycluster',
'node.name' => 'node01'
})
end


Above is another example of configuring an elasticsearch instance. While normally path.data must be supplied in the configuration hash. We’re taking advantage of the fact that it’s one of the three paths that will automatically be populated in the configuration hash if we use the corresponding resource attribute. If you’d like to see our entire template for elasticsearch.yml, it’s in the cookbook repository too.

Run Elasticsearch as a system service

Now that our instance has some configuration files written, it’s time to be sure it has a system service. The elasticsearch_service defines a system service and enables it on boot. Note that, however, like the packaged version of Elasticsearch, this resource does not start the service automatically, to protect you from accidental data loss or joining the wrong cluster (this is extremely common on some cloud providers). If you’d like to start the service immediately, you can configure the service_actions parameter of the resource with any actions you want passed to Chef’s service resource implementation. In our example below, we are asking Chef to start the enable (start on boot) and start the service right now.

elasticsearch_service 'elasticsearch' do
service_actions [:enable, :start]
end

In addition to using the provided init script, you can ask the elasticsearch_service resource to use a template from your own wrapper cookbook. While we haven’t done that in our demo, here is the syntax for what it would look like:



elasticsearch_service 'elasticsearch' do
init_source 'my_custom_initscript.erb'
init_cookbook 'elasticsearch-chef-blog-demo'
service_actions [:enable, :start]
end


Finally, it’s worth pointing out that you can notify elasticsearch_service with any Chef::Resource::Service notifications and they will be passed through to the underlying service resource. We take advantage of this in the next section.

Install some plugins for Elasticsearch

Last, but not certainly not least, we install some plugins. To do this, we’re using the elasticsearch_plugin resource. Note that the resource name must be the same as the plugin name, and you can use the url parameter to specify the argument(s) used for the Elasticsearch plugin install command. If you wish to name the resource something else, you can provide the plugin_name parameter as well.

elasticsearch_plugin 'head' do
url 'mobz/elasticsearch-head'
notifies :restart, 'elasticsearch_service[elasticsearch]', :delayed
end


Note that the example above sends a restart notification to the service, automatically restarting Elasticsearch if the plugin was changed in any way (if we installed or removed it on this chef run). This could be dangerous in a cluster, if too many nodes are restarted at once, or if a plugin causes Elasticsearch to fail to start. 

Here are some examples of using the additional attributes too. Note how some of the special supported plugins require no url parameter at all, and how you can use a version in the URL.

elasticsearch_plugin 'shield' do
action :install
end
elasticsearch_plugin 'kopf' do
url 'lmenezes/elasticsearch-kopf/1.5.7'
action :install
end
elasticsearch_plugin 'xyzzy' do
name 'kopf'
url 'lmenezes/elasticsearch-kopf'
action :install
end


If you’d like to download the complete working example, the complete cookbook can be downloaded at Github. Also, note that we provided many examples above, but we’re going to use the very basic ones for the provided sample code/cookbook.

Upload your work to Hosted Chef

We’ll use Berkshelf to get all of our dependencies, and again, use Berkshelf to upload them all to Hosted Chef. After running berks install && berks upload, you should see the two relevant cookbooks we care about being fetched and eventually uploaded to Hosted Chef:

root@chef-workstation01:~/elasticsearch-chef-blog-demo# chef exec berks install && chef exec berks upload
Resolving cookbook dependencies...
Uploaded elasticsearch (2.0.0) to: 'https://api.chef.io:443/organizations/es-chef-demo'
Uploaded elasticsearch-chef-blog-demo (0.1.0) to: 'https://api.chef.io:443/organizations/es-chef-demo'


Deploying Elasticsearch on the node using Chef

Now that we have a cookbook uploaded and ready to go, we’ll bootstrap the fresh server we created as part of the list of pre-requisites (remember, it should have Java pre-installed, too). We’ll specify that we want to use our brand new wrapper cookbook on it, as well. 

The file-by-file diffs have been snipped from the log below for brevity.

root@chef-workstation01:~/elasticsearch-chef-blog-demo# knife bootstrap -r "recipe[elasticsearch-chef-blog-demo::default]" -N es01 104.130.170.231
Connecting to 104.130.170.231
root@104.130.170.231's password:
104.130.170.231 -----> Installing Chef Omnibus (-v 12)
104.130.170.231 downloading https://www.opscode.com/chef/install.sh
104.130.170.231 to file /tmp/install.sh.4973/install.sh
104.130.170.231 trying wget...
104.130.170.231 Downloading Chef 12 for ubuntu...
104.130.170.231 downloading https://www.opscode.com/chef/metadata?v=12&prerelease=false&nightlies=false&p=ubuntu&pv=14.04&m=x86_64
104.130.170.231 to file /tmp/install.sh.4977/metadata.txt
104.130.170.231 trying wget...
104.130.170.231 url https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/14.04/x86_64/chef_12.5.1-1_amd64.deb
104.130.170.231 md5 d8fec2da288e94a7e2d649803a9d70f4
104.130.170.231 sha256 656a4c4a8fd64d74d1d970fb0d07076d6f1d8230d37d751f2c3698a52d82c070
104.130.170.231 downloaded metadata file looks valid...
104.130.170.231 downloading https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/14.04/x86_64/chef_12.5.1-1_amd64.deb
104.130.170.231 to file /tmp/install.sh.4977/chef_12.5.1-1_amd64.deb
104.130.170.231 trying wget...
104.130.170.231 Comparing checksum with sha256sum...
104.130.170.231 Installing Chef 12
104.130.170.231 installing with dpkg...
104.130.170.231 Selecting previously unselected package chef.
(Reading database ... 52958 files and directories currently installed.)
104.130.170.231 Preparing to unpack .../chef_12.5.1-1_amd64.deb ...
104.130.170.231 Unpacking chef (12.5.1-1) ...
104.130.170.231 Setting up chef (12.5.1-1) ...
104.130.170.231 Thank you for installing Chef!
104.130.170.231 Starting the first Chef Client run...
104.130.170.231 Starting Chef Client, version 12.5.1
104.130.170.231 Creating a new client identity for es01 using the validator key.
104.130.170.231 resolving cookbooks for run list: ["elasticsearch-chef-blog-demo::default"]
104.130.170.231 Synchronizing Cookbooks:
104.130.170.231 - elasticsearch-chef-blog-demo (0.1.0)
104.130.170.231 - elasticsearch (2.0.0)
104.130.170.231 - yum (3.8.2)
104.130.170.231 - apt (2.9.2)
104.130.170.231 - chef-sugar (3.1.1)
104.130.170.231 - ark (0.9.0)
104.130.170.231 - windows (1.38.4)
104.130.170.231 - 7-zip (1.0.2)
104.130.170.231 - chef_handler (1.2.0)
104.130.170.231 Compiling Cookbooks...
104.130.170.231 Converging 5 resources
104.130.170.231 Recipe: elasticsearch-chef-blog-demo::default
104.130.170.231 * elasticsearch_user[elasticsearch] action create
104.130.170.231 * group[elasticsearch] action create
104.130.170.231 - create elasticsearch
104.130.170.231 * user[elasticsearch] action create
104.130.170.231 - create user elasticsearch
104.130.170.231
104.130.170.231 * group[elasticsearch] action nothing (skipped due to action :nothing)
104.130.170.231 * user[elasticsearch] action nothing (skipped due to action :nothing)
104.130.170.231 * elasticsearch_install[elasticsearch] action install
104.130.170.231 * remote_file[/var/chef/cache/elasticsearch-2.0.0.deb] action create
104.130.170.231 - create new file /var/chef/cache/elasticsearch-2.0.0.deb
104.130.170.231 - update content in file /var/chef/cache/elasticsearch-2.0.0.deb from none to f846ca
104.130.170.231 (file sizes exceed 10000000 bytes, diff output suppressed)
104.130.170.231 - change mode from '' to '0644'
104.130.170.231 * dpkg_package[/var/chef/cache/elasticsearch-2.0.0.deb] action install
104.130.170.231 - install version 2.0.0 of package /var/chef/cache/elasticsearch-2.0.0.deb
104.130.170.231
104.130.170.231 * remote_file[/var/chef/cache/elasticsearch-2.0.0.deb] action nothing (skipped due to action :nothing)
104.130.170.231 * dpkg_package[/var/chef/cache/elasticsearch-2.0.0.deb] action nothing (skipped due to action :nothing)
104.130.170.231 * elasticsearch_configure[elasticsearch] action manage
104.130.170.231 * directory[/etc/elasticsearch] action create
104.130.170.231 - change mode from '0750' to '0755'
104.130.170.231 - change owner from 'root' to 'elasticsearch'
104.130.170.231 * directory[/etc/elasticsearch/scripts] action create
104.130.170.231 - change mode from '0750' to '0755'
104.130.170.231 - change owner from 'root' to 'elasticsearch'
104.130.170.231 * directory[/var/log/elasticsearch] action create (up to date)
104.130.170.231 * directory[/usr/share/elasticsearch] action create
104.130.170.231 - change owner from 'root' to 'elasticsearch'
104.130.170.231 - change group from 'root' to 'elasticsearch'
104.130.170.231 * template[elasticsearch.in.sh] action create
104.130.170.231 - update content in file /etc/default/elasticsearch from 333105 to 1d15e2
104.130.170.231 --- /etc/default/elasticsearch 2015-10-22 08:35:02.000000000 +0000
104.130.170.231 +++ /etc/default/.elasticsearch.in.sh20151123-5118-10fssxf 2015-11-23 15:27:07.265261999 +0000
104.130.170.231 @@ -1,73 +1,20 @@
104.130.170.231 ################################
104.130.170.231 +# THIS FILE IS MANAGED BY CHEF
104.130.170.231 +################################
104.130.170.231 +CONF_DIR="/etc/elasticsearch"
104.130.170.231 +DATA_DIR="/usr/share/elasticsearch"
104.130.170.231 +ES_GROUP="elasticsearch"
104.130.170.231 +ES_HEAP_SIZE="256m"
104.130.170.231 +ES_HOME="/usr/share/elasticsearch"
104.130.170.231 +ES_JAVA_OPTS="-server -Djava.awt.headless=true -Djava.net.preferIPv4Stack=true -Xms256m -Xmx256m -Xss256k -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true "
104.130.170.231 +ES_USER="elasticsearch"
104.130.170.231 +LOG_DIR="/var/log/elasticsearch"
104.130.170.231 +MAX_LOCKED_MEMORY="unlimited"
104.130.170.231 +MAX_OPEN_FILES="64000"
104.130.170.231 +PID_DIR="/var/run/elasticsearch"
104.130.170.231 - change mode from '0644' to '0755'
104.130.170.231 * template[logging.yml] action create
104.130.170.231 - update content in file /etc/elasticsearch/logging.yml from 58b25a to 4ce76a
104.130.170.231 --- /etc/elasticsearch/logging.yml 2015-10-22 08:35:02.000000000 +0000
104.130.170.231 +++ /etc/elasticsearch/.logging.yml20151123-5118-91px6c 2015-11-23 15:27:07.277261999 +0000
104.130.170.231 @@ -1,38 +1,40 @@
104.130.170.231 +# ----- Configuration set by Chef ---------------------------------------------
104.130.170.231 - change mode from '0750' to '0755'
104.130.170.231 - change owner from 'root' to 'elasticsearch'
104.130.170.231 * template[elasticsearch.yml] action create
104.130.170.231 - update content in file /etc/elasticsearch/elasticsearch.yml from b17a8d to ebb6be
104.130.170.231 --- /etc/elasticsearch/elasticsearch.yml 2015-10-22 08:35:02.000000000 +0000
104.130.170.231 +++ /etc/elasticsearch/.elasticsearch.yml20151123-5118-7cipao 2015-11-23 15:27:07.289261999 +0000
104.130.170.231 @@ -1,97 +1,57 @@
104.130.170.231 +# THIS FILE IS MANAGED BY CHEF, DO NOT EDIT MANUALLY, YOUR CHANGES WILL BE OVERWRITTEN!
104.130.170.231 - change mode from '0750' to '0755'
104.130.170.231 - change owner from 'root' to 'elasticsearch'
104.130.170.231
104.130.170.231 * directory[/etc/elasticsearch] action nothing (skipped due to action :nothing)
104.130.170.231 * directory[/etc/elasticsearch/scripts] action nothing (skipped due to action :nothing)
104.130.170.231 * directory[/var/log/elasticsearch] action nothing (skipped due to action :nothing)
104.130.170.231 * directory[/usr/share/elasticsearch] action nothing (skipped due to action :nothing)
104.130.170.231 * template[elasticsearch.in.sh] action nothing (skipped due to action :nothing)
104.130.170.231 * template[logging.yml] action nothing (skipped due to action :nothing)
104.130.170.231 * template[elasticsearch.yml] action nothing (skipped due to action :nothing)
104.130.170.231 * elasticsearch_service[elasticsearch] action configure
104.130.170.231 * directory[/var/run/elasticsearch] action create (up to date)
104.130.170.231 * template[/etc/init.d/elasticsearch] action create (up to date)
104.130.170.231 * service[elasticsearch] action enable
104.130.170.231 - enable service service[elasticsearch]
104.130.170.231 * service[elasticsearch] action start
104.130.170.231 - start service service[elasticsearch]
104.130.170.231
104.130.170.231 * directory[/var/run/elasticsearch] action nothing (skipped due to action :nothing)
104.130.170.231 * template[/etc/init.d/elasticsearch] action nothing (skipped due to action :nothing)
104.130.170.231 * service[elasticsearch] action nothing (skipped due to action :nothing)
104.130.170.231 * elasticsearch_plugin[head] action install
104.130.170.231
104.130.170.231 * elasticsearch_service[elasticsearch] action restart
104.130.170.231 * service[elasticsearch] action restart
104.130.170.231 - restart service service[elasticsearch]
104.130.170.231
104.130.170.231
104.130.170.231 Running handlers:
104.130.170.231 Running handlers complete
104.130.170.231 Chef Client finished, 19/36 resources updated in 09 seconds


As you can see in the log above, we went from a fresh server to an Elasticsearch server in 9 seconds. You can also see service[elasticsearch] was restarted because the head plugin was installed. If we run chef-client a second time, you can also see that Elasticsearch is not restarted, and everything reports (up to date):

root@es01:~# chef-client
Starting Chef Client, version 12.5.1
resolving cookbooks for run list: ["elasticsearch-chef-blog-demo::default"]
Synchronizing Cookbooks:
- elasticsearch-chef-blog-demo (0.1.0)
- elasticsearch (2.0.0)
- yum (3.8.2)
- apt (2.9.2)
- chef-sugar (3.1.1)
- ark (0.9.0)
- windows (1.38.4)
- 7-zip (1.0.2)
- chef_handler (1.2.0)
Compiling Cookbooks...
Converging 5 resources
Recipe: elasticsearch-chef-blog-demo::default
* elasticsearch_user[elasticsearch] action create
* group[elasticsearch] action create (up to date)
* user[elasticsearch] action create (up to date)
(up to date)
* group[elasticsearch] action nothing (skipped due to action :nothing)
* user[elasticsearch] action nothing (skipped due to action :nothing)
* elasticsearch_install[elasticsearch] action install
* remote_file[/var/chef/cache/elasticsearch-2.0.0.deb] action create (up to date)
* dpkg_package[/var/chef/cache/elasticsearch-2.0.0.deb] action install (up to date)
(up to date)
* remote_file[/var/chef/cache/elasticsearch-2.0.0.deb] action nothing (skipped due to action :nothing)
* dpkg_package[/var/chef/cache/elasticsearch-2.0.0.deb] action nothing (skipped due to action :nothing)
* elasticsearch_configure[elasticsearch] action manage
* directory[/etc/elasticsearch] action create (up to date)
* directory[/etc/elasticsearch/scripts] action create (up to date)
* directory[/var/log/elasticsearch] action create (up to date)
* directory[/usr/share/elasticsearch] action create (up to date)
* template[elasticsearch.in.sh] action create (up to date)
* template[logging.yml] action create (up to date)
* template[elasticsearch.yml] action create (up to date)
(up to date)
* directory[/etc/elasticsearch] action nothing (skipped due to action :nothing)
* directory[/etc/elasticsearch/scripts] action nothing (skipped due to action :nothing)
* directory[/var/log/elasticsearch] action nothing (skipped due to action :nothing)
* directory[/usr/share/elasticsearch] action nothing (skipped due to action :nothing)
* template[elasticsearch.in.sh] action nothing (skipped due to action :nothing)
* template[logging.yml] action nothing (skipped due to action :nothing)
* template[elasticsearch.yml] action nothing (skipped due to action :nothing)
* elasticsearch_service[elasticsearch] action configure
* directory[/var/run/elasticsearch] action create (up to date)
* template[/etc/init.d/elasticsearch] action create (up to date)
* service[elasticsearch] action enable (up to date)
* service[elasticsearch] action start (up to date)
(up to date)
* directory[/var/run/elasticsearch] action nothing (skipped due to action :nothing)
* template[/etc/init.d/elasticsearch] action nothing (skipped due to action :nothing)
* service[elasticsearch] action nothing (skipped due to action :nothing)
* elasticsearch_plugin[head] action install (up to date)
Running handlers:
Running handlers complete
Chef Client finished, 0/34 resources updated in 01 seconds


Testing your Elasticsearch instance

Now that we have a working Elasticsearch node, we can run some commands to validate that it’s up and working. We’ll do some basic health checks to see if it’s responding, as well as check the health of the new node: 

root@es01:~# service elasticsearch status
* elasticsearch is running
root@es01:~# curl http://127.0.0.1:9200/
{
"name" : "node01",
"cluster_name" : "mycluster",
"version" : {
"number" : "2.0.0",
"build_hash" : "de54438d6af8f9340d50c5c786151783ce7d6be5",
"build_timestamp" : "2015-10-22T08:09:48Z",
"build_snapshot" : false,
"lucene_version" : "5.2.1"
},
"tagline" : "You Know, for Search"
}
root@es01:~# curl http://127.0.0.1:9200/_cat/health?v
epoch timestamp cluster status node.total node.data shards pri relo init unassign pending_tasks max_task_wait_time active_shards_percent
1448293963 15:52:43 mycluster green 1 1 0 0 0 0 0 0 - 100.0%


Conclusion and Next steps

I hope this post has been useful in deploying Elasticsearch with Chef. Next time, I’ll demonstrate how to build a cluster of three nodes, configure firewall rules, and install Java before Elasticsearch. In the mean time, I hope you’ll check out the new 2.0.0 version of the Elasticsearch Cookbook, as well as Elasticsearch and all of the other projects that participated in the release bonanza. If you have any questions or concerns, please leave a comment below or reach out to me using the contact links on the left-hand side of the blog. See also the original post about Chef Solo and Elasticsearch from 2012.

The original blog post has been published on the author's personal blog. Editor's note: The original post (http://martinb3.io/deploying-elasticsearch-chef-cookbook/) has been taken down since this blog was published.

martin-smith-headshot-2.jpg

Martin Smith is a DevOps Engineer at Rackspace. He is a systems administrator and software developer, with a passion for automation engineering and contributing to open source communities.