Running site plugins with Elasticsearch 5.0

Way back in Elasticsearch 0.17, Elasticsearch gained the ability to serve static web pages, and site plugins were born. Site plugins allowed users to write Javascript applications which provide graphical user interfaces to Elasticsearch.

This spawned a flurry of development by Elasticsearch users producing plugins for monitoring Elasticsearch stats, for index management, segment merging visualisations, analyser debugging, clustering, and more.

The two most popular site plugins today are Head and Kopf, both of which combine monitoring with index management.

All good so far. Now for the bad news…

Site plugins are not supported in Elasticsearch 5.0

Why are we removing this popular feature from Elasticsearch?

Elasticsearch is not designed to be a web server. Serving static files was just a hack that was easy to add on top of the HTTP REST interface that Elasticsearch does provide. What’s the harm in serving static files?

Well, it turns out that just serving static files can be harmful. Two out of the seven security vulnerabilities ever discovered in Elasticsearch had to do with site plugins. That’s significant, especially for a non-essential feature. In contrast, two of the other vulnerabilities were due to dynamic scripting, and we wrote a whole new scripting language to solve that problem!

Elasticsearch now runs under the Java Security Manager. We have locked down the privileges that Elasticsearch core requires to run to the bare minimum. We are moving functionality out of core and into modules to further restrict privilege escalation and file access to the smallest chunk of code possible. We do all of this with the aim of restricting the exploit possibilities open to any hacker who finds a zero day vulnerability.

Running a web server for a non-essential feature is not consistent with this goal.

On top of that, hosting web applications on Elasticsearch encourages the bad practice of exposing Elasticsearch to the Internet, while it should be running in an isolated, more secure network.

Running site plugins with Elasticsearch 5.0

While many popular site plugins like Head, Kopf (soon to be replaced by Cerebro), and HQ already include instructions for running outside of Elasticsearch, it’s easy to do on your own as well. A site plugin consists of static HTML, Javascript, CSS, and image files, which can be served by the web server of your choice, even (for local use) Python’s SimpleHTTPServer:

cd my_plugin/
python -m SimpleHTTPServer 8000

Because site plugins make requests directly to Elasticsearch from the user’s browser, a little configuration is required to instruct Elasticsearch to allow Cross Origin Resource Sharing.

For example, you could add the following to the elasticsearch.yml config file:

http.cors.enabled: true
http.cors.allow-origin: /https?:\/\/localhost(:[0-9]+)?/

The allow-origin setting should be a regular expression that matches the address of the web server hosting the site plugin. You can read more about the CORS settings in the HTTP module documentation.

If your Elasticsearch server is using the Security feature in X-Pack (the replacement for Shield in 5.0), then you will also need to add the following settings:

http.cors.allow-credentials: true
http.cors.allow-headers: X-Requested-With,X-Auth-Token,Content-Type, Content-Length, Authorization

Beyond site plugins

If you are a plugin author and would like to move beyond the restrictions of writing an application with static web files then consider developing a Kibana plugin instead.

Kibana comes with a Node.js web server and plugins can include server-side functionality. For instance, all calls to Elasticsearch could be made directly from Kibana’s backend server instead of from the user’s browser.

There is even a Kibana plugin generator to help you get started with your own plugin, and a video from Elastic{ON}16 describing the process. As always, helpful Kibana devs are on IRC and in the forum to help with any questions.