Search UI with Elasticsearch

This tutorial will guide you through the process of creating a Search UI with Elasticsearch directly, using the elasticsearch-connector. We will be using a sample movie data-set of around 1000 movies.

Within this tutorial, we assume that you have Node.js installed on your machine.

First we need to setup Elasticsearch. The easiest way to do this is to create an Elasticsearch instance via Elastic Cloud.

We are going to issue commands via Kibana’s dev tools console. You can alternatively use a REST client like Postman to achieve this.

First we need to create an index for our data. We can do this simply via the following request:

PUT /my-example-movies

Elasticsearch will acknowledge our request in the response.

Next we need to setup an API key to access the data from the index. We can do this via Kibana’s Stack Management API Keys page (<your Kibana endpoint>/app/management/security/api_keys). Note that security needs to be enabled for this option to be available.

Notice here we are only giving read privileges for this api key. You will need to setup an api key with write privileges to add and update data to the index.

{
  "superuser": {
    "cluster": ["all"],
    "indices": [
      {
        "names": ["my-example-movies"],
        "privileges": ["read"],
        "allow_restricted_indices": false
      }
    ]
  }
}

Once saved, you are presented with the api-key. Copy this and keep it safe. We will need to use this further down in the tutorial.

If you’re going to be accessing Elasticsearch directly from a browser and the Elasticsearch host domain doesn’t match your site’s domain, you will need to enable CORS.

CORS is a browser mechanism which enables controlled access to resources located outside of the current domain. In order for the browser to make requests to Elasticsearch, CORS configuration headers need to specified in the Elasticsearch configuration.

You can do this in cloud by going to the deployment settings for your Elasticsearch instance, click "Edit user settings and plugins" and under "user settings", add the CORS configuration below:

http.cors.allow-origin: "*"
http.cors.enabled: true
http.cors.allow-credentials: true
http.cors.allow-methods: OPTIONS, HEAD, GET, POST, PUT, DELETE
http.cors.allow-headers: X-Requested-With, X-Auth-Token, Content-Type, Content-Length, Authorization, Access-Control-Allow-Headers, Accept, x-elastic-client-meta

then save. Your Elasticsearch instance will be restarted and the CORS configuration will be active.

Next we need to setup the index fields, ready for us to ingest data.

The mapping for an index depends on the data you want to index and the features you want.

We want to be able to search on title. We need only one field of type text.

{
  "properties": {
    "title": {
      "type": "text"
    }
  }
}

We want to be able to search and product facets for writers field. We need two fields of different types: keyword and text.

{
  "properties": {
    "writers": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword"
        }
      }
    }
  }
}

We want to be able to filter on a date field. We only need one date field.

{
  "properties": {
    "released": {
      "type": "date"
    }
  }
}

We want to be able to filter on a numeric field. We only need one numeric field. Can be a choice of integer, float and more documented here

{
  "properties": {
    "imdbRating": {
      "type": "float"
    }
  }
}

For our movie data-set, we will be using the following fields:

The mapping file will be as follows, and we’ll once again use Kibana’s dev tools console to update the mapping file for our index.

PUT /my-example-movies/_mapping
{
  "properties": {
    "title": {
      "type": "text",
      "fields": {
        "suggest": {
          "type": "search_as_you_type"
        }
      }
    },
    "plot": {
      "type": "text"
    },
    "genre": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword"
        }
      }
    },
    "actors": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword"
        }
      }
    },
    "directors": {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword"
        }
      }
    },
    "released": {
      "type": "date"
    },
    "imdbRating": {
      "type": "float"
    },
    "url": {
      "type": "keyword"
    },
    "movie_completion": {
      "type": "completion"
    }
  }
}

Elasticsearch will acknowledge the request in the response.

We also want to provide autocomplete functionality, so we need to setup fields for autocomplete.

For suggestions, we want to suggest terms that appear within the actors, directors and genre fields. For quick result hits, we want to suggest movies that partially match the title field.

In the above example:

Now with our index and mapping file created, we are ready to index some data! We will use the bulk API to index our data.

We will use the following request. In this example we will be indexing the first movie in the data-set to verify that the data fields is being indexed correctly.

PUT /my-example-movies/_bulk
{ "index": {}}
{"title": "The Godfather", "released": "1972-03-23T23:00:00.000Z","genre": ["Crime", "Drama"],"directors": ["Francis Ford Coppola"],"actors": ["Marlon Brando", "Al Pacino", "James Caan", "Richard S. Castellano"],"plot": "The aging patriarch of an organized crime dynasty transfers control of his clandestine empire to his reluctant son.","imdbRating": "9.2", "movie_completion": ["Crime", "Drama", "Marlon Brando", "Al Pacino", "James Caan", "Richard S. Castellano"], "url": "https://www.imdb.com/title/tt0068646/"}

First, download the Search-UI’s starter app from github by

curl https://codeload.github.com/elastic/app-search-reference-ui-react/tar.gz/master | tar -xz

and should appear as a folder called app-search-reference-ui-react-main.

Navigate to the root to the folder and install the dependencies using the following command:

yarn

Within the folder, we can now install the @elastic/search-ui-elasticsearch-connector library with Yarn.

yarn add @elastic/search-ui-elasticsearch-connector

Make sure to check and update Search UI dependencies to the latest version. You can find the latest version by going to NPM’s page for @elastic/search-ui.

Open the project within your favorite editor.

Within src/App.js, change line 3 to import the Elasticsearch connector. You no longer need the app-search connector.

import ElasticsearchAPIConnector from "@elastic/search-ui-elasticsearch-connector";

and then update the options to the connector

const connector = new ElasticsearchAPIConnector({
  cloud: {
    id: "<my-elastic-cloud-id>"
  },
  apiKey: "<api-key>",
  index: "my-example-movies"
});

If you’re using Elastic Cloud, you can find your cloud id within your deployment’s details.

alternatively, if you’re using an on-premise Elasticsearch instance, you can connect via specifying the host.

const connector = new ElasticsearchAPIConnector({
  host: "http://localhost:9200",
  index: "my-example-movies"
});

Next lets configure Search UI for our needs! Navigate to the config within app.js and update the following:

const config = {
  searchQuery: {
    search_fields: {
      title: {
        weight: 3
      },
      plot: {},
      genre: {},
      actors: {},
      directors: {}
    },
    result_fields: {
      title: {
        snippet: {}
      },
      plot: {
        snippet: {}
      }
    },
    disjunctiveFacets: ["genre.keyword", "actors.keyword", "directors.keyword"],
    facets: {
      "genre.keyword": { type: "value" },
      "actors.keyword": { type: "value" },
      "directors.keyword": { type: "value" },
      released: {
        type: "range",
        ranges: [
          {
            from: "2012-04-07T14:40:04.821Z",
            name: "Within the last 10 years"
          },
          {
            from: "1962-04-07T14:40:04.821Z",
            to: "2012-04-07T14:40:04.821Z",
            name: "10 - 50 years ago"
          },
          {
            to: "1962-04-07T14:40:04.821Z",
            name: "More than 50 years ago"
          }
        ]
      },
      imdbRating: {
        type: "range",
        ranges: [
          { from: 1, to: 3, name: "Pants" },
          { from: 3, to: 6, name: "Mediocre" },
          { from: 6, to: 8, name: "Pretty Good" },
          { from: 8, to: 10, name: "Excellent" }
        ]
      }
    }
  },
  autocompleteQuery: {
    results: {
      resultsPerPage: 5,
      search_fields: {
        "title.suggest": {
          weight: 3
        }
      },
      result_fields: {
        title: {
          snippet: {
            size: 100,
            fallback: true
          }
        },
        url: {
          raw: {}
        }
      }
    },
    suggestions: {
      types: {
        results: { fields: ["movie_completion"] }
      },
      size: 4
    }
  },
  apiConnector: connector,
  alwaysSearchOnInitialLoad: true
};

In the above example, we configured the:

For more information on configuration, visit the API configuration docs.

We are going to do several steps here:

<div className="App">
  <ErrorBoundary>
    <Layout
      header={
        <SearchBox
          autocompleteMinimumCharacters={3}
          autocompleteResults={{
            linkTarget: "_blank",
            sectionTitle: "Results",
            titleField: "title",
            urlField: "url",
            shouldTrackClickThrough: true
          }}
          autocompleteSuggestions={true}
          debounceLength={0}
        />
      }
      sideContent={
        <div>
          {wasSearched && <Sorting label={"Sort by"} sortOptions={[]} />}
          <Facet key={"1"} field={"genre.keyword"} label={"genre"} />
          <Facet key={"2"} field={"actors.keyword"} label={"actors"} />
          <Facet key={"3"} field={"directors.keyword"} label={"directors"} />
          <Facet key={"4"} field={"released"} label={"released"} />
          <Facet key={"4"} field={"imdbRating"} label={"imdb rating"} />
        </div>
      }
      bodyContent={<Results shouldTrackClickThrough={true} />}
      bodyHeader={
        <React.Fragment>
          {wasSearched && <PagingInfo />}
          {wasSearched && <ResultsPerPage />}
        </React.Fragment>
      }
      bodyFooter={<Paging />}
    />
  </ErrorBoundary>
</div>

Lets run the project with the command:

yarn start

and then view the results in the browser at http://localhost:3000/

Lets recap of the steps we have covered:

Next you can add more data into the index, update the results view to display more fields, and deploy the app.