Ingest data through API

[preview] This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

The Elasticsearch APIs enable you to ingest data through code. You can use the APIs of one of the language clients or the Elasticsearch HTTP APIs. The examples on this page use the HTTP APIs to demonstrate how ingesting works in Elasticsearch through APIs. If you want to ingest timestamped data or have a more complex ingestion use case, check out Beats or Logstash.

You can index multiple JSON documents to an index and make it searchable using the bulk API.

The following example uses the bulk API to ingest book-related data into an index called books. The API call creates the index if it doesn’t exist already.

curl -X POST "${ES_URL}/_bulk?pretty" \
-H "Authorization: ApiKey ${API_KEY}" \
-H "Content-Type: application/json" \
-d'
{ "index" : { "_index" : "books" } }
{"title": "Snow Crash", "author": "Neal Stephenson", "release_date": "1992-06-01", "page_count": 470}
{ "index" : { "_index" : "books" } }
{"title": "Revelation Space", "author": "Alastair Reynolds", "release_date": "2000-03-15", "page_count": 585}
{ "index" : { "_index" : "books" } }
{"title": "1984", "author": "George Orwell", "release_date": "1985-06-01", "page_count": 328}
{ "index" : { "_index" : "books" } }
{"title": "Fahrenheit 451", "author": "Ray Bradbury", "release_date": "1953-10-15", "page_count": 227}
{ "index" : { "_index" : "books" } }
{"title": "Brave New World", "author": "Aldous Huxley", "release_date": "1932-06-01", "page_count": 268}
{ "index" : { "_index" : "books" } }
{"title": "The Blind Assassin", "author": "Margaret Atwood", "release_date": "2000-09-02", "page_count": 536}
'

The API returns a response similar to this:

{
  "errors": false,
  "took": 902,
  "items": [
    {
      "index": {
        "_index": "books",
        "_id": "MCYbQooByucZ6Gimx2BL",
        "_version": 1,
        "result": "created",
        "_shards": {
          "total": 1,
          "successful": 1,
          "failed": 0
        },
        "_seq_no": 0,
        "_primary_term": 1,
        "status": 201
      }
    },
    ...
  ]
}

The API call creates an index called books and adds six documents to it. All those documents have the title, author, release_date, and page_count fields with associated values. This data is now searchable.

You can check if a book is in the index by calling the search API and specifying either of the properties of the book in a match query, for example:

curl "${ES_URL}/books/_search?pretty" \
-H "Authorization: ApiKey ${API_KEY}" \
-H "Content-Type: application/json" \
-d'
{
  "query": {
    "match": {
      "title": "Snow Crash"
    }
  }
}
'

The API response contains an array of hits. Each hit represents a document that matches the query. The response contains the whole document. Only one document matches this query.

Use the index API to ingest a single document to an index. Following the previous example, a new document will be added to the books index.

curl -X POST "${ES_URL}/books/_doc/" \
-H "Authorization: ApiKey ${API_KEY}" \
-H "Content-Type: application/json" \
-d'
{
  "title": "Neuromancer",
  "author": "William Gibson",
  "release_date": "1984-07-01",
  "page_count": "271"
}
'

The API call indexes the new document into the books index. Now you can search for it!