IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

› ›

API logs API

edit

API logs API

edit

Displays API request and response data at the Engine level.

Account-level operations, such as requests to Credentials, do not appear...

... What does appear? Everything else!

API Endpoint	Logged?
`engines`	Yes
`documents`	Yes
`search`	Yes
`synonyms`	Yes
`clickthrough`	Yes
`analytics`	Yes
`curations`	Yes
`logs/api`	Yes
`credentials`	No

Every authenticated GET, PUT, POST or DELETE received during a search query, analytics request, document creation - any Engine event of any kind - will be recorded within the API Log.

Even requests against the API Log itself are recorded within the API Log.

API Log data is robust. The examples each show one result in full, for clarity.

Log retention

edit

Since 7.9.0, App Search retains API event data according to your Enterprise Search log retention settings.

See Log retention in the Enterprise Search documentation.

Log settings

edit

Since 7.9.0, you can disable API event logging using the App Search log settings.

See Log settings guide.

Authentication

edit

For authentication, the API Logs endpoint requires...

The name of your Engine: [ENGINE]
A Private API Key: [PRIVATE_API_KEY]

curl -X GET 'https://[instance id].ent-search.[region].[provider].cloud.es.io/api/as/v1/engines/[ENGINE]/logs/api' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer [PRIVATE_API_KEY]'

Response Object

edit

By default, you will receive 10 results per page in ascending order.

A result will include:

Time-stamp
HTTP method
Full request path
Status code
Request body and response
User Agent queried

// An example JSON payload from the logs/api endpoint.
{
  "results": [
    {
      "timestamp": string,
      "http_method": string,
      "path": string,
      "full_request_path": string,
      "status": number,
      "request_body": string,
      "response_body": string,
      "user_agent": string
    }
  ],
  "meta": {
    "query": string,
    "filters": {
      "date": {
        "from": string,
        "to": string
      }
    },
    "page": {
      "current": number,
      "total_pages": number,
      "total_results": number,
      "size": number
    }
  }
}

Searching the API Log

edit

GET /api/as/v1/engines/[ENGINE]/logs/api

POST /api/as/v1/engines/[ENGINE]/logs/api

query (optional): You can search over the full_request_path of an API Log event. Use this to specify a particular endpoint, like analytics, search, curations and so on.
sort_direction (optional): Would you like to have your results ascending, oldest to newest, or descending, newest to oldest? The key accepts asc or desc. Defaults to ascending.
page (optional): Can provide... current for current page total_pages for the net number of pages total_results for the overall number of results size for the amount of results per page. Expects integers within all fields.

Example - A GET request for 5 pages of search events between October 15th and 16th with 20 results on each page, in descending order.

curl -X GET 'https://[instance id].ent-search.[region].[provider].cloud.es.io/api/as/v1/engines/national-parks-demo/logs/api' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer private-xxxxxxxxxxxxxxxxxxxx' \
-d '{
  "filters": {
    "date": {
      "from": "2018-10-15T00:00:00+00:00",
      "to": "2018-10-16T00:00:00+00:00"
    }
  },
  "page": {
    "total_results": 100,
    "size": 20
  },
  "query": "search",
  "sort_direction": "desc"
}'

Example Response

{
  "results": [
    {
      "timestamp": "2018-10-15T20:42:14Z",
      "http_method": "POST",
      "path": "/api/as/v1/engines/national-parks-demo/search",
      "full_request_path": "/api/as/v1/engines/national-parks-demo/search",
      "status": 200,
      "request_body": "{\n  \"query\": \"everglade\"\n}",
      "response_body": "{\"meta\":{\"warnings\":[],\"page\":{\"current\":1,\"total_pages\":1,\"total_results\":3,\"size\":10},\"request_id\":\"164c8262b4a82039def310f8139f68d2\"},\"results\":[{\"nps_link\":{\"raw\":\"https://www.nps.gov/ever/index.htm\"},\"title\":{\"raw\":\"Everglades\"},\"date_established\":{\"raw\":\"1934-05-30T05:00:00+00:00\"},\"world_heritage_site\":{\"raw\":\"true\"},\"states\":{\"raw\":[\"Florida\"]},\"description\":{\"raw\":\"The Everglades are the largest tropical wilderness in the United States. This mangrove and tropical rainforest ecosystem and marine estuary is home to 36 protected species, including the Florida panther, American crocodile, and West Indian manatee. Some areas have been drained and developed; restoration projects aim to restore the ecology.\"},\"visitors\":{\"raw\":930907.0},\"id\":{\"raw\":\"park_everglades\"},\"location\":{\"raw\":\"25.32,-80.93\"},\"square_km\":{\"raw\":6106.6},\"acres\":{\"raw\":1508968.1},\"_meta\":{\"score\":1.0028129}},{\"nps_link\":{\"raw\":\"https://www.nps.gov/voya/index.htm\"},\"title\":{\"raw\":\"Voyageurs\"},\"date_established\":{\"raw\":\"1971-01-08T06:00:00+00:00\"},\"world_heritage_site\":{\"raw\":\"false\"},\"states\":{\"raw\":[\"Minnesota\"]},\"description\":{\"raw\":\"This park protecting four lakes near the Canada\\u2013US border is a site for canoeing, kayaking, and fishing. The park also preserves a history populated by Ojibwe Native Americans, French fur traders called voyageurs, and gold miners. Formed by glaciers, the region features tall bluffs, rock gardens, islands, bays, and several historic buildings.\"},\"visitors\":{\"raw\":241912.0},\"id\":{\"raw\":\"park_voyageurs\"},\"location\":{\"raw\":\"48.5,-92.88\"},\"square_km\":{\"raw\":883.0},\"acres\":{\"raw\":218200.15},\"_meta\":{\"score\":0.0062864386}},{\"nps_link\":{\"raw\":\"https://www.nps.gov/wrst/index.htm\"},\"title\":{\"raw\":\"Wrangell\\u2013St. Elias\"},\"date_established\":{\"raw\":\"1980-12-02T06:00:00+00:00\"},\"world_heritage_site\":{\"raw\":\"true\"},\"states\":{\"raw\":[\"Alaska\"]},\"description\":{\"raw\":\"An over 8 million acres (32,375 km2) plot of mountainous country\\u2014the largest National Park in the system\\u2014protects the convergence of the Alaska, Chugach, and Wrangell-Saint Elias Ranges, which include many of the continent's tallest mountains and volcanoes, including the 18,008-foot Mount Saint Elias. More than a quarter of the park is covered with glaciers, including the tidewater Hubbard Glacier, piedmont Malaspina Glacier, and valley Nabesna Glacier.\"},\"visitors\":{\"raw\":79047.0},\"id\":{\"raw\":\"park_wrangell\\u2013st.-elias\"},\"location\":{\"raw\":\"61,-142\"},\"square_km\":{\"raw\":33682.6},\"acres\":{\"raw\":8323146.48},\"_meta\":{\"score\":0.00437395}}]}",
      "user_agent": "curl/7.54.0"
    },

    ## ... Truncated!

  ],
  "meta": {
    "query": "search",
    "filters": {
      "date": {
        "from": "2018-10-15T00:00:00+00:00",
        "to": "2018-10-16T00:00:00+00:00"
      }
    },
    "sort_direction": "desc",
    "page": {
      "current": 1,
      "total_pages": 1,
      "total_results": 9,
      "size": 20
    }
  }
}

Filtering the API Log

edit

Filter the API Log.

A time-frame must be provided.

You can combine all parameters within the filters object.

GET /api/as/v1/engines/[ENGINE]/logs/api

POST /api/as/v1/engines/[ENGINE]/logs/api

filters/date (required): Requires a nested date object under filters containing from and to keys. The keys must contain dates formatted in accordance with RFC3339. You may omit the time: YYYY-MM-DD.
filters/status (optional): Filter based on a particular status code: 400, 401, 403, 429, 200.
filters/method (optional): Filter based on a particular HTTP method: GET, POST, PUT, PATCH, DELETE.

Date Filter

edit

Example - A GET request to receive all API Log events between October 15th and 16th.

curl -X GET 'https://[instance id].ent-search.[region].[provider].cloud.es.io/api/as/v1/engines/national-parks-demo/logs/api' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer private-xxxxxxxxxxxxxxxxxxxx' \
-d '{
  "filters": {
    "date": {
      "from": "2018-10-15T00:00:00+00:00",
      "to": "2018-10-16T00:00:00+00:00"
    }
  }
}'

Example Response

{
  "results": [
    {
      "timestamp": "2018-10-15T17:01:33Z",
      "http_method": "GET",
      "path": "/api/as/v1/engines/national-parks-demo/search",
      "full_request_path": "/api/as/v1/engines/national-parks-demo/search",
      "status": 200,
      "request_body": "{\"result_fields\": {\"states\": {\"raw\": {}}, \"location\": {\"raw\": {}}, \"title\": {\"raw\": {}}}, \"query\": \"california\", \"group\": {\"field\": \"states\"}}",
      "response_body": "{\"meta\":{\"warnings\":[],\"page\":{\"current\":1,\"total_pages\":1,\"total_results\":2,\"size\":10},\"request_id\":\"700de28b9ae89e05e7114353eed7fbee\"},\"results\":[{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Channel Islands\"},\"location\":{\"raw\":\"34.01,-119.42\"},\"id\":{\"raw\":\"park_channel-islands\"},\"_meta\":{\"score\":5.2469006},\"_group\":[{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Joshua Tree\"},\"location\":{\"raw\":\"33.79,-115.9\"},\"id\":{\"raw\":\"park_joshua-tree\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Kings Canyon\"},\"location\":{\"raw\":\"36.8,-118.55\"},\"id\":{\"raw\":\"park_kings-canyon\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Lassen Volcanic\"},\"location\":{\"raw\":\"40.49,-121.51\"},\"id\":{\"raw\":\"park_lassen-volcanic\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Pinnacles\"},\"location\":{\"raw\":\"36.48,-121.16\"},\"id\":{\"raw\":\"park_pinnacles\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Redwood\"},\"location\":{\"raw\":\"41.3,-124\"},\"id\":{\"raw\":\"park_redwood\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Sequoia\"},\"location\":{\"raw\":\"36.43,-118.68\"},\"id\":{\"raw\":\"park_sequoia\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\"]},\"title\":{\"raw\":\"Yosemite\"},\"location\":{\"raw\":\"37.83,-119.5\"},\"id\":{\"raw\":\"park_yosemite\"},\"_meta\":{\"score\":5.2469006}},{\"states\":{\"raw\":[\"California\",\"Nevada\"]},\"title\":{\"raw\":\"Death Valley\"},\"location\":{\"raw\":\"36.24,-116.82\"},\"id\":{\"raw\":\"park_death-valley\"},\"_meta\":{\"score\":3.3266773}}],\"_group_key\":\"California\"},{\"states\":{\"raw\":[\"California\",\"Nevada\"]},\"title\":{\"raw\":\"Death Valley\"},\"location\":{\"raw\":\"36.24,-116.82\"},\"id\":{\"raw\":\"park_death-valley\"},\"_meta\":{\"score\":3.3266773},\"_group\":[],\"_group_key\":\"Nevada\"}]}",
      "user_agent": "python-requests/2.19.1"
    },

    ## ... Truncated!

  ],
  "meta": {
    "query": "",
    "filters": {
      "date": {
        "from": "2018-10-15T00:00:00+00:00",
        "to": "2018-10-16T00:00:00+00:00"
      }
    },
    "sort_direction": "asc",
    "page": {
      "current": 1,
      "total_pages": 2,
      "total_results": 20,
      "size": 10
    }
  }
}

Status Filter

edit

Example - A GET request to receive all API Log events between February 1st and 5th with a status code of 400.

curl -X GET 'https://[instance id].ent-search.[region].[provider].cloud.es.io/api/as/v1/engines/national-parks-demo/logs/api' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer private-xxxxxxxxxxxxxxxxxxxx' \
-d '{
  "filters": {
    "date": {
      "from": "2019-02-01T00:00:00+00:00",
      "to": "2019-02-05T00:00:00+00:00"
    },
    "status": "400"
  }
}'

Example Response

{
  "results": [
    {
      "timestamp": "2019-02-01T16:06:06Z",
      "http_method": "POST",
      "path": "/api/as/v1/engines/national-parks-demo/query_suggestion",
      "full_request_path": "/api/as/v1/engines/national-parks-demo/query_suggestion",
      "status": 400,
      "request_body": "{\"query\":\"z\",\"types\":{\"documents\":{\"fields\":[\"title\",\"state\"]}}}",
      "response_body": "{\"errors\":[\"Types documents fields contains invalid value: state\"]}",
      "user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0.2 Safari/605.1.15"
    },

    # ... Truncated!

  ],
  "meta": {
    "query": "",
    "filters": {
      "date": {
        "from": "2019-02-01T00:00:00+00:00",
        "to": "2019-02-05T00:00:00+00:00"
      },
      "status": "400"
    },
    "sort_direction": "asc",
    "page": {
      "current": 1,
      "total_pages": 3,
      "total_results": 27,
      "size": 10
    }
  }
}

Method Filter

edit

Example - A GET request to receive all API Log events between February 1st and 5th with a method of POST.

curl -X GET 'https://[instance id].ent-search.[region].[provider].cloud.es.io/api/as/v1/engines/national-parks-demo/logs/api' \
-H 'Content-Type: application/json' \
-H 'Authorization: Bearer private-xxxxxxxxxxxxxxxxxxxx' \
-d '{
  "filters": {
    "date": {
      "from": "2019-02-01T00:00:00+00:00",
      "to": "2019-02-05T00:00:00+00:00"
    },
    "method": "POST"
  }
}'

Example Response

{
  "results": [
    {
      "timestamp": "2019-02-01T01:13:04Z",
      "http_method": "POST",
      "path": "/api/as/v1/engines/national-parks-demo/search.json",
      "full_request_path": "/api/as/v1/engines/national-parks-demo/search.json",
      "status": 200,
      "request_body": "{\"query\":\"example\",\"facets\":{\"states\":{\"type\":\"value\",\"size\":10},\"world_heritage_site\":{\"type\":\"value\",\"size\":10}},\"result_fields\":{\"description\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"nps_link\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"location\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"states\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"title\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"visitors\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"world_heritage_site\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"acres\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"square_km\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"date_established\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}},\"id\":{\"raw\":{},\"snippet\":{\"size\":100,\"fallback\":true}}},\"search_fields\":{\"description\":{},\"nps_link\":{},\"location\":{},\"states\":{},\"title\":{},\"visitors\":{},\"world_heritage_site\":{},\"acres\":{},\"square_km\":{},\"date_established\":{},\"id\":{}},\"page\":{\"current\":1,\"size\":20},\"filters\":{\"all\":[]}}",

    # ... Truncated!

  ],
  "meta": {
    "query": "",
    "filters": {
      "date": {
        "from": "2019-02-01T00:00:00+00:00",
        "to": "2019-02-05T00:00:00+00:00"
      },
      "method": "POST"
    },
    "sort_direction": "asc",
    "page": {
      "current": 1,
      "total_pages": 18,
      "total_results": 173,
      "size": 10
    }
  }
}

What’s Next?

edit

Your API Log is a well-spring of valuable Engine-level insight. If you want to start looking deeper into how visitors run queries against your Engines, the Analytics and Clickthrough endpoints are worth exploring. If you want to start improving the relevance of your queries, Synonyms are an excellent place to start.

« Analytics queries API Click API »