Content sources API reference
editContent sources API reference
editThis is a technical API reference. Refer to the Custom API sources for a conceptual walkthrough.
In this API reference
editContent source JSON representation
editContent sources represent the individual connections to 3rd party services, and the documents indexed from those connections. Before using the API to create, access, update, or delete content sources, it is important to be familiar with the JSON notation that will be present in many of these response bodies.
Content source fields
edit-
id
(required) -
The string that uniquely identifies the content source. This ID is generated by Workplace Search.
-
service_type
(required) -
The string representing the content source’s service type.
-
created_at
(required) -
The string in RFC-3339 format representing the date/time at which this content source was originally created.
-
last_updated_at
(required) -
The string in RFC-3339 format representing the date/time at which this content source was last updated.
-
is_remote
(required) -
The boolean for whether or not this content source is a "remote" content source. See Standard content sources vs remote content sources.
-
details
(required) -
The list of objects that provide metadata for the content source and the account which authenticated/connected it. These details will be displayed in the source details Overview page.
While this key is required, the list may be empty.
Any objects present in this list will contain the following keys:
-
title
(required) -
The string for the human readable name of this piece of metadata.
-
description
(required) -
The string value to be displayed under the corresponding title.
-
-
groups
(required) -
The list of objects which represent Workplace Search groups that have access to this content source. Each object contains two keypairs, one for
id
and one forname
. -
name
(required) -
The string for the human readable display name of this content source.
-
schema
(required) -
The object representing the schema that each document in this content source must adhere to. The keys of this object can be anything constructed of lower-case letters, numbers, and the underscore (
_
) character. These keys define the field names that may be present on indexed documents. The values of these keypairs must be one oftext
,number
,date
, orgeolocation
. -
display
(required) -
The object representing the display details which govern how documents are displayed in search results.
The
display
object contains the following keys:-
title_field
(required) -
The string field name that will be displayed as a search result’s title.
-
subtitle_field
(required) -
The string field name that will be displayed as a search result’s subtitle. If
null
, the search result will not display a subtitle. -
description_field
(required) -
The string field name that will be displayed as a search result’s long-form description or body. In the UI, this content will be truncated to fit. If
null
, the search result will not have a long-form description/body. -
url_field
(required) -
The string field name that will be used to link a given search result to its source of truth.
-
detail_fields
(required) -
The list of objects representing how the detailed view of a search result item is displayed, when clicked on.
The
detail_fields
objects each contain the following keys:-
field_name
(required) -
The string field name that this detail pertains to.
-
label
(required) -
The string for the human readable display of this field.
-
-
color
(optional) -
A string hex-color code. The
color
determines what color to fill the custom source icon with in the search UI.color
is only present for custom content sources.
-
-
context
(required) -
The string specifying whether this content source is available to groups of users, or a single user. The value can be either "organization" or "private."
-
is_searchable
(required) -
The boolean for whether or not this content source can currently be searched over on the search page.
-
document_count
(conditional) -
The integer of how many documents are currently indexed in this content source. This field is not applicable to Remote content sources.
-
last_indexed_at
(conditional) -
The string in RFC-3339 format representing the date/time when documents were last indexed into this content source. This may be null if documents have not yet been indexed. This field is not applicable to Remote content sources.
Example
editBelow is an example content source’s JSON representation:
{ "id": "603ebf0440b06a31115f4bad", "service_type": "custom", "created_at": "2021-03-02T22:41:08+00:00", "last_updated_at": "2021-03-02T22:41:16+00:00", "is_remote": false, "details": [ { "title": "Cited Source", "description": "http://nps.gov" }, { "title": "Usage", "description": "For Enterprise Search Examples" } ], "groups": [ { "id": "603ea1a440b06a9c11fd832d", "name": "Default" } ], "name": "National Parks", "schema": { "description": "text", "title": "text", "nps_link": "text", "states": "text", "visitors": "number", "square_km": "number", "world_heritage_site": "text", "date_established": "date", "location": "geolocation", "acres": "number" }, "display": { "title_field": "title", "subtitle_field": "states", "description_field": "description", "url_field": "nps_link", "detail_fields": [ { "field_name": "states", "label": "States" }, { "field_name": "description", "label": "Description" }, { "field_name": "date_established", "label": "Established" }, { "field_name": "visitors", "label": "Number of Visitors" }, { "field_name": "location", "label": "Location" }, { "field_name": "world_heritage_site", "label": "World Heritage Site?" }, { "field_name": "square_km", "label": "Square Kilometers" }, { "field_name": "acres", "label": "Acres" } ], "color": "#000000" }, "context": "organization", "is_searchable": true, "document_count": 59, "last_indexed_at": "2021-03-02T22:41:15+00:00" }
List All content sources
editRetrieve a paginated list of all content sources.
GET /api/ws/v1/sources
Request headers
edit-
Authorization
-
Bearer token permitting API access.
For more authentication methods see API Authentication Reference
Request parameters
edit-
page[current]
(optional) -
The integer page of results to retrieve. Default is
1
. -
page[size]
(optional) -
The integer size of results to retrieve in a single page. Default is
25
Request examples
editcurl \ --request 'GET' \ --url 'https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io/api/ws/v1/sources' \ --header "Authorization: Bearer $ACCESS_TOKEN"
curl \ --request 'GET' \ --url 'http://localhost:3002/api/ws/v1/sources' \ --header "Authorization: Bearer $ACCESS_TOKEN"
With pagination:
curl \ --request 'GET' \ --url 'http://localhost:3002/api/ws/v1/sources?page[current]=2&page[size]=1' \ --header "Authorization: Bearer $ACCESS_TOKEN"
Responses
edit-
200 OK
-
The paginated list of content sources is returned.
{ "meta": { "page": { "current": 1, "total_pages": 1, "total_results": 4, "size": 25 } }, "results": [ { <ContentSourceOne> }, { <ContentSourceTwo> }, { <ContentSourceThree> }, { <ContentSourceFour> } ] }
See the above content source JSON representation.
-
400 Bad Request
-
The request body was incomplete or invalid. The response body provides additional information.
-
401 Unauthorized
-
The authorization header was missing from the request or authentication failed.
-
404 Not Found
-
The authenticated user is not an Admin user, and therefore does not have permission.
Get a content source
editRetrieve a single content source by its ID.
GET /api/ws/v1/sources/<content-source-id>
Request headers
edit-
Authorization
-
Bearer token permitting API access.
For more authentication methods see API Authentication Reference
Request parameters
edit-
content-source-id
(required) -
The string ID for the content source to be retrieved.
Request examples
editcurl \ --request 'GET' \ --url "https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io/api/ws/v1/sources/$CONTENT_SOURCE_ID" \ --header "Authorization: Bearer $ACCESS_TOKEN"
curl \ --request 'GET' \ --url "http://localhost:3002/api/ws/v1/sources/$CONTENT_SOURCE_ID" \ --header "Authorization: Bearer $ACCESS_TOKEN"
Responses
edit-
200 OK
-
The single content sources is returned.
{ <ContentSource> }
See the above content source JSON representation.
-
401 Unauthorized
-
The authorization header was missing from the request or authentication failed.
-
404 Not Found
-
The authenticated user is not an Admin user, and therefore does not have permission.
OR
The provided
content_source_id
does not exist.
Create a content source
editCreate a single custom content source.
Only custom content sources can be created with this endpoint. Other service types cannot be created programmatically.
POST /api/ws/v1/sources
Request headers
edit-
Authorization
-
Bearer token permitting API access.
For more authentication methods see API Authentication Reference
-
Content-Type
-
application/json
. This endpoint accepts only JSON POST data.
Request body
edit- Content source definition (required)
-
The JSON object defining the content source to be created.
-
name
(required) -
The string for the human readable display name of this content source.
The
name
may not start or end with special characters.The
name
may not include special characters other than underscore (_
).The
name
may not be longer than 64 characters. -
schema
(optional) -
The object representing the schema that each document in this content source must adhere to. The keys of this object can be anything constructed of lower-case letters, numbers, and the underscore (
_
) character. These keys define the field names that may be present on indexed documents. The values of these keypairs must be one oftext
,number
,date
, orgeolocation
.If omitted, you cannot specify a
display
at creation time.You can update
schema
later by indexing documents (the schema will be inferred) or by using the update endpoint. -
display
(optional) -
The object representing the display details which govern how documents will be displayed in search results.
The
display
object contains the following keys:-
title_field
(required) -
The string field name that will be displayed as a search result’s title.
This field must exist in the
schema
. -
url_field
(required) -
The string field name that will be used to link a given search result to its source of truth.
This field must exist in the
schema
. -
subtitle_field
(optional) -
The string field name that will be displayed as a search result’s subtitle.
This field must exist in the
schema
.If
null
, no field will be assigned as a subtitle, and subtitles will not be displayed for any search result in this content source. -
description_field
(optional) -
The string field name that will be displayed as a search result’s long-form description or body. In the UI, this content will be truncated to fit.
This field must exist in the
schema
.If
null
, no field will be assigned as a description, and descriptions will not be displayed for any search result in this content source. -
detail_fields
(optional) -
The list of objects representing how the detailed view of a search result item should be displayed when clicked on.
The
detail_fields
objects must each contain the following keys:-
field_name
(required) -
The string field name that this detail pertains to.
This field must exist in the
schema
. -
label
(required) -
The string for the human readable display of this field.
-
-
color
(optional) -
A string hex-color code. The
color
determines what color to fill the custom source icon with in the search UI.color
must be a 7-character string, beginning with a hash (#
) followed by 6 hexadecimal digits (0-9, A-F). Other color strings or code formats will not be accepted.
-
-
is_searchable
(optional) -
The boolean for whether or not this content source should be searchable.
Example
editBelow is an example content source definition’s JSON representation:
{ "name": "National Parks", "schema": { "description": "text", "title": "text", "nps_link": "text", "states": "text", "visitors": "number", "square_km": "number", "world_heritage_site": "text", "date_established": "date", "location": "geolocation", "acres": "number" }, "display": { "title_field": "title", "subtitle_field": "states", "description_field": "description", "url_field": "nps_link", "detail_fields": [ { "field_name": "states", "label": "States" }, { "field_name": "description", "label": "Description" }, { "field_name": "date_established", "label": "Established" }, { "field_name": "visitors", "label": "Number of Visitors" }, { "field_name": "location", "label": "Location" }, { "field_name": "world_heritage_site", "label": "World Heritage Site?" }, { "field_name": "square_km", "label": "Square Kilometers" }, { "field_name": "acres", "label": "Acres" } ], "color": "#000000" }, "is_searchable": true }
Request examples
editcurl \ --request 'POST' \ --url 'https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io/api/ws/v1/sources/' \ --header 'Content-Type: application/json' \ --header "Authorization: Bearer $ACCESS_TOKEN" \ --data ' { "name": "Test", "schema": { "title": "text", "url": "text", "my_field": "text" }, "display": { "title_field": "title", "url_field": "url", "detail_fields": [ { "field_name": "my_field", "label": "My Field" } ], "color": "#111111" }, "is_searchable": true } '
curl \ --request 'POST' \ --url 'http://localhost:3002/api/ws/v1/sources/' \ --header 'Content-Type: application/json' \ --header "Authorization: Bearer $ACCESS_TOKEN" \ --data ' { "name": "Test", "schema": { "title": "text", "url": "text", "my_field": "text" }, "display": { "title_field": "title", "url_field": "url", "detail_fields": [ { "field_name": "my_field", "label": "My Field" } ], "color": "#111111" }, "is_searchable": true } '
Responses
edit-
200 OK
-
The single created content source is returned.
{ <ContentSource> }
See the above content source JSON representation.
-
400 Bad Request
-
The request body did not meet requirements. See the
errors
array to determine what may be wrong.{ "errors": [ "The property '#/schema/my_field' value \"integer\" did not match one of the following values: text, geolocation, number, date", "The property '#/display' did not contain a required property of 'title_field'", "The property '#/is_searchable' did not match any of the following types: boolean", "The property '#/name' value \"&special_chars*\" did not match the regex '(?-mix:\\A[\\w\\s.()'\\-\\/]*\\z)'" ] }
-
400 Invalid Display
-
The request body included a
display
configuration that, while syntactically correct, did not align with the content source’sschema
. This was not detected until after the content source was created, but did prevent the display settings from being applied as expected. The response includes both the current state of the new content source, and the errors in configuring the display.{ "created" : { <ContentSource> }, "errors" : [ "Invalid display setting: title_field: \"title\" is not a field on the content source", "Invalid display setting: subtitle_field: \"states\" is not a field on the content source", "Invalid display setting: description_field: \"description\" is not a field on the content source", "Invalid display setting: url_field: \"nps_link\" is not a field on the content source", "Invalid display setting: detail_fields: \"states\" is not a field on the content source, \"description\" is not a field on the content source, \"date_established\" is not a field on the content source, \"visitors\" is not a field on the content source, \"location\" is not a field on the content source, \"world_heritage_site\" is not a field on the content source, \"square_km\" is not a field on the content source, \"acres\" is not a field on the content source" ] }
See the above content source JSON representation.
-
401 Unauthorized
-
The authorization header was missing from the request or authentication failed.
-
404 Not Found
-
The authenticated user is not an Admin user, and therefore does not have permission.
Update a content source
editUpdate a single custom content source.
Only custom content sources can be updated with this endpoint. Other service types cannot be updated programmatically.
PUT /api/ws/v1/sources/<content-source-id>
Request headers
edit-
Authorization
-
Bearer token permitting API access.
For more authentication methods see API Authentication Reference
-
Content-Type
-
application/json
. This endpoint accepts only JSON PUT data.
Request parameters
edit-
content-source-id
(required) -
The string ID for the content source to be updated.
Request body
edit- Content source definition (required)
-
The JSON object defining the new state of the content source.
Unlike the
create
request body, all parameters of this body are required. You may find it useful to first do a get request, to ensure you send the complete current values to avoid unwanted changes. -
name
(required) -
The string for the human readable display name of this content source.
The
name
may not start or end with special characters.The
name
may not include special characters other than underscore (_
).The
name
may not be longer than 64 characters. -
schema
(required) -
The object representing the schema that each document in this content source must adhere to. The keys of this object can be anything constructed of lower-case letters, numbers, and the underscore (
_
) character. These keys define the field names that may be present on indexed documents. The values of these keypairs must be one oftext
,number
,date
, orgeolocation
.If the content source already contains documents, this
schema
must not attempt to change the types of fields for document that do not have an implicit type conversion available to them. For example, a document of{ "field_one": 42, "field_two": "the answer" }
a schema update of
{ "field_one": "text", "field_two": "text" }
is fine, because
42
(numeric) can implicitly be converted to"42"
(string) without issue. However, a schema update of{ "field_one": "number", "field_two": "number" }
will fail, because there is no implicit numeric value for
"the answer"
. -
display
(required) -
The object representing the display details which govern how documents will be displayed in search results.
The
display
object contains the following keys:-
title_field
(required) -
The string field name that will be displayed as a search result’s title.
This field must exist in the
schema
. -
url_field
(required) -
The string field name that will be used to link a given search result to its source of truth.
This field must exist in the
schema
. -
subtitle_field
(optional) -
The string field name that will be displayed as a search result’s subtitle.
This field must exist in the
schema
.If
null
, no field will be assigned as a subtitle, and subtitles will not be displayed for any search result in this content source. -
description_field
(optional) -
The string field name that will be displayed as a search result’s long-form description or body. In the UI, this content will be truncated to fit.
This field must exist in the
schema
.If
null
, no field will be assigned as a description, and descriptions will not be displayed for any search result in this content source. -
detail_fields
(optional) -
The list of objects representing how the detailed view of a search result item should be displayed when clicked on.
The
detail_fields
objects must each contain the following keys:-
field_name
(required) -
The string field name that this detail pertains to.
This field must exist in the
schema
. -
label
(required) -
The string for the human readable display of this field.
-
-
color
(required) -
A string hex-color code. The
color
determines what color to fill the custom source icon with in the search UI.color
must be a 7-character string, beginning with a hash (#
) followed by 6 hexadecimal digits (0-9, A-F). Other color strings or code formats will not be accepted.
-
-
is_searchable
(required) -
The boolean for whether or not this content source should be searchable.
Example
editBelow is an example content source definition’s JSON representation:
{ "name": "National Parks", "schema": { "description": "text", "title": "text", "nps_link": "text", "states": "text", "visitors": "number", "square_km": "number", "world_heritage_site": "text", "date_established": "date", "location": "geolocation", "acres": "number" }, "display": { "title_field": "title", "subtitle_field": "states", "description_field": "description", "url_field": "nps_link", "detail_fields": [ { "field_name": "states", "label": "States" }, { "field_name": "description", "label": "Description" }, { "field_name": "date_established", "label": "Established" }, { "field_name": "visitors", "label": "Number of Visitors" }, { "field_name": "location", "label": "Location" }, { "field_name": "world_heritage_site", "label": "World Heritage Site?" }, { "field_name": "square_km", "label": "Square Kilometers" }, { "field_name": "acres", "label": "Acres" } ], "color": "#000000" }, "is_searchable": true }
Request examples
editcurl \ --request 'PUT' \ --url "https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io/api/ws/v1/sources/$CONTENT_SOURCE_ID" \ --header 'Content-Type: application/json' \ --header "Authorization: Bearer $ACCESS_TOKEN" \ --data ' { "name": "Test Update", "schema": { "title": "text", "url": "text", "my_field": "text" }, "display": { "title_field": "title", "url_field": "url", "detail_fields": [ { "field_name": "my_field", "label": "My Updated Field Display" } ], "color": "#2222" }, "is_searchable": true } '
curl \ --request 'PUT' \ --url "http://localhost:3002/api/ws/v1/sources/$CONTENT_SOURCE_ID" \ --header 'Content-Type: application/json' \ --header "Authorization: Bearer $ACCESS_TOKEN" \ --data ' { "name": "Test Update", "schema": { "title": "text", "url": "text", "my_field": "text" }, "display": { "title_field": "title", "url_field": "url", "detail_fields": [ { "field_name": "my_field", "label": "My Updated Field Display" } ], "color": "#2222" }, "is_searchable": true } '
Responses
edit-
200 OK
-
The single updated content source is returned.
{ <ContentSource> }
See the above content source JSON representation.
-
400 Bad Request
-
The request body did not meet requirements. See the
errors
array to determine what may be wrong.{ "errors": [ "The property '#/schema/my_field' value \"integer\" did not match one of the following values: text, geolocation, number, date", "The property '#/display' did not contain a required property of 'title_field'", "The property '#/is_searchable' did not match any of the following types: boolean", "The property '#/name' value \"&special_chars*\" did not match the regex '(?-mix:\\A[\\w\\s.()'\\-\\/]*\\z)'" ] }
-
400 Illegal Schema Change
-
The request body included a
schema
that, while syntactically correct, would require field type changes for document values that cannot match the new type. This was not detected until after some updates may have been applied, but did prevent the schema update from being applied as expected. The response includes both the current state of the updated content source, and the errors in updating the schema.{ "updated" : { <ContentSource> }, "errors": [ "Field 'description' cannot be changed to type 'float'. Example IDs that would fail: [\"park_acadia\", \"park_badlands\", \"park_big-bend\", \"park_bryce-canyon\", \"park_canyonlands\", \"park_capitol-reef\", \"park_carlsbad-caverns\", \"park_congaree\", \"park_cuyahoga-valley\", \"park_denali\"]" ] }
See the above content source JSON representation.
-
400 Invalid Display
-
The request body included a
display
configuration that, while syntactically correct, did not align with the content source’sschema
. This was not detected until after some updates may have been applied, but did prevent the display settings from being applied as expected. The response includes both the current state of the updated content source, and the errors in configuring the display.{ "updated" : { <ContentSource> }, "errors" : [ "Invalid display setting: title_field: \"title\" is not a field on the content source", "Invalid display setting: subtitle_field: \"states\" is not a field on the content source", "Invalid display setting: description_field: \"description\" is not a field on the content source", "Invalid display setting: url_field: \"nps_link\" is not a field on the content source", "Invalid display setting: detail_fields: \"states\" is not a field on the content source, \"description\" is not a field on the content source, \"date_established\" is not a field on the content source, \"visitors\" is not a field on the content source, \"location\" is not a field on the content source, \"world_heritage_site\" is not a field on the content source, \"square_km\" is not a field on the content source, \"acres\" is not a field on the content source" ] }
See the above content source JSON representation.
-
401 Unauthorized
-
The authorization header was missing from the request or authentication failed.
-
404 Not Found
-
The specified content source ID does not exist
OR
The authenticated user is not an Admin user, and therefore does not have permission.
Remove a content source
editRemove a single content source by its ID.
Only organization content sources can be deleted with this endpoint. Private content sources cannot be deleted with this endpoint.
DELETE /api/ws/v1/sources/<content-source-id>
Request headers
edit-
Authorization
-
Bearer token permitting API access.
For more authentication methods see API Authentication Reference
Request parameters
edit-
content-source-id
(required) -
The string ID for the content source to be removed.
Request examples
editcurl \ --request 'DELETE' \ --url "https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io/api/ws/v1/sources/$CONTENT_SOURCE_ID" \ --header "Authorization: Bearer $ACCESS_TOKEN"
curl \ --request 'DELETE' \ --url "http://localhost:3002/api/ws/v1/sources/$CONTENT_SOURCE_ID" \ --header "Authorization: Bearer $ACCESS_TOKEN"
Responses
edit-
200 OK
-
An acknowledgement that the content source was deleted
{ "deleted" : true }
-
400 Bad Request
-
The provided
content_source_id
is for a private content source, which is not allowed.{ "errors": ["Invalid parameter(s): Private content sources cannot be deleted by this API"] }
-
401 Unauthorized
-
The authorization header was missing from the request or authentication failed.
-
404 Not Found
-
The authenticated user is not an Admin user, and therefore does not have permission.
OR
The provided
content_source_id
does not exist.
Get a document
editRetrieve a single content source document, given the IDs of both the content source and the document.
GET /api/ws/v1/sources/<content-source-id>/documents/<document-id>
Request headers
edit-
Authorization
-
Bearer token permitting API access.
For more authentication methods see API Authentication Reference
Request parameters
edit-
content-source-id
(required) -
The string ID for the content source that contains the desired document.
-
document-id
(required) -
The string ID for the document to be retrieved.
Request examples
editcurl \ --request 'GET' \ --url "https://f3fae9ab3e4f423d9d345979331ef3a1.ent-search.us-east-1.aws.cloud.es.io/api/ws/v1/sources/$CONTENT_SOURCE_ID/documents/$DOCUMENT_ID" \ --header "Authorization: Bearer $ACCESS_TOKEN"
curl \ --request 'GET' \ --url "http://localhost:3002/api/ws/v1/sources/$CONTENT_SOURCE_ID/documents/$DOCUMENT_ID" \ --header "Authorization: Bearer $ACCESS_TOKEN"
Responses
edit-
200 OK
-
The single content source document is returned.
The only guaranteed response fields are
id
,source
,content_source_id
, andlast_updated
. The others are content source specific, and must be part of the content source’s schema. You can view a content source’s schema with the get content source API.{ "id": "some_string_id", "source": "custom", "content_source_id": "603d2c8319cbdbc907675c02", "last_updated": "2021-03-02T22:41:16+00:00", "some_text_field": "some text value", "some_number_field": 42, "some_date_field": "2021-03-02T22:41:16+00:00", "some_geolocation_field": "44.35,-68.21", "_allow_permissions": ["perm1", "perm2"], "_deny_permissions": ["perm3", "perm4"] }
-
401 Unauthorized
-
The authorization header was missing from the request or authentication failed.
-
404 Not Found
-
The authenticated user is not an Admin user, and therefore does not have permission.
OR
The authenticated user has specified a private content source that is not theirs, and they therefore do not have permission.
OR
The specified document has document-level-permissions which prohibit access to the authenticated user.
OR
The provided
content_source_id
does not exist.OR
The provided
document_id
does not exist.