IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

« Exists API Flush API »

› › ›

Field usage stats API

edit

Field usage stats API

edit

This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

Returns field usage information for each shard and field of an index. Field usage statistics are automatically captured when queries are running on a cluster. A shard-level search request that accesses a given field, even if multiple times during that request, is counted as a single use.

response = client.indices.field_usage_stats(
  index: 'my-index-000001'
)
puts response

GET /my-index-000001/_field_usage_stats

Request

edit

GET /<index>/_field_usage_stats

Prerequisites

edit

If the Elasticsearch security features are enabled, you must have the manage index privilege for the target index or index alias.

Path parameters

edit

<index>: (Optional, string) Comma-separated list or wildcard expression of index names used to limit the request.

Query parameters

edit

allow_no_indices

(Optional, Boolean) If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

expand_wildcards

(Optional, string) Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are:

all: Match any data stream or index, including hidden ones.
open: Match open, non-hidden indices. Also matches any non-hidden data stream.
closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
none: Wildcard patterns are not accepted.

ignore_unavailable

(Optional, Boolean) If false, the request returns an error if it targets a missing or closed index. Defaults to false.

wait_for_active_shards

(Optional, string) The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). Default: 1, the primary shard.

See Active shards.

master_timeout

(Optional, time units) Period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. Defaults to 30s. Can also be set to -1 to indicate that the request should never timeout.

timeout

(Optional, time units) Period to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. Defaults to 30s.

fields

(Optional, string) Comma-separated list or wildcard expressions of fields to include in the statistics.

Response body

edit

The response body reports the per-shard usage count of the data structures that back the fields in the index. A given request will increment each count by a maximum value of 1, even if the request accesses the same field multiple times.

any

(integer) Denotes any kind of use of the field (e.g. via the inverted index, stored fields, doc values, etc.) such that any usage is counted once for a given search request.

inverted_index

(object) The inverted index is enabled by the index mapping parameter and configured by setting the index_options for the field.

Properties of inverted_index:

terms: (integer) Denotes the usages of terms in the inverted index, answering the question "Is this field’s inverted index used?".
postings: (integer) Denotes the usage of the posting list which contains the document ids for a given term.
proximity: (integer) Denotes any kind of usage of either positions, offsets or payloads in the inverted index such that any usage is counted once for a given search request.
positions: (integer) Denotes the usage of position data (order of the term) in the inverted index.
term_frequencies: (integer) Denotes the usage of the term frequencies in the inverted index which are used to calculate scores.
offsets: (integer) Denotes the usage of the offsets in the inverted index which store the start and end character offsets of the terms.
payloads: (integer) Denotes the usage of payloads in the inverted index, e.g. via the delimited payload token filter, or by user-defined analysis components and plugins.

stored_fields

(integer) Denotes the usage of stored fields. These are enabled via the store mapping option, and accessed by specifying the stored_fields query option. Note that the _source and _id fields are stored by default and their usage is counted here.

doc_values

(integer) Denotes the usage of doc values, which are primarily used for sorting and aggregations. These are enabled via the doc_values mapping parameter.

points

(integer) Denotes the usage of the Lucene PointValues which are the basis of most numeric field data types, including spacial data types, numbers, dates, and more. These are used by queries/aggregations for ranges, counts, bucketing, min/max, histograms, spacial, etc.

norms

(integer) Denotes the usage of norms which contain index-time boost values used for scoring.

term_vectors

(integer) Denotes the usage of term vectors which allow for a document’s terms to be retrieved at search time. Usages include highlighting and the More Like This Query.

knn_vectors

(integer) Denotes the usage of the knn_vectors field type, primarily used for k-nearest neighbor (kNN) search.

Examples

edit

The following request retrieves field usage information of index my-index-000001 on the currently available shards.

response = client.indices.field_usage_stats(
  index: 'my-index-000001'
)
puts response

GET /my-index-000001/_field_usage_stats

The API returns the following response:

{
    "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
    },
    "my-index-000001": {
        "shards": [
            {
                "tracking_id": "MpOl0QlTQ4SYYhEe6KgJoQ",
                "tracking_started_at_millis": 1625558985010,
                "routing": {
                    "state": "STARTED",
                    "primary": true,
                    "node": "gA6KeeVzQkGURFCUyV-e8Q",
                    "relocating_node": null
                },
                "stats" : {
                    "all_fields": { 
                        "any": "6",
                        "inverted_index": {
                            "terms" : 1,
                            "postings" : 1,
                            "proximity" : 1,
                            "positions" : 0,
                            "term_frequencies" : 1,
                            "offsets" : 0,
                            "payloads" : 0
                        },
                        "stored_fields" : 2,
                        "doc_values" : 1,
                        "points" : 0,
                        "norms" : 1,
                        "term_vectors" : 0,
                        "knn_vectors" : 0
                    },
                    "fields": {
                        "_id": { 
                            "any" : 1,
                            "inverted_index": {
                                "terms" : 1,
                                "postings" : 1,
                                "proximity" : 1,
                                "positions" : 0,
                                "term_frequencies" : 1,
                                "offsets" : 0,
                                "payloads" : 0
                            },
                            "stored_fields" : 1,
                            "doc_values" : 0,
                            "points" : 0,
                            "norms" : 0,
                            "term_vectors" : 0,
                            "knn_vectors" : 0
                        },
                        "_source": {...},
                        "context": {...},
                        "message.keyword": {...}
                    }
                }
            }
        ]
    }
}

	Reports the sums of the usage-counts for all fields in the index (on the listed shard).
	The field name for which the following usage-counts are reported (on the listed shard).

« Exists API Flush API »