IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.

› › ›

Analyze index disk usage API

edit

Analyze index disk usage API

edit

This functionality is in technical preview and may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features.

Analyzes the disk usage of each field of an index or data stream. This API might not support indices created in previous Elasticsearch versions. The result of a small index can be inaccurate as some parts of an index might not be analyzed by the API.

response = client.indices.disk_usage(
  index: 'my-index-000001',
  run_expensive_tasks: true
)
puts response

POST /my-index-000001/_disk_usage?run_expensive_tasks=true

Request

edit

POST /<target>/_disk_usage

Prerequisites

edit

If the Elasticsearch security features are enabled, you must have the manage index privilege for the target index, data stream, or alias.

Path parameters

edit

<target>: (Required, string) Comma-separated list of data streams, indices, and aliases used to limit the request. It’s recommended to execute this API with a single index (or the latest backing index of a data stream) as the API consumes resources significantly.

Query parameters

edit

allow_no_indices

(Optional, Boolean) If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices. This behavior applies even if the request targets other open indices. For example, a request targeting foo*,bar* returns an error if an index starts with foo but no index starts with bar.

Defaults to true.

expand_wildcards

(Optional, string) Type of index that wildcard patterns can match. If the request can target data streams, this argument determines whether wildcard expressions match hidden data streams. Supports comma-separated values, such as open,hidden. Valid values are:

all: Match any data stream or index, including hidden ones.
open: Match open, non-hidden indices. Also matches any non-hidden data stream.
closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
none: Wildcard patterns are not accepted.

Defaults to open.

flush

(Optional, Boolean) If true, the API performs a flush before analysis. If false, the response may not include uncommitted data. Defaults to true.

ignore_unavailable

(Optional, Boolean) If false, the request returns an error if it targets a missing or closed index. Defaults to false.

run_expensive_tasks

(Required, Boolean) Analyzing field disk usage is resource-intensive. To use the API, this parameter must be set to true. Defaults to false.

wait_for_active_shards

(Optional, string) The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). Default: 1, the primary shard.

See Active shards.

Examples

edit

response = client.indices.disk_usage(
  index: 'my-index-000001',
  run_expensive_tasks: true
)
puts response

POST /my-index-000001/_disk_usage?run_expensive_tasks=true

The API returns:

{
    "_shards": {
        "total": 1,
        "successful": 1,
        "failed": 0
    },
    "my-index-000001": {
        "store_size": "929mb", 
        "store_size_in_bytes": 974192723,
        "all_fields": {
            "total": "928.9mb", 
            "total_in_bytes": 973977084,
            "inverted_index": {
                "total": "107.8mb",
                "total_in_bytes": 113128526
            },
            "stored_fields": "623.5mb",
            "stored_fields_in_bytes": 653819143,
            "doc_values": "125.7mb",
            "doc_values_in_bytes": 131885142,
            "points": "59.9mb",
            "points_in_bytes": 62885773,
            "norms": "2.3kb",
            "norms_in_bytes": 2356,
            "term_vectors": "2.2kb",
            "term_vectors_in_bytes": 2310,
            "knn_vectors": "0b",
            "knn_vectors_in_bytes": 0
        },
        "fields": {
            "_id": {
                "total": "49.3mb",
                "total_in_bytes": 51709993,
                "inverted_index": {
                    "total": "29.7mb",
                    "total_in_bytes": 31172745
                },
                "stored_fields": "19.5mb", 
                "stored_fields_in_bytes": 20537248,
                "doc_values": "0b",
                "doc_values_in_bytes": 0,
                "points": "0b",
                "points_in_bytes": 0,
                "norms": "0b",
                "norms_in_bytes": 0,
                "term_vectors": "0b",
                "term_vectors_in_bytes": 0,
                "knn_vectors": "0b",
                "knn_vectors_in_bytes": 0
            },
            "_primary_term": {...},
            "_seq_no": {...},
            "_version": {...},
            "_source": {
                "total": "603.9mb",
                "total_in_bytes": 633281895,
                "inverted_index": {...},
                "stored_fields": "603.9mb", 
                "stored_fields_in_bytes": 633281895,
                "doc_values": "0b",
                "doc_values_in_bytes": 0,
                "points": "0b",
                "points_in_bytes": 0,
                "norms": "0b",
                "norms_in_bytes": 0,
                "term_vectors": "0b",
                "term_vectors_in_bytes": 0,
                "knn_vectors": "0b",
                "knn_vectors_in_bytes": 0
            },
            "context": {
                "total": "28.6mb",
                "total_in_bytes": 30060405,
                "inverted_index": {
                    "total": "22mb",
                    "total_in_bytes": 23090908
                },
                "stored_fields": "0b",
                "stored_fields_in_bytes": 0,
                "doc_values": "0b",
                "doc_values_in_bytes": 0,
                "points": "0b",
                "points_in_bytes": 0,
                "norms": "2.3kb",
                "norms_in_bytes": 2356,
                "term_vectors": "2.2kb",
                "term_vectors_in_bytes": 2310,
                "knn_vectors": "0b",
                "knn_vectors_in_bytes": 0
            },
            "context.keyword": {...},
            "message": {...},
            "message.keyword": {...}
        }
    }
}

	The store size of only analyzed shards of the index.
	The total size of fields of the analyzed shards of the index. This total is usually smaller than the index size specified in <1> as some small metadata files are ignored and some parts of data files might not be scanned by the API.
	The stored size of the `_id` field
	The stored size of the `_source` field. As stored fields are stored together in a compressed format, the sizes of stored fields are estimates and can be inaccurate. The stored size of the `_id` field is likely underestimated while the `_source` field is overestimated.

« Analyze API Clear cache API »