Logs data stream
editLogs data stream
editLogs data streams and the logsdb index mode are in tech preview and may be changed or removed in the future. Don’t use logs data streams or logsdb index mode in production.
A logs data stream is a data stream type that stores log data more efficiently.
In benchmarks, log data stored in a logs data stream used ~2.5 times less disk space than a regular data stream. The exact impact will vary depending on your data set.
The following features are enabled in a logs data stream:
-
Synthetic source, which omits storing the
_source
field. When the document source is requested, it is synthesized from document fields upon retrieval. -
Index sorting. This yields a lower storage footprint. By default indices are sorted by
host.name
and@timestamp
fields at index time. -
More space efficient compression for fields with
doc_values
enabled.
Create a logs data stream
editTo create a logs data stream, set your index template index.mode
to logsdb
:
resp = client.indices.put_index_template( name="my-index-template", index_patterns=[ "logs-*" ], data_stream={}, template={ "settings": { "index.mode": "logsdb" } }, priority=101, ) print(resp)
const response = await client.indices.putIndexTemplate({ name: "my-index-template", index_patterns: ["logs-*"], data_stream: {}, template: { settings: { "index.mode": "logsdb", }, }, priority: 101, }); console.log(response);
PUT _index_template/my-index-template { "index_patterns": ["logs-*"], "data_stream": { }, "template": { "settings": { "index.mode": "logsdb" } }, "priority": 101 }
The index mode setting. |
|
The index template priority. By default, Elasticsearch ships with an index template with a |
After the index template is created, new indices that use the template will be configured as a logs data stream. You can start indexing data and using the data stream.