New

The executive guide to generative AI

Read more
About usPartnersSupport|Login
« Detections requirements About detection rules »
Elastic Docs ›Elastic Security Solution [8.17] ›Detections and alerts

Using logsdb index mode with Elastic Security

edit

Using logsdb index mode with Elastic Security

edit

To use the synthetic _source feature, you must have the appropriate subscription. Refer to the subscription page for Elastic Cloud and Elastic Stack/self-managed for the breakdown of available features and their associated subscription tiers.

This topic explains the impact of using logsdb index mode with Elastic Security.

With logsdb index mode, the original _source field is not stored in the index but can be reconstructed using synthetic _source.

When the _source is reconstructed, modifications are possible. Therefore, there could be a mismatch between users' expectations and how fields are formatted.

Continue reading to find out how this affects specific Elastic Security components.

Logsdb index mode is fully supported, and is recommended for new Elastic Security deployments. Logsdb is not recommended for existing Elastic Security deployments unless users fully understand and accept the documented changes to detection alert documents, runtime fields, and rule actions (refer to the sections below), and have ensured that their deployment has sufficient excess hot data tier CPU capacity to support the logsdb ingesting and indexing process. Enabling logsdb without sufficient excess hot data tier CPU capacity may result in data ingestion backups and or security detection rule timeouts and errors.

Alerts

edit

When alerts are generated, the _source event is copied into the alert to retain the original data. When the logsdb index mode is applied, the _source event stored in the alert is reconstructed using synthetic _source.

If you’re switching to use logsdb index mode, the _source field stored in the alert might look different in certain situations:

Alerts generated by the following rule types could be affected:

  • Custom query
  • Event correlation (non-sequence only)
  • Non-aggregate rule types (for example, ES|QL rules that use non-aggregating queries)

Alerts that are generated by threshold, machine learning, and event correlation sequence rules are not affected since they do not contain copies of the original source.

Rule actions

edit

While we do not recommend using _source for actions, in cases where the action relies on the _source, the same limitations and changes apply.

If you send alert notifications by enabling actions to the external systems that have workflows or automations based on fields formatted from the original source, they may be affected. In particular, this can happen when the fields used are arrays of objects.

We recommend checking and adjusting the rule actions using _source before switching to logsdb index mode.

Runtime fields

edit

Runtime fields that reference _source may be affected. Some runtime fields might not work and need to be adjusted. For example, if an event was indexed with the value of agent.name in the dot-notation form, it will be returned in the nested form and might not work.

The following is an example of accessing _source that works with the logsdb index mode enabled:

"source": """  emit(params._source.agent.name + "_____" + doc['agent.name'].value ); """
"source": """  emit(params._source['agent']['name'] + "_____" + doc['agent.name'].value );  """
"source": """  emit(field('agent.name').get(null) + "_____" + doc['agent.name'].value ); """
"source": """  emit($('agent.name', null) + "_____" + doc['agent.name'].value ); """
Copy as curlTry in Elastic 

The following will not work with synthetic source (logsdb index mode enabled):

"source": """  emit(params._source['agent.name'] + "_____" + doc['agent.name'].value );  """
Copy as curlTry in Elastic 
« Detections requirements About detection rules »

On this page

Was this helpful?
Feedback