IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Entity Analytics dashboard Cloud Native Vulnerability Management Dashboard »
Elastic Docs ›Elastic Security Solution [8.15] ›Dashboards

Data Quality dashboard

edit

Data Quality dashboard

edit

The Data Quality dashboard shows you whether your data is correctly mapped to the Elastic Common Schema (ECS). Successful mapping enables you to search, visualize, and interact with your data throughout Elastic Security and Kibana.

The Data Quality dashboard

Use the Data Quality dashboard to:

  • Check one or multiple indices for unsuccessful mappings, to help you identify problems (the indices used by Elastic Security appear by default).
  • View the amount of data stored in each of your indices.
  • View detailed information about the fields in checked indices.
  • Track unsuccessful mappings by creating a case or Markdown report based on data quality results.

The Data Quality dashboard doesn’t show data from cold or frozen data tiers. It also doesn’t display data from remote clusters using cross-cluster search. To view data from another cluster, log in to that cluster’s Kibana instance.

Requirements

To use the Data Quality dashboard, you need at least the following privileges for each index you want to check:

Check indices

edit

Data does not appear in the dashboard until a user selects indices to check.

  • Check multiple indices: To check all indices in the current data view, click Check all at the top of the dashboard. A progress indicator will appear.

To customize which indices are checked when you click Check all, change the current data view.

  • Check a single index: To check a single index, expand it using the arrow on the left. Checking a single index is faster than checking all indices.

Once checked, an index’s data quality results persist indefinitely. You can see when the index was last checked, and generate updated results at any time.

Data quality results are stored in a data stream using the following index pattern: .kibana-data-quality-dashboard-results-<spaceId>, where <spaceId> is the ID of the active Kibana space. For example, results from the default space are stored in: .kibana-data-quality-dashboard-results-default.

Visualize checked indices

edit

The treemap that appears at the top of the dashboard shows the relative size of your indices. The color of each index’s node refers to its status:

  • Blue: Not yet checked.
  • Green: Checked, no incompatible fields found.
  • Red: Checked, one or more incompatible fields found.

Click a node in the treemap to expand the corresponding index.

Learn more about checked index fields

edit

After an index is checked, an X (❌) or a checkmark (✅) appears in its Result column. The X (❌) indicates mapping problems in an index. To view index details, including which fields weren’t successfully mapped, click the arrow next to the result to expand it.

An expanded index with some failed results in the Data Quality dashboard

When you expand a result, the Summary tab immediately helps you visualize the status of fields in that index. The other tabs display more details about particular fields, grouped by their mapping status.

Fields in the Same family category have the correct search behavior, but might have different storage or performance characteristics (for example, you can index strings to both text and keyword fields). To learn more, refer to Field data types.

Export data quality results

edit

You can share data quality results to help track your team’s remediation efforts. First, follow the instructions under Check indices to generate results, then either:

  • Export results for all indices in the current data view:

    1. At the top of the dashboard, under the Check all button, are two buttons that allow you to share results. Exported results include all the data which appears in the dashboard.
    2. Click Add to new case to open a new case.
    3. Click Copy to clipboard to copy a Markdown report to your clipboard.

  • Export results for one index:

    1. Expand an index that has at least one incompatible field by clicking the arrow to the left of its Result.
    2. From the Summary or Incompatible fields tab, select Add to new case to open a new case.
    3. From the Summary, Incompatible fields, or Same family tab, click Copy to clipboard to copy a Markdown report to your clipboard.

For more information about how to fix mapping problems, refer to Mapping.

« Entity Analytics dashboard Cloud Native Vulnerability Management Dashboard »