IMPORTANT: No additional bug fixes or documentation updates will be released for this version. For the latest information, see the current release documentation.
« Products and Solutions that Support ECS Additional Information »
Elastic Docs ›Elastic Common Schema (ECS) Reference [1.8] ›Migrating to ECS

Converting a Custom Implementation

edit

Converting a Custom Implementation

edit

A common schema helps you correlate and use data from various sources.

Fields for most Elastic modules and solutions (version 7.0 and later) are mapped to the Elastic Common Schema. You may want to map data from other implementations to ECS to help you correlate data across all of your products and solutions.

Before you start a conversion, be sure that you understand the basics below.

Core and extended levels

edit

Make sure you understand the distinction between Core and Extended fields, as explained in the Guidelines and Best Practices.

An approach to mapping an existing implementation

edit

Here’s the recommended approach for converting an existing implementation to ECS.

  1. Review each field in your original event and map it to the relevant ECS field.

    • Start by mapping your field to the relevant ECS Core field.
    • If a relevant ECS Core field does not exist, map your field to the relevant ECS Extended field.
    • If no relevant ECS Extended field exists, consider keeping your field with its original details, or possibly renaming it using ECS naming guidelines and attempt to map one or more of your original event fields to it.

  2. Review each ECS Core field, and attempt to populate it.

    • Review your original event data again
    • Consider populating the field based on additional meta-data such as static information (e.g. add event.category:authentication even if your auth events don’t mention the word "authentication")
    • Consider capturing additional environment meta-data, such as information about the host, container or cloud instance.
  3. Review other extended fields from any field set you are already using, and attempt to populate it as well.
  4. Set ecs.version to the version of the schema you are conforming to. This will allow you to upgrade your sources, pipelines and content (like dashboards) smoothly in the future.

Using a spreadsheet to plan your migration

edit

Using a spreadsheet to plan the migration from pre-existing source fields to ECS is a common practice. It’s a good way to address each of your fields methodically among colleagues.

If the data source is either a structured log, or if you already have a pipeline producing events with these non-ECS field names, the tool ECS Mapper may help you get started in performing all of these field renames.

After exporting your mapping spreadsheet to CSV, ECS Mapper will convert your field mapping to equivalent pipelines for Beats, Elasticsearch, and Logstash. Learn more at ECS Mapper.

« Products and Solutions that Support ECS Additional Information »