Migrating from 8.7 to 8.8

Behavioral analytics were introduced in beta in Elastic 8.7. In version 8.8, the data model changes to support dashboards and future features. To use these new features, you must migrate to the new data model. Follow the steps in this document to migrate.

1. Create a collection

edit

Any existing analytics collections can continue to receive data, but to use the new analytics features, you must create new collections. You will need to create a new collection to store your analytics data. You can do this through the Kibana UI. Refer to Get started with Behavioral Analytics.

2. Add existing events to the new collection

edit

If you want to keep your existing events, you will need to:

Transform the data using the data structure outlined in Update events.
Reindex them into the new datastream. The datastream name is based on the collection name that you created. For example, if you created a collection called website, the datastream name will be website-behavioral_analytics-events.

Refer to the Elasticsearch JavaScript client documentation for an example of how to reindex documents.

3. Update events

edit

You will need to update your events to the new data structure.

Required fields

All events

All events require the following fields:

@timestamp (no change since 8.7)

data_stream(identifies the datastream the event belongs to)

{
  "@timestamp": 1682671778730,
  "data_stream": {
    "namespace": "<your-collection>", // Name of your collection
    "type": "behavioral_analytics", // Fixed value
    "dataset": "events" // Fixed value
  }
}

For the following events, you will need to update certain fields.

Search Event

This event represents when a user searches.

Required fields:

search.query

{
  search: {
    "query": "laptop", // required
    "filters": {
      "brand": ["apple"]
    },
    page: {
      current: 1,
      size: 10,
    },
    results: { // optional
      items: [
        {
          document: {
            id: "123",
            index: "products"
          },
          page: {
            url: "http://ww.my-website.com/url"
          }
        }
      ],
      total_results: 100
    },
    sort: {
      name: "relevance"
    }
  },
  event: {
    action: "search"
  },
  session: {
    id: "123"
  },
  user: {
    id: "456"
  }
}

Search Click Event

This event represents when a user clicks a document from a search.

Required Fields:

search.query

Any of: document or page. Both are preferable.

{
  search: {
    "query": "laptop", // required
    "filters": {
      "brand": ["apple"]
    },
    page: {
      current: 1,
      size: 10,
    },
    results: { // optional
      items: [
        {
          document: {
            id: "123",
            index: "products"
          },
          page: {
            url: "http://ww.my-website.com/url"
          }
        }
      ],
      total_results: 100
    },
    sort: {
      name: "relevance"
    }
  },
  event: {
    action: "search_click"
  },
  document: { id: "123", index: "products" },
  page: {
    url: "http://localhost:3000/javascript-tracker",
    title: "my product detail"
  },
  session: {
    id: "123"
  },
  user: {
    id: "456"
  }
}

Page View Event

This event represents when a user clicks a document from a search.

Required Fields:

Any of: document or page. Both are preferable.

{
  document: { id: "123", index: "products" },
  page: {
    url: "http://localhost:3000/javascript-tracker",
    title: "my product detail"
  },
  event: {
    action: "page_view"
  },
  session: {
    id: "123"
  },
  user: {
    id: "456"
  }
}

Behavioral analytics does not support custom events in 8.8. You will not be able to use custom events created in 8.7 in the new data model.

4. Update your tracker integration

edit

Depending on how you integrated the tracker (using either the browser-tracker or the javascript-tracker), you will need to update the integration to continue to capture events.

Follow the steps in your collection’s integration page to update your integration. Find this under Search > Behavioral Analytics > <my-collection> > Integration.

« Migrating from self-managed App Search Enterprise Search server known issues »