Adding filter capabilities to Vega Sankey visualizations in Kibana

In this blog we are going to discover how to enable Vega Sankey visualizations to create Kibana Filters.

Vega is a standout tool in the world of data visualization, allowing you to create detailed and interactive visual displays of your data. Using a simple JSON syntax, Vega allows you to define how your visualizations look and behave. It's a key part of the Kibana ecosystem, meaning you can use it alongside other Kibana visualizations within your dashboards. An in-depth exploration of how Vega works can be found in this blog.

Vega shines when you need more than what Kibana's standard visualization tools offer. It lets you build custom visualizations tailored to your specific needs, whether you're dealing with complex data relationships or want a unique visual style.

A few years back, we shared a blog on how to create a Sankey chart in Kibana. Sankey charts are great for illustrating the relationship between two data fields. While the chart worked well, as Kibana grew with new features, a limitation became clear: the inability to create a Kibana filter directly from the chart.

In this blog we are going to discover how we enabled the Sankey visualization from the previous blog to create Kibana Filters.

NOTE: when the code is copied to Kibana, it may present you a warning message stating:The URL is big and Kibana might stop working. To resolve this error go to Kibana Advanced settings and enable Store in Session.

Adjust Vega code

Kibana Filter

When you use Vega within Kibana, you gain access to special functions that allow you to adjust Kibana's settings directly from your Vega visualizations. These functions let you modify the Kibana context settings, enhancing the interactivity of your dashboards.

Here, we'll focus on using the kibanaAddFilter function. This function allows you to add a filter to the Kibana dashboard where your visualization is displayed. Keep in mind that these functions are exclusive to Vega and are not available in Vega-Lite.

Filter when clicking

We’re using the kibanaAddFilter function to add a Filter to Kibana when a stack is clicked in the visualization. Since the two stacks are both based on different fields, we need to make sure we are actually filtering on the correct field. This requires a conditional expression to determine which stack was clicked.

In Vega, you can use conditional expressions with the pattern: { condition ? if value : else value }. For our condition, we check if the clicked stack is stk1. This stk1 has been defined in the data section of our Vega code, and is used for the left stack in the chart. If the condition is true, it means the left stack was clicked, and we should filter on the geo.src field. If false, we filter on the geo.dest field, which corresponds to the right stack. With this logic, we can update the groupSelector in the signals section to reflect the correct filtering behavior.

    {
      name: groupSelector
      value: false
      on: [
        {
          // Clicking groupMark sets this signal to the filter values
          events: @groupMark:click!
          update: '''{stack:datum.stack == "stk1" ? kibanaAddFilter({"match_phrase": {"machine.os.keyword":datum.grpId}}) : kibanaAddFilter({"match_phrase": {"geo.dest":datum.grpId}}) }'''
        }
      ]
    }

Remove unused code

Now that the filtering is handled at the Kibana level, we can simplify our Vega visualization by removing some unnecessary logic. First, we can eliminate the filter logic from the data section of the Vega specification.

        // when a country is selected, filter out unrelated data
        {
          type: filter
          expr: !groupSelector || groupSelector.stk1 == datum.stk1 || groupSelector.stk2 == datum.stk2
        }

Secondly, we can remove the logic to show the “show all”-button from the signals.

        {
          // Clicking "show all" button, or double-clicking anywhere resets it
          events: [
            {type: "click", markname: "groupReset"}
            {type: "dblclick"}
          ]
          update: "false"
        }

Finally, the “show all” button itself can also be removed.

    {
      // Create a "show all" button. Shown only when a country is selected.
      type: group
      data: [
        // We need to make the button show only when groupSelector signal is true.
        // Each mark is drawn as many times as there are elements in the backing data.
        // Which means that if values list is empty, it will not be drawn.
        // Here I create a data source with one empty object, and filter that list
        // based on the signal value. This can only be done in a group.
        {
          name: dataForShowAll
          values: [{}]
          transform: [{type: "filter", expr: "groupSelector"}]
        }
      ]
      // Set button size and positioning
      encode: {
        enter: {
          xc: {signal: "width/2"}
          y: {value: 30}
          width: {value: 80}
          height: {value: 30}
        }
      }
      marks: [
        {
          // This group is shown as a button with rounded corners.
          type: group
          // mark name allows signal capturing
          name: groupReset
          // Only shows button if dataForShowAll has values.
          from: {data: "dataForShowAll"}
          encode: {
            enter: {
              cornerRadius: {value: 6}
              fill: {value: "#f5f5f5"}
              stroke: {value: "#c1c1c1"}
              strokeWidth: {value: 2}
              // use parent group's size
              height: {
                field: {group: "height"}
              }
              width: {
                field: {group: "width"}
              }
            }
            update: {
              // groups are transparent by default
              opacity: {value: 1}
            }
            hover: {
              opacity: {value: 0.7}
            }
          }
          marks: [
            {
              type: text
              // if true, it will prevent clicking on the button when over text.
              interactive: false
              encode: {
                enter: {
                  // center text in the parent group
                  xc: {
                    field: {group: "width"}
                    mult: 0.5
                  }
                  yc: {
                    field: {group: "height"}
                    mult: 0.5
                    offset: 2
                  }
                  align: {value: "center"}
                  baseline: {value: "middle"}
                  fontWeight: {value: "bold"}
                  text: {value: "Show All"}
                }
              }
            }
          ]
        }
      ]
    }

Change dataset

To fully utilize the Sankey visualization, you need to apply it to your own dataset. In this guide, we'll adjust the visualization to work with the Logs Kibana sample dataset. Start by ensuring the Sample web logs are loaded. Then, modify the Vega code to fit the new dataset by changing the index to kibana_sample_data_logs.

If @timestamp isn't used as the timestamp field for your dataset, you'll need to update the %timefield% setting. Setting the %timefield% settings ensures we are able to use the Kibana time range. Therefore, the Kibana time range should correspond to the time range of your dataset. To change the field used for one of the stacks, make adjustments in two areas: data retrieval and filtering logic. Since the Logs Kibana sample dataset has only one geo.src value, we'll change the field to machine.os.keyword.

Conclusion

Lens is the go-to visualization editor in Kibana because it's both flexible and user-friendly. However, for certain visualization types, Vega offers the flexibility to create visuals that Lens doesn't support, like the Sankey chart we discussed in this blog. Initially, the Sankey visualization couldn't create Kibana filters, a feature available in Lens. This blog shows how you can create Kibana filters with Vega using the kibanaAddFilter function.

Vega Code

The resulting Vega configuration can be found here in the Github Repository.

Note: we have updated the vega version to version 5, since that is the latest version at time of writing.

Want to get Elastic certified? Find out when the next Elasticsearch Engineer training is running!

Elasticsearch is packed with new features to help you build the best search solutions for your use case. Dive into our sample notebooks to learn more, start a free cloud trial, or try Elastic on your local machine now.

Report an issue