Keeping up with Kibana: This week in Kibana for April 5th, 2019

By
Lee Drengenberg

Hiring

Security

Feature Controls

The review process for integrating feature controls into applications is almost done! We expect to merge to master very soon.


Platform

New Platform

UI plugin service and Generate core API docs from TSDoc comments are merged.

Notification plugin migration and decoupling ui/chrome from Angular are in good progress.  The team continues making progress on http service and application service.


Saved Object

This week we modified the export API to support including object dependencies automatically (#34225). Import APIs have been modified to further satisfy UI edge cases (#34161). A PR converting the UI to use the new server side APIs is in review process (#33513) and should be merged soon. We are currently working on supporting any type of saved object on export.


Alerting Service

We have been working on an adjusted plan for speeding up Alerting service and unblock solution teams. Security team has taken over the implementation of the Secret Service. Platform team will keep on driving the Alerting service project focusing on the Actions service and the Alerting framework.

Telemetry

When Monitoring is enabled, telemetry usage collectors are fired every 10 seconds. A fix to changing it to once per day is WIP. https://github.com/elastic/kibana/pull/34609


Encrypted Telemetry data is in good progress. https://github.com/elastic/kibana/pull/33121


Operations

  • Remove ts-loader from canvas storybook webpack config #33896
  • Migrate from tslint #33826
  • Fix lint rule for canvas #34650

Geo-Maps App

Geojson upload [IN PROGRESS]

  • Round-trip POC ready

Dashboard embeddability [COMPLETED]

  • Maps are embeddable in a dashboard now.

Filtering [IN PROGRESS]

  • Layer-based filtering completed.
  • Close to merging tooltip-filtering. Needs final touches. #33635
  • Design work for the introduction of a toolbar. This toolbar will accommodate all current and future tools. This includes the new drawing tools, as well as all the existing go-to tool #33686

Misc

  • Lots of gardening and tending to the details this week
    • Better experience
      • Remove JQuery dependency with new drag&drop #34553
      • Simplify configuration of extent-based-filtering by having the Maps app automatically inspect document count #34456
    • Better docs
      • Docs show the xpack icon for maps now #34170
      • Explain how the search bar works in an app that shows multiple index-patterns in a single view #34315
    • Bug fixes
      • Fix TOC jitter #34322
      • Fix runtime error that was logged to the console in certain edge-case when starting up the app #34321
      • Show correct app name in breadcrumb when saving a map with a new name #34287
      • WMS URLs now preserve query-params #34200


Geo-Elastic Maps Service

New Architecture for Tile Service [COMPLETED]

  • Tile service for all versions of Kibana has been migrated to a new, faster platform
  • The existing basemap style is modified slightly with better labels while still retaining the same color scheme
  • This new platform will enable us to add additional basemap styles (dark theme, satellite) in the future


Vector Maps

App Architecture

Interpreter in visualize

This week we enabled interpreter in Visualize by default. As a safety measure, we added kibana.yml flag to turn it off (#33832). This is the last step in a series of refactoring changes completed over the course of 6.x that position us to move Visualize and other apps to the new more modular data access layer.


In addition, we renamed endpoints from /api/canvas to /api/interpreter to reflect the more general use of this API (#34410) and improved semantics of tagcloud interpreter function (#34534).


New Platform Migration

This week our ui/public cleanup efforts have continued [#26505], with Liza recently merging changes to consolidate less-frequently used Angular directives [#34060, #34232], and to continue stripping out implicit ui/autoload dependencies [#33807].


Luke has been working on a strategy for creating legacy plugins in the shape of the new platform [#34350], so that we can get rid of ui/public imports and gradually move modules to their new "homes" behind the scenes.


Peter has begun de-angularizing index patterns [#34418], and we plan to complete this work soon. This will make index patterns one of the first items to move out of ui/public and into a proper plugin.

Kibana App (Visualizations, Sharing, Discover, Reporting)

New visual editor

This week we presented our progress to the Kibana all-team, including an overview of our mission statement and many videos of our work. Our focus this week was on what we are calling the "demo track," where we focused on building out critical features as quickly as possible, such as:

  • Hacky Drag and drop
  • Field configuration
  • Split charts
  • Better suggestions
  • Live previews of charts



Next week we will continue going down the "demo track" and also try to answer the following architectural questions:

  • What programming style is most extensible? More OOP / more functional?
  • Configuration Stress Testing: Pipeline Aggregations
  • Configuration Stress Testing: Aggregations with lots of options like Multi Filters
  • Rollup or other limitations on allowed arguments/aggregations in query building
  • Data from multiple indices
  • Saving to/Loading from expression as source of truth?
  • Is EUI drag and drop enough for our use case?

Elastic Charts

This week we published a set of useful features on version 3.7.1:

  • You can now specify a custom name for each chart series instead of relying only on the id as series name. This can be used also if you are really “evil” and you want to name all your series with the same name :)
  • You can specify to hide a series from the legend, used mainly in TSVB
  • Line annotations landed in 3.6.0. You can now add as many line annotations as you prefer on the chart, with their own style and marker, vertically or horizontally


In the meantime, we fixed a few bugs and we kept our test coverage level to a nice 94%:

  • Now you can display correctly also all negative or negative/positive datasets correctly
  • We fixed our production dependencies and adding the elastic-charts package to Kibana now works without any CI failure on ts type checks.


Canvas

The early parts of a cycle are all about getting started, so that’s what we’re featuring this week: The early stages of several ideas. Some of these are scheduled for 7.1, others are experimental and hey, maybe they’ll ship, maybe not. But if we don’t tell you about the experimental stuff how will you know to hassle us until we commit to them?


C is for Cookie Content

That’s good enough for me. Alex has been hard at work creating workpads to feature our products and solutions. Have the content itself is half the value, but the other half is the invaluable feedback we get hearing his experiences and ideas for making things better. Here’s the start of the filebeat workpad.



Custom Elements

The name is a work in progress, and Catherine has been making progress on this exciting feature that will allow us to build get more of that C for Content we talked about above. This is the template element thing we’ve been talking about for...forever. But now there’s code! And a screenshot! It’s happening! We’re also decluttering the elements options button explosion in the top right and making it easier to group/ungroup elements via a handy menu.



Vega element

While the Plotly renderer can only be an external plugin for now (due to no lazy loading and an otherwise fixable, common security issue) we already have Vega in our codebase. Element templates are coming up and it’d be an easy way to generate a variety of plots. A quick example (Vega on the left, current horizontal bar chart on the right):

There could be two kinds of uses:

  • Prepared elements - similar to how current Canvas plots work, but a much broader diversity of plots (boxplot, violin, Sankey, multilayer plots…), better performance and way more flexibility (bespoke graphic design, custom interactions, view data transforms etc.)
  • Generic vega and vega-lite elements: allow users to make their totally custom visualizations, and/or bundle them up via element templates 

What else?

Never enough for you is it? We broke, and then fixed, the asset manager, in the process adding tests. We found a gnarly issue in which telemetry was shipping waaaaaay too much data, fixed that too. Some tests are ready to go for the time filter and we fixed a bug in our grouping. We had a bit of discussion about how to solve the conceptual issue of using your upstream datatable in downstream styling. Variables were whispered quietly and a tiny, backwards compatible, extension to the grammar was tested out. No promises. Maybe some hassle would help?


Design

Drag and drop component now available in EUI

This week we merged a beta version of a new Drag and Drop component into EUI. We wanted to start with a simple feature set to start and within a day it found its way into Kibana’s maps application. If you find yourself using it or need more from it please let the design team know.


Selectable component now available in EUI

Also this week the team added a new component EuiSelectable which helps with the management of searchable lists. We’d been repeating the same sort of search and select system for filtering systems like filter groups and the nav and decided to formalize the pattern with something flexible in more situations. For example, when you need a searchable list within a popover.


Search Profiler cleanup

One of the more forgotten UIs in Kibana got a cleanup when we merged a PR for the Search Profiler dev tool. We primarily did this to fix some issues in dark mode but took the opportunity to briefly make it look closer to EUI standards even though this was touched just the CSS layer.


Uptime cleanup

With a bunch of help from Justin the design team was able to do a design PR on the Uptime app. This was mostly a quick restyling and we hope to revisit and improve things a bit more once the app begins to start using the new Elastic Charts library.


Cheers,
Kibana