Keeping up with Kibana: This week in Kibana for April 16, 2018

By
Jin Mu

What do internationalization and localization mean? Why are we localizing Kibana? What are the popular JavaScript localization frameworks in the market? How did we choose a localization framework?

Recently we evaluated various JavaScript localization frameworks and would like to share our experience with you. We'll answer the above questions and hopefully give you insight for your own localization project.

Internationalization

Internationalization is the design and development of a product, application, or service that enables easy localization for audiences that vary in culture, region, or language. Internationalization is often written as i18n, where 18 is the number of letters between i and n in the English word.

Localization

Localization is the adaptation of your product, application, or service to the cultural, legal, and other requirements of a specific language, country, or population.  Localization is more than just translation of the user interface—it also includes customization related to:

  • Number formats, including date and time, phone numbers, duration, currency, and units of measure
  • Bi-directional display
  • Symbols, icons, and colors
  • Text and graphics
  • Local regulations
  • Legal requirements

Like internationalization, localization has a short form. It's often written as l10n, where 10 is the number of letters between l and n.

What is the difference between i18n and l10n?

Internationalization lays the foundation for easy localization. In other words, internationalization is the architecture and platform and localization is the data.

Why are we localizing Kibana?

As Elastic continues to grow, we have more and more customers who are not native English speakers. Localizing Kibana helps us better serve our users in countries such as China and Germany.

Challenges of localization

Localizing an application requires a tremendous amount of information, consideration, and effort. Every language has different rules and conventions, and adapting to them can be tricky.

In the JavaScript world, there is no standard way to implement localization. A lot of frameworks and tools exist—some are simple and address a limited set of use cases, some are big, complicated, or hard to extend, and some work only in a specific environment or UI framework. Sometimes developers must write a customized component to address specific needs.

Localizing Kibana is even more challenging because it relies on several UI frameworks (React and Angular) and requires localization in different environments (browser and NodeJS).

Popular i18n and i10n frameworks

Below are six popular frameworks that provide solutions to localizing a JavaScript application.

  • Angular-translate. This JavaScript translation library for AngularJS 1.x applications provides a set of filters and directives for client-side localization. It supports asynchronous loading of i18n data, full pluralization through MessageFormat, and more.
  • React-intl. This library provides React components and an API to format dates, numbers, and strings, including pluralization and handling translations. It's an open-source project from Yahoo, and part of Format.js.
  • i18next. This internationalization framework works in browser and NodeJS, providing plugins for many UI frameworks, including React and Angular.
  • MessageFormat.js. This lightweight and framework-agnostic JavaScript library is based on ICU message format.
  • Format.js. This modular collection of JavaScript libraries for internationalization focuses on formatting numbers, dates, and strings.
  • Fluent.js. This modern internationalization framework is built around the latest web platform technologies and custom FTL message format that work in browser and NodeJS.

Deciding which framework to use

In addition to common localization considerations such as default message support, easy message extraction, rich pluralization format, and good syntax ergonomic, we also took into account these aspects specific to Kibana.

  • The localization framework must be UI framework agnostic and easily consumable in all parts of Kibana because Angular and React co-live in Kibana.
  • The localization framework must support Node.js server-side generated content.

The following table shows the evaluation of the six frameworks against our requirements.

Metrics Angular-translate React-intl i18next Messageformat.js Format.js for Angular Fluent
ICU Message format support Yes Yes No Yes Yes No (custom FTL syntax)
Easy-to-read/good syntax ergonomics Yes (default ICU message-format syntax) Yes (default ICU message-format syntax) No (Angular wrapper requires passing all parameters as a string to the single attribute, react Trans component uses not obvious placeholders for messages) Yes (default ICU message-format syntax, flexible components API) Yes (default ICU message-format syntax, flexible components API) Yes (selectors and pluralization syntax may be cumbersome, FTL allows to add comments and reference one message within another)
Pluralization Yes Yes Yes, but not as flexible as ICU message-format (there are languages with multiple plural forms, but i18next pluralization only supports singular and plural) Yes Yes Yes (incl. CLDR pl. cat., follows Intl.PluralRules)
Date and Time Yes Yes No Yes Yes Yes (follows Intl.DateTimeFormat)
Number (incl. percent, currency) Yes Yes No Yes Yes Yes (follows Intl.NumberFormat)
Duration Yes Yes No Yes Yes Yes (via custom formatter)
BiDi/RTL support No No No No No Yes
Text with nested formatting Yes Yes Yes, but translation messages for React requres additional placeholders Yes, but React integration is complicated Yes Yes
Attributes (placeholder, title etc.) Yes, but without default messages Yes, but Not very convenient (doesn't use render callback approach) Yes Yes Yes Yes
Attribute with variables interpolation Yes Yes Yes Yes Yes Yes
Default message Yes, but filter doesn't support default messages Yes Yes, but pluralization doesn't support default messages Yes Yes Yes
Exisitng tools for ID/message extraction Yes (angular-translate-extract, gulp-angular-translate-extract) Yes (babel-plugin-react-intl) Yes (i18next-scanner) No, have to create our own tools No No (requires our own tooling similar to babel-plugin-react-intl and i18next-scanner built on top of fluent-syntax FTL parser)
Community 4k+ stars, ~300k downloads per month, last published 4 months ago, 126 contributors 7k+ stars, ~500k downloads per month, last published 7 months ago, 39 contributors 3k+ stars, ~400k downloads per month, last published 1 day ago, 97 contributors Messageformat.js itself has 1.2k+ stars, ~400k downloads per month, last published 9 hours ago, 34 contributors Intl MessageFormat itself has 400+ stars, ~600k downloads per month, last published 6 months ago, 19 contributors ~1 year old, 59 stars, last publish 6 days ago, 3 active full-time maintainers, used mainly by Firefox and other Mozilla projects
Plugin system support Yes (custom interpolators) Yes (custom formatters) Yes Yes (custom formatters) Yes (custom formatters) Yes (custom formatters/functions)
Angular/React support out of the box Yes, but only Angular Yes, but only React Yes, but Angular wrapper is too complicated compared to Angular-translate. No, requires custom wrappers Custom wrapper Yes, but Angular may require some additional work
Can be used on the server side (Node.js)? No Core library (Intl MessageFormat) can be used on the server side Yes Yes Yes (single engine for React, Angular and Node.js) Yes (single engine for React, Angular and Node.js)
Overall Summary Uses old messageformat.js dependency under the hood (v1.0.2) which leads to poor conformance with the ICU MessageFormat specifications with respect to quoting and escaping. It also has some syntax limitations, for example, filter doesn't support default messages. Large community, build on the JavaScript Intl built-ins and industry-wide i18n standards, well-documented API Pluralization is limited, Angular wrapper is not easy to use. Namespaces, graceful fallbacks, a lot of modules built for and around i18next, well-documented API We can make translation syntax for both of Angular and React really similar. We are able to modify components API, it gives us more flexibility. We can replace pluralization engine at any time Uses Intl MessageFormat under the hood (single engine for React, Angular and Node.js). We are able to modify Angular wrapper API, it gives us more flexibility. Have to write some additional tools for messages extraction Young and capable. Can be used in React, Angular and Node.js with small and well documented codebase. Main risks are low adoption and Non-universal language file format (FTL, hard to say where its limits are). Will require custom tooling, nothing extraordinary though (very similar to tools for other better known frameworks)

Conclusion

There is no perfect framework that meets all requirements. In deciding which framework to use, we had to balance the out-of-the-box features with the stability, flexibility, and priority of the requirements. We also had to consider our users and contributors to make sure they are continuously supported,

As a result, we decided to move forward with the solution based on the FormatJS core libs. We based our decision on the following:

  • It has a relatively small code base and good documentation
  • It uses well-known ICU message syntax
  • It can be used on the server side (Node.JS), in React (react-intl) and AngularJS (a custom component that we'll build).

You can implement localization in almost any framework. We encourage you to analyze your business scenarios, and then search and find the best option for your application.

Happy localizing!