This documentation contains work-in-progress information for future Elastic Stack and Cloud releases. Use the version selector to view supported release docs. It also contains some Elastic Cloud serverless information. Check out our serverless docs for more details.
« ICU collation keyword field Japanese (kuromoji) analysis plugin »
Elastic Docs ›Elasticsearch Plugins and Integrations [master] ›Analysis plugins ›ICU analysis plugin

ICU transform token filter

edit

ICU transform token filter

edit

Transforms are used to process Unicode text in many different ways, such as case mapping, normalization, transliteration and bidirectional text handling.

You can define which transformation you want to apply with the id parameter (defaults to Null), and specify text direction with the dir parameter which accepts forward (default) for LTR and reverse for RTL. Custom rulesets are not yet supported.

For example:

PUT icu_sample
{
  "settings": {
    "index": {
      "analysis": {
        "analyzer": {
          "latin": {
            "tokenizer": "keyword",
            "filter": [
              "myLatinTransform"
            ]
          }
        },
        "filter": {
          "myLatinTransform": {
            "type": "icu_transform",
            "id": "Any-Latin; NFD; [:Nonspacing Mark:] Remove; NFC" 
          }
        }
      }
    }
  }
}

GET icu_sample/_analyze
{
  "analyzer": "latin",
  "text": "你好" 
}

GET icu_sample/_analyze
{
  "analyzer": "latin",
  "text": "здравствуйте" 
}

GET icu_sample/_analyze
{
  "analyzer": "latin",
  "text": "こんにちは" 
}

This transforms transliterates characters to Latin, and separates accents from their base characters, removes the accents, and then puts the remaining text into an unaccented form.

Returns ni hao.

Returns zdravstvujte.

Returns kon'nichiha.

For more documentation, Please see the user guide of ICU Transform.

« ICU collation keyword field Japanese (kuromoji) analysis plugin »