NOTE: You are looking at documentation for an older release. For the latest information, see the current release documentation.
ICU Folding Token Filter
editICU Folding Token Filter
editCase folding of Unicode characters based on UTR#30, like the
ASCII-folding token filter
on steroids. It registers itself as the icu_folding token filter and is
available to all indices:
PUT icu_sample
{
"settings": {
"index": {
"analysis": {
"analyzer": {
"folded": {
"tokenizer": "icu_tokenizer",
"filter": [
"icu_folding"
]
}
}
}
}
}
}
The ICU folding token filter already does Unicode normalization, so there is no need to use Normalize character or token filter as well.
Which letters are folded can be controlled by specifying the
unicodeSetFilter parameter, which accepts a
UnicodeSet.
The following example exempts Swedish characters from folding. It is important
to note that both upper and lowercase forms should be specified, and that
these filtered character are not lowercased which is why we add the
lowercase filter as well:
PUT icu_sample
{
"settings": {
"index": {
"analysis": {
"analyzer": {
"swedish_analyzer": {
"tokenizer": "icu_tokenizer",
"filter": [
"swedish_folding",
"lowercase"
]
}
},
"filter": {
"swedish_folding": {
"type": "icu_folding",
"unicodeSetFilter": "[^åäöÅÄÖ]"
}
}
}
}
}
}