Keyword Repeat Token Filter

edit

Keyword Repeat Token Filter

edit

The keyword_repeat token filter Emits each incoming token twice once as keyword and once as a non-keyword to allow an unstemmed version of a term to be indexed side by side with the stemmed version of the term. Given the nature of this filter each token that isn’t transformed by a subsequent stemmer will be indexed twice. Therefore, consider adding a unique filter with only_on_same_position set to true to drop unnecessary duplicates.

Here is an example:

index :
    analysis :
        analyzer :
            myAnalyzer :
                type : custom
                tokenizer : standard
                filter : [lowercase, keyword_repeat, porter_stem, unique_stem]
            unique_stem:
                type: unique
                only_on_same_position : true