Unique Token Filter

edit

The unique token filter can be used to only index unique tokens during analysis. By default it is applied on all the token stream. If only_on_same_position is set to true, it will only remove duplicate tokens on the same position.