New

The executive guide to generative AI

Read more
About usPartnersSupport|Login

WARNING: Version 0.90 of Elasticsearch has passed its EOL date.

This documentation is no longer being maintained and may be removed. If you are running this version, we strongly advise you to upgrade. For the latest information, see the current release documentation.

« Standard Tokenizer Keyword Tokenizer »
Elastic Docs ›Reference [0.90] ›Analysis ›Tokenizers

Edge NGram Tokenizer

edit

Edge NGram Tokenizer

edit

A tokenizer of type edgeNGram.

This tokenizer is very similar to nGram but only keeps n-grams which start at the beginning of a token.

The following are settings that can be set for a edgeNGram tokenizer type:

Setting Description Default value

min_gram

Minimum size in codepoints of a single n-gram

1.

max_gram

Maximum size in codepoints of a single n-gram

2.

token_chars

(Since 0.90.2) Characters classes to keep in the tokens, Elasticsearch will split on characters that don’t belong to any of these classes.

[] (Keep all characters)

token_chars accepts the following character classes:

letter

for example a, b, ï or 京

digit

for example 3 or 7

whitespace

for example " " or "\n"

punctuation

for example ! or "

symbol

for example $ or √

Example

edit
    curl -XPUT 'localhost:9200/test' -d '
    {
        "settings" : {
            "analysis" : {
                "analyzer" : {
                    "my_edge_ngram_analyzer" : {
                        "tokenizer" : "my_edge_ngram_tokenizer"
                    }
                },
                "tokenizer" : {
                    "my_edge_ngram_tokenizer" : {
                        "type" : "edgeNGram",
                        "min_gram" : "2",
                        "max_gram" : "5",
                        "token_chars": [ "letter", "digit" ]
                    }
                }
            }
        }
    }'

    curl 'localhost:9200/test/_analyze?pretty=1&analyzer=my_edge_ngram_analyzer' -d 'FC Schalke 04'
    # FC, Sc, Sch, Scha, Schal, 04

side deprecated

edit

There used to be a side parameter up to 0.90.1 but it is now deprecated. In order to emulate the behavior of "side" : "BACK" a reverse token filter should be used together with the edgeNGram token filter. The edgeNGram filter must be enclosed in reverse filters like this:

    "filter" : ["reverse", "edgeNGram", "reverse"]

which essentially reverses the token, builds front EdgeNGrams and reverses the ngram again. This has the same effect as the previous "side" : "BACK" setting.

« Standard Tokenizer Keyword Tokenizer »

On this page

Was this helpful?
Feedback