Flatten Graph Token Filter
editFlatten Graph Token Filter
editThis functionality is marked as experimental in Lucene
The flatten_graph
token filter accepts an arbitrary graph token
stream, such as that produced by
Synonym Graph Token Filter, and flattens it into a single
linear chain of tokens suitable for indexing.
This is a lossy process, as separate side paths are squashed on top of one another, but it is necessary if you use a graph token stream during indexing because a Lucene index cannot currently represent a graph. For this reason, it’s best to apply graph analyzers only at search time because that preserves the full graph structure and gives correct matches for proximity queries.
For more information on this topic and its various complexities, please read the Lucene’s TokenStreams are actually graphs blog post.