You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Alan Woodward (JIRA)" <ji...@apache.org> on 2018/12/03 12:18:01 UTC
[jira] [Commented] (LUCENE-8564) Make it easier to iterate over
graphs in tokenstreams
[ https://issues.apache.org/jira/browse/LUCENE-8564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16707106#comment-16707106 ]
Alan Woodward commented on LUCENE-8564:
---------------------------------------
Updated patch with improved end() processing, passes all tests so I think it's good to go?
> Make it easier to iterate over graphs in tokenstreams
> -----------------------------------------------------
>
> Key: LUCENE-8564
> URL: https://issues.apache.org/jira/browse/LUCENE-8564
> Project: Lucene - Core
> Issue Type: Task
> Reporter: Alan Woodward
> Assignee: Alan Woodward
> Priority: Major
> Attachments: LUCENE-8564.patch, LUCENE-8564.patch, LUCENE-8564.patch
>
>
> We have a number of TokenFilters that read ahead in the token stream (eg synonyms, shingles) and ideally these would understand token graphs as well as linear streams. FixedShingleFilter already has some mechanisms to deal with graphs; this issue is to extract this logic into a GraphTokenStream class that can then be reused by other token filters
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org