You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Uwe Schindler (JIRA)" <ji...@apache.org> on 2017/01/13 12:52:26 UTC
[jira] [Commented] (LUCENE-7630) EdgeNGramTokenFilter drops
payloads
[ https://issues.apache.org/jira/browse/LUCENE-7630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15821733#comment-15821733 ]
Uwe Schindler commented on LUCENE-7630:
---------------------------------------
Hi, could you create a Pull Request and add the link here?
About your branch: I would not use cloneAttributes() because thats slow for this simple case. cloneAttributes() only helps if you want to modify the attributes in the AttributeSource that was created, but is not useful for simple save/restore use cases.
For your case, you should simple use captureState(), save the state object and then call restorestate() instead of clearAttributes(). After restoring you can adapt term text and positions/offsets. In addition when you clone or capture state, the call to clearAttributes() is useless and also slows down. When restoring states, everything is restored, so the additional clearing before is not needed.
> EdgeNGramTokenFilter drops payloads
> -----------------------------------
>
> Key: LUCENE-7630
> URL: https://issues.apache.org/jira/browse/LUCENE-7630
> Project: Lucene - Core
> Issue Type: Bug
> Components: modules/analysis
> Affects Versions: master (7.0)
> Reporter: Nathan Gass
> Priority: Minor
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> Using an EdgeNGramTokenFilter after a DelimitedPayloadTokenFilter discards the payloads, where as most other filters copy the payload to the new tokens.
> I added a test for this issue and a possible fix at https://github.com/xabbu42/lucene-solr/tree/edgepayloads
> Greetings
> Nathan Gass
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org