You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Wouter Heijke (JIRA)" <ji...@apache.org> on 2008/09/18 10:01:45 UTC
[jira] Created: (LUCENE-1391) Token type and flags values get lost
when using ShingleMatrixFilter
Token type and flags values get lost when using ShingleMatrixFilter
-------------------------------------------------------------------
Key: LUCENE-1391
URL: https://issues.apache.org/jira/browse/LUCENE-1391
Project: Lucene - Java
Issue Type: Bug
Components: contrib/analyzers
Affects Versions: 2.4
Reporter: Wouter Heijke
While using the new ShingleMatrixFilter I noticed that a token's type and flags get lost while using this filter. ShingleFilter does respect these values like the other filters I know.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Assigned: (LUCENE-1391) Token type and flags values get lost
when using ShingleMatrixFilter
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Karl Wettin reassigned LUCENE-1391:
-----------------------------------
Assignee: Karl Wettin
> Token type and flags values get lost when using ShingleMatrixFilter
> -------------------------------------------------------------------
>
> Key: LUCENE-1391
> URL: https://issues.apache.org/jira/browse/LUCENE-1391
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/analyzers
> Affects Versions: 2.4
> Reporter: Wouter Heijke
> Assignee: Karl Wettin
>
> While using the new ShingleMatrixFilter I noticed that a token's type and flags get lost while using this filter. ShingleFilter does respect these values like the other filters I know.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1391) Token type and flags values get
lost when using ShingleMatrixFilter
Posted by "Wouter Heijke (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632191#action_12632191 ]
Wouter Heijke commented on LUCENE-1391:
---------------------------------------
Hi Karl,
Yes the unigram tokens is what i'm referring to. The output tokens can have 'Shingle', or something I can define, even concattenated types could be handy in some cases!
Now that I come to it, I'm also missing setTokenType like ShingleFilter has.
> Token type and flags values get lost when using ShingleMatrixFilter
> -------------------------------------------------------------------
>
> Key: LUCENE-1391
> URL: https://issues.apache.org/jira/browse/LUCENE-1391
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/analyzers
> Affects Versions: 2.4
> Reporter: Wouter Heijke
> Assignee: Karl Wettin
>
> While using the new ShingleMatrixFilter I noticed that a token's type and flags get lost while using this filter. ShingleFilter does respect these values like the other filters I know.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org
[jira] Commented: (LUCENE-1391) Token type and flags values get
lost when using ShingleMatrixFilter
Posted by "Karl Wettin (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/LUCENE-1391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12632188#action_12632188 ]
Karl Wettin commented on LUCENE-1391:
-------------------------------------
Hi Wouter,
are you refering to unigram tokens? They should of course keep the original values, I'll take a look at that.
I'm not sure how one would/could retain the token type or flags in an output token that is a combination of multiple input tokens as their values might differ.
> Token type and flags values get lost when using ShingleMatrixFilter
> -------------------------------------------------------------------
>
> Key: LUCENE-1391
> URL: https://issues.apache.org/jira/browse/LUCENE-1391
> Project: Lucene - Java
> Issue Type: Bug
> Components: contrib/analyzers
> Affects Versions: 2.4
> Reporter: Wouter Heijke
>
> While using the new ShingleMatrixFilter I noticed that a token's type and flags get lost while using this filter. ShingleFilter does respect these values like the other filters I know.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org