You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Nicolas Lalevée (JIRA)" <ji...@apache.org> on 2008/01/25 11:18:34 UTC
[jira] Commented: (LUCENE-1150) The token types of the standard
tokenizer is not accessible
[ https://issues.apache.org/jira/browse/LUCENE-1150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12562424#action_12562424 ]
Nicolas Lalevée commented on LUCENE-1150:
-----------------------------------------
here is my workaround until it is fixed into the Lucene API :
{code:java}
package org.apache.lucene.analysis.standard;
public class TokenTypeAccessor {
public static final String APOSTROPHE_TYPE = StandardTokenizerImpl.TOKEN_TYPES[StandardTokenizerImpl.APOSTROPHE];
public static final String ACRONYM_TYPE = StandardTokenizerImpl.TOKEN_TYPES[StandardTokenizerImpl.ACRONYM];
public static final String HOST_TYPE = StandardTokenizerImpl.TOKEN_TYPES[StandardTokenizerImpl.HOST];
}
{code}
> The token types of the standard tokenizer is not accessible
> -----------------------------------------------------------
>
> Key: LUCENE-1150
> URL: https://issues.apache.org/jira/browse/LUCENE-1150
> Project: Lucene - Java
> Issue Type: Bug
> Components: Analysis
> Affects Versions: 2.3
> Reporter: Nicolas Lalevée
>
> The StandardTokenizerImpl not being public, these token types are not accessible :
> {code:java}
> public static final int ALPHANUM = 0;
> public static final int APOSTROPHE = 1;
> public static final int ACRONYM = 2;
> public static final int COMPANY = 3;
> public static final int EMAIL = 4;
> public static final int HOST = 5;
> public static final int NUM = 6;
> public static final int CJ = 7;
> /**
> * @deprecated this solves a bug where HOSTs that end with '.' are identified
> * as ACRONYMs. It is deprecated and will be removed in the next
> * release.
> */
> public static final int ACRONYM_DEP = 8;
> public static final String [] TOKEN_TYPES = new String [] {
> "<ALPHANUM>",
> "<APOSTROPHE>",
> "<ACRONYM>",
> "<COMPANY>",
> "<EMAIL>",
> "<HOST>",
> "<NUM>",
> "<CJ>",
> "<ACRONYM_DEP>"
> };
> {code}
> So no custom TokenFilter can be based of the token type. Actually even the StandardFilter cannot be writen outside the org.apache.lucene.analysis.standard package.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-dev-help@lucene.apache.org