You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Shawn Konopinsky <sk...@blueprint.org> on 2005/01/11 21:44:48 UTC
Token Characters
Hey There,
Wondering where I can find a list of the set of characters that the
StandardAnalyzer will tokenize on when indexing text in Lucene.
Best,
Shawn.
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org
Re: Token Characters
Posted by Otis Gospodnetic <ot...@yahoo.com>.
The best place to look is:
./src/java/org/apache/lucene/analysis/standard/StandardTokenizer.jj
You can see it at:
http://cvs.apache.org/viewcvs.cgi/jakarta-lucene/src/java/org/apache/lucene/analysis/standard/
Otis
--- Shawn Konopinsky <sk...@blueprint.org> wrote:
> Hey There,
>
> Wondering where I can find a list of the set of characters that the
> StandardAnalyzer will tokenize on when indexing text in Lucene.
>
> Best,
> Shawn.
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
> For additional commands, e-mail: lucene-user-help@jakarta.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: lucene-user-unsubscribe@jakarta.apache.org
For additional commands, e-mail: lucene-user-help@jakarta.apache.org