You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by "Peter T. Brown" <pe...@headlesshunter.com> on 2005/08/04 19:15:07 UTC
Using Custom Analyzer to create case sensitive queries
I have a custom analyzer I've created so that certain fields, which are
indexed as Keywords, can be search exactly as they are typed in. Using this
custom analyzer however produces not hits. Using Luke, I can see that the
field values are identical to the values passed in on the query...
My query is: class.name:(someCaseSensitiveString.that.Can.include.Periods)
My custom Analyzer:
public static class DefaultAnalyzer extends Analyzer {
public TokenStream tokenStream(String fieldName, Reader reader) {
LOG.debug("analyzing: " + fieldName);
if ("class.name".equals(fieldName)) {
LOG.debug("not analyzing");
return new CharTokenizer(reader) {
protected boolean isTokenChar(char c) {
return true;
}
};
} else {
LOG.debug("analyzing");
StandardTokenizer tokenizer = new StandardTokenizer(reader);
TokenStream result = null;
result = new LowerCaseFilter(tokenizer);
result = new StopFilter(result,
StopAnalyzer.ENGLISH_STOP_WORDS);
result = new PorterStemFilter(result);
return result;
}
}
}
Any ideas?
Thank You
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Using Custom Analyzer to create case sensitive queries
Posted by "Peter T. Brown" <pe...@headlesshunter.com>.
Thank You.
Yes, I did check that and the CharTokenizer is correctly being used for the
field in question.
How should I use PerFieldAnalyzerWrapper ?
> From: Otis Gospodnetic <ot...@yahoo.com>
> Reply-To: <ja...@lucene.apache.org>
> Date: Thu, 4 Aug 2005 10:25:36 -0700 (PDT)
> To: <ja...@lucene.apache.org>
> Subject: Re: Using Custom Analyzer to create case sensitive queries
>
> PerFieldAnalyzerWrapper
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Using Custom Analyzer to create case sensitive queries
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Add some debugging to the if statement that handles "class.name" field.
My first guess is that you're not getting in that if case when
querying.
If that turns out to be true, consider using PerFieldAnalyzerWrapper to
accomplish the same thing.
Otis
--- "Peter T. Brown" <pe...@headlesshunter.com> wrote:
> I have a custom analyzer I've created so that certain fields, which
> are
> indexed as Keywords, can be search exactly as they are typed in.
> Using this
> custom analyzer however produces not hits. Using Luke, I can see that
> the
> field values are identical to the values passed in on the query...
>
> My query is:
> class.name:(someCaseSensitiveString.that.Can.include.Periods)
>
>
> My custom Analyzer:
>
> public static class DefaultAnalyzer extends Analyzer {
> public TokenStream tokenStream(String fieldName, Reader
> reader) {
> LOG.debug("analyzing: " + fieldName);
>
> if ("class.name".equals(fieldName)) {
> LOG.debug("not analyzing");
> return new CharTokenizer(reader) {
> protected boolean isTokenChar(char c) {
> return true;
> }
> };
> } else {
> LOG.debug("analyzing");
> StandardTokenizer tokenizer = new
> StandardTokenizer(reader);
> TokenStream result = null;
> result = new LowerCaseFilter(tokenizer);
> result = new StopFilter(result,
> StopAnalyzer.ENGLISH_STOP_WORDS);
> result = new PorterStemFilter(result);
> return result;
> }
> }
> }
>
>
>
>
> Any ideas?
>
>
> Thank You
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org