You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Paul Taylor <pa...@fastmail.fm> on 2009/09/30 00:16:32 UTC
Problem searching non analyzed fields
I use the same Analyzer for both creating an index and searching
however I'm having a problem with some fields that I added with
Field.Index.NOT_ANALYZED, how can I enforce they are also search without
being analysed.
I did this for some fields containg Guids and products codes because I
didn't want them modifying/tokenizing at all and it works fine for
Guids, however one of these fields contains Amazon Id of B00004Y6O9 and
I think when I search for this it fails because the analyzer I'm using
lowercases all text and nowhere do I specify when parsing not to use the
analyzer when searching, but of course its not getting lowercased when
added to the index.
code extract:
IndexSearcher is = IndexSearcher(IndexReader.open(new NIOFSDirectory(new
File(indexDir + '/' + indexName + '/'), null), true));
QueryParser qp = new QueryParser(defaultFields.get(0), analyzer);
TopScoreDocCollector collector = TopScoreDocCollector.create(offset +
limit, true);
searcher.search(parser.parse(query), collector);
So how do I tell the parser not to analyse some fields, Im guessing the
idea is to subclass the QueryParser but how, or should I be using
Field.Index.ANALYZED when adding to the index
thanks Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
RE: Problem searching non analyzed fields
Posted by Uwe Schindler <uw...@thetaphi.de>.
Hallo,
If you do not analyze fields during indexing, you cannot really use
QueryParser on the search side (because the QueryParser itsself always
analyzes the entered query string).
If you added some fields using NOT_ANALYZED, just use a simple "new
TermQuery(new Term(field, term))" to search for them. This is simplier and
does not analyze. If you are creating such queries programmatically and have
special "id" fields, it is the classical case for TermQueries.
Or, as Robert said, use an Analyzer, but use a special one, not corrupting
your ASINs.
Uwe
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de
> -----Original Message-----
> From: Paul Taylor [mailto:paul_t100@fastmail.fm]
> Sent: Wednesday, September 30, 2009 12:17 AM
> To: java-user@lucene.apache.org
> Subject: Problem searching non analyzed fields
>
> I use the same Analyzer for both creating an index and searching
> however I'm having a problem with some fields that I added with
> Field.Index.NOT_ANALYZED, how can I enforce they are also search without
> being analysed.
>
> I did this for some fields containg Guids and products codes because I
> didn't want them modifying/tokenizing at all and it works fine for
> Guids, however one of these fields contains Amazon Id of B00004Y6O9 and
> I think when I search for this it fails because the analyzer I'm using
> lowercases all text and nowhere do I specify when parsing not to use the
> analyzer when searching, but of course its not getting lowercased when
> added to the index.
>
> code extract:
> IndexSearcher is = IndexSearcher(IndexReader.open(new NIOFSDirectory(new
> File(indexDir + '/' + indexName + '/'), null), true));
> QueryParser qp = new QueryParser(defaultFields.get(0), analyzer);
> TopScoreDocCollector collector = TopScoreDocCollector.create(offset +
> limit, true);
> searcher.search(parser.parse(query), collector);
>
> So how do I tell the parser not to analyse some fields, Im guessing the
> idea is to subclass the QueryParser but how, or should I be using
> Field.Index.ANALYZED when adding to the index
>
> thanks Paul
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Problem searching non analyzed fields
Posted by Paul Taylor <pa...@fastmail.fm>.
Robert Muir wrote:
> try checking out PerFieldAnalyzerWrapper, so you can specify how each field
> is handled, i.e. some fields with KeywordAnalyzer, other fields with
> StandardAnalyzer, etc.
Thanks, yes actually I realize these fields do need some analysis
because I want to the search to be case insensitive, so Im going to
create my own analyzer like KeywordAnalyzer but with additional
LowercaseFilter, and then use with PerFieldAnalyserWrapper as suggested.
Paul
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Problem searching non analyzed fields
Posted by Robert Muir <rc...@gmail.com>.
try checking out PerFieldAnalyzerWrapper, so you can specify how each field
is handled, i.e. some fields with KeywordAnalyzer, other fields with
StandardAnalyzer, etc.
On Tue, Sep 29, 2009 at 6:16 PM, Paul Taylor <pa...@fastmail.fm> wrote:
> I use the same Analyzer for both creating an index and searching however
> I'm having a problem with some fields that I added with
> Field.Index.NOT_ANALYZED, how can I enforce they are also search without
> being analysed.
>
> I did this for some fields containg Guids and products codes because I
> didn't want them modifying/tokenizing at all and it works fine for Guids,
> however one of these fields contains Amazon Id of B00004Y6O9 and I think
> when I search for this it fails because the analyzer I'm using lowercases
> all text and nowhere do I specify when parsing not to use the analyzer when
> searching, but of course its not getting lowercased when added to the index.
>
> code extract:
> IndexSearcher is = IndexSearcher(IndexReader.open(new NIOFSDirectory(new
> File(indexDir + '/' + indexName + '/'), null), true));
> QueryParser qp = new QueryParser(defaultFields.get(0), analyzer);
> TopScoreDocCollector collector = TopScoreDocCollector.create(offset +
> limit, true);
> searcher.search(parser.parse(query), collector);
>
> So how do I tell the parser not to analyse some fields, Im guessing the
> idea is to subclass the QueryParser but how, or should I be using
> Field.Index.ANALYZED when adding to the index
>
> thanks Paul
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
--
Robert Muir
rcmuir@gmail.com