You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by ejblom <ej...@gmail.com> on 2012/01/16 16:30:25 UTC
LUCENE_35 index keyword analyzer only doesn't like indexed
sentences
Dear Lucene-developers,
I switched to using Lucene 3.5 a few weeks ago and suddenly sentences are
not correctly indexed anymore. Basically, fields can be correctly queried if
they contain one term but if there are multiple terms, the analyzer fails (i
use the latest Luke for testing).
So my query is: contains:foo
it works on a document with field
contains:foo
but fails on
contains:ba foo
For indexing, this is how I start the writer
KeywordAnalyzer analyzer = new KeywordAnalyzer();
Directory index = FSDirectory.open(output);
IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_35,
analyzer);
IndexWriter writer = null;
try {
writer = new IndexWriter(index, config);
} catch (IOException ex) {
ex.printStackTrace();
}
And the particular field is index like this:
doc.add(new Field("contains", text.toLowerCase(), Field.Store.YES,
Field.Index.ANALYZED));
Pretty rivial right?
Worked fine for my old (2.9) index but fails for 3.5... am I missing a very
important setting?
Thanks in advance!
EJ Blom
--
View this message in context: http://lucene.472066.n3.nabble.com/LUCENE-35-index-keyword-analyzer-only-doesn-t-like-indexed-sentences-tp3663508p3663508.html
Sent from the Lucene - Java Users mailing list archive at Nabble.com.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: LUCENE_35 index keyword analyzer only doesn't like indexed sentences
Posted by Ian Lea <ia...@gmail.com>.
Hard to believe this ever worked. KeywordAnalyzer '"Tokenizes" the
entire stream as a single token' i.e. there will only be one term. So
your document contains:ba foo would only match a search on ba foo, not
a search on foo. Are you sure you should be using KeywordAnalyzer?
Not usually used on sentences and StandardAnalyzer would appear to
make more sense here.
--
Ian.
On Mon, Jan 16, 2012 at 3:30 PM, ejblom <ej...@gmail.com> wrote:
>
> Dear Lucene-developers,
>
> I switched to using Lucene 3.5 a few weeks ago and suddenly sentences are
> not correctly indexed anymore. Basically, fields can be correctly queried if
> they contain one term but if there are multiple terms, the analyzer fails (i
> use the latest Luke for testing).
>
> So my query is: contains:foo
>
> it works on a document with field
> contains:foo
> but fails on
> contains:ba foo
>
> For indexing, this is how I start the writer
>
> KeywordAnalyzer analyzer = new KeywordAnalyzer();
> Directory index = FSDirectory.open(output);
> IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_35,
> analyzer);
>
> IndexWriter writer = null;
>
> try {
> writer = new IndexWriter(index, config);
> } catch (IOException ex) {
> ex.printStackTrace();
> }
>
> And the particular field is index like this:
>
> doc.add(new Field("contains", text.toLowerCase(), Field.Store.YES,
> Field.Index.ANALYZED));
>
> Pretty rivial right?
>
> Worked fine for my old (2.9) index but fails for 3.5... am I missing a very
> important setting?
>
> Thanks in advance!
>
> EJ Blom
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/LUCENE-35-index-keyword-analyzer-only-doesn-t-like-indexed-sentences-tp3663508p3663508.html
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org