You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Cam Bazz <ca...@gmail.com> on 2008/09/15 18:45:01 UTC
instantiated index in 2.4
Hello,
I have been looking at instantiated index in the trunk. Does this come
with a searcher? Are the adds reflected directly to the index?
Or is it just an experimental thing only with reader and writer?
Best.
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: instantiated index in 2.4
Posted by Karl Wettin <ka...@gmail.com>.
Hi Darren,
How large is your corpus? The speed you can expect depends on how much
data you load it with. There is a graph in the package level javadocs
that shows this:
http://lucene.apache.org/java/2_4_0/api/contrib-instantiated/org/apache/lucene/store/instantiated/package-summary.html
karl
27 okt 2008 kl. 20.09 skrev Darren Govoni:
> Has anyone gotten some initial performance observations about
> instantiated index?
>
> I replaced my RAMDirectory searcher with one and it was slower or
> about
> the same. The note about it claims 100x possible performance
> improvement. Maybe there is a data size beyond which its performance
> excels.
>
> thanks.
>
> On Mon, 2008-09-15 at 19:45 +0300, Cam Bazz wrote:
>> Hello,
>>
>> I have been looking at instantiated index in the trunk. Does this
>> come
>> with a searcher? Are the adds reflected directly to the index?
>>
>> Or is it just an experimental thing only with reader and writer?
>>
>> Best.
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: instantiated index in 2.4
Posted by Darren Govoni <da...@ontrenet.com>.
Has anyone gotten some initial performance observations about
instantiated index?
I replaced my RAMDirectory searcher with one and it was slower or about
the same. The note about it claims 100x possible performance
improvement. Maybe there is a data size beyond which its performance
excels.
thanks.
On Mon, 2008-09-15 at 19:45 +0300, Cam Bazz wrote:
> Hello,
>
> I have been looking at instantiated index in the trunk. Does this come
> with a searcher? Are the adds reflected directly to the index?
>
> Or is it just an experimental thing only with reader and writer?
>
> Best.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: instantiated index in 2.4
Posted by Cam Bazz <ca...@gmail.com>.
Hello Karl;
This is good good good news. It works.
However, I added a document like
doc.add(new Field("f", "a", Field.Store.YES,
Field.Index.NOT_ANALYZED_NO_NORMS));
and then searched. The score is 0.3~ for the found document. should
not it be 1.0?
also it will find when searched for "f","b" only with a lower score.
Best.
On Mon, Sep 15, 2008 at 11:06 PM, Karl Wettin <ka...@gmail.com> wrote:
>
> 15 sep 2008 kl. 18.51 skrev Karl Wettin:
>>
>>> Are the adds reflected directly to the index?
>>
>> Yes. An InstantiatedIndexReader is always current.
>> You will probably still have to reconstruct your searcher.
>> I never really looked in to what happends if you don't.
>
> The second statement was wrong. There is no need to reconstruct the
> searcher, new documents are immediatly available as search results after a
> commit:
>
> public class TestSearch extends TestCase {
>
> public void test() throws Exception {
>
> InstantiatedIndex index = new InstantiatedIndex();
> InstantiatedIndexReader reader = new InstantiatedIndexReader(index);
> IndexSearcher searcher = new IndexSearcher(reader);
> InstantiatedIndexWriter writer = new InstantiatedIndexWriter(index);
>
> Document doc;
> Collector collector;
>
> doc = new Document();
> doc.add(new Field("f", "a", Field.Store.NO, Field.Index.NOT_ANALYZED));
> writer.addDocument(doc);
> writer.commit();
>
> collector = new Collector();
> searcher.search(new TermQuery(new Term("f", "a")), collector);
> assertEquals(1, collector.hits);
>
> doc = new Document();
> doc.add(new Field("f", "a", Field.Store.NO, Field.Index.NOT_ANALYZED));
> writer.addDocument(doc);
> writer.commit();
>
> collector = new Collector();
> searcher.search(new TermQuery(new Term("f", "a")), collector);
> assertEquals(2, collector.hits);
>
> }
>
> public static class Collector extends HitCollector {
> private int hits = 0;
> public void collect(int doc, float score) {
> hits++;
> }
> }
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: instantiated index in 2.4
Posted by Karl Wettin <ka...@gmail.com>.
15 sep 2008 kl. 18.51 skrev Karl Wettin:
>
>> Are the adds reflected directly to the index?
>
> Yes. An InstantiatedIndexReader is always current.
> You will probably still have to reconstruct your searcher.
> I never really looked in to what happends if you don't.
The second statement was wrong. There is no need to reconstruct the
searcher, new documents are immediatly available as search results
after a commit:
public class TestSearch extends TestCase {
public void test() throws Exception {
InstantiatedIndex index = new InstantiatedIndex();
InstantiatedIndexReader reader = new
InstantiatedIndexReader(index);
IndexSearcher searcher = new IndexSearcher(reader);
InstantiatedIndexWriter writer = new
InstantiatedIndexWriter(index);
Document doc;
Collector collector;
doc = new Document();
doc.add(new Field("f", "a", Field.Store.NO,
Field.Index.NOT_ANALYZED));
writer.addDocument(doc);
writer.commit();
collector = new Collector();
searcher.search(new TermQuery(new Term("f", "a")), collector);
assertEquals(1, collector.hits);
doc = new Document();
doc.add(new Field("f", "a", Field.Store.NO,
Field.Index.NOT_ANALYZED));
writer.addDocument(doc);
writer.commit();
collector = new Collector();
searcher.search(new TermQuery(new Term("f", "a")), collector);
assertEquals(2, collector.hits);
}
public static class Collector extends HitCollector {
private int hits = 0;
public void collect(int doc, float score) {
hits++;
}
}
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: instantiated index in 2.4
Posted by Karl Wettin <ka...@gmail.com>.
15 sep 2008 kl. 18.45 skrev Cam Bazz:
> I have been looking at instantiated index in the trunk. Does this come
> with a searcher?
Pass an InstantiatedIndexReader to the constructor of an IndexSearcher.
> Are the adds reflected directly to the index?
Yes. An InstantiatedIndexReader is always current. You will probably
still have to reconstruct your searcher. I never really looked in to
what happends if you don't.
Recommended use is that you consider an InstantiatedIndex immutable,
mainly because it is lockless.
karl
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org