You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Li Li <fa...@gmail.com> on 2010/08/26 09:24:16 UTC
instantiated contrib
I have about 70k document, the total indexed size is about 15MB(the
orginal text files' size).
dir=new RAMDirectory();
IndexWriter write=new IndexWriter(dir,...;
for(loop){
writer.addDocument(doc);
}
writer.optimize();
writer.close();
IndexReader ir=IndexReader.open(dir,true);
InstantiatedIndex ii=new InstantiatedIndex(ir);
InstantiatedIndexReader iir=new InstantiatedIndexReader(ii);
is=new IndexSearcher(ir);
is2=new IndexSearcher(iir);
I calculate the time by:
long searchStart=System.nanoTime();
TopDocs docs=is.search(bQuery,Integer.MAX_VALUE);
long searchEnd=System.nanoTime();
I searched 10,000 documents and the time of RAMDirectory
and instantiated
the time used is time1: 21s(21812978000 ns) time2:
20s(20713817000 ns)
I also calulate the time including get field value:
total1: 23852ms total2: 22610ms
it seems instantiated is not much faster than
RAMDirectory. Is there any thing wrong I used? my max memory is 4GB
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: instantiated contrib
Posted by Karl Wettin <ka...@gmail.com>.
Can you tell us what your queries are? Is it simple term queries,
phrases, fuzzy, etc?
I think the bad guy here is the term "hotel", that so many documents
contains it. You could try loading the full index to II and see how
long time it take to match just that term and compare. And then try a
more unique term.
If this indeed is the bad guy here are two solutions that might work
depending on your needs:
Replace all queries for "hotel" with a filter and remove "hotel" from
your term queries. This might decrease p/r for phrases etc.
Index and query using shingles to create more unique terms. This will
also improve p/r for phrases if you don't already do phrase queries
and if you do it should save you time at query time as you might no
longer need the phrase queries. But the index size will grow.
I'm sure there are other solutions too, but to be quite honest I think
your corpus is pushing the limit for when it make sense with an II. My
guess is that you can get it perhaps 2x faster than a RAMDirectory if
you tune it right.
karl
27 aug 2010 kl. 05.34 skrev Li Li:
> if I index only 7k documents, the time comparison:
> time1: 7602331019 time2: 4246878035 total1: 10736 total2: 7393
> it seems II is faster than RAMDirectory.
>
> My indexed texts are all hotel names (chinese and english, litter
> french).
> it has about 100k terms. terms such as hotel is very frequent and
> hotel name is very rare(exception Hotel Chain).
> so I guess it's distribution is a litter term is very frequent and
> other term is very rare.
>
> 2010/8/27 Karl Wettin <ka...@gmail.com>:
>> My mail client died while sending this mail.. Sorry for any
>> duplicate.
>>
>> It is strange that it should take 20 second to gather fields, this
>> is the
>> only thing that really suprises me. I'd expect it to be instant
>> compared to
>> RAMDirectory. It is hard to say from the information you provided.
>> Did you
>> perhaps lazy load field values from your RAMDirectory and not
>> retrieve them,
>> or something like that?
>>
>> Why your queries are slow is also hard to say, there can be many
>> reaons. 70k
>> documents can be quite a few documents for II if they contain
>> enough text.
>> Here are a few questions that may or may not be helpful:
>>
>> What is the content of the documents? Do they contain a lot of the
>> same
>> text? Or are they all rather unique? The major thing that makes II
>> faster
>> than RAMDirectory is that it does not have to deserialize values
>> from the
>> bytestream. As the index grows binary searching for documents
>> containing a
>> given term will start consume more time than deserializing the index.
>>
>> What speed do you see if you only load 10% (7k)?
>>
>> Did you see the graphics in the package level javadocs?
>> http://lucene.apache.org/java/3_0_2/api/all/org/apache/lucene/store/instantiated/package-summary.html
>>
>>
>> karl
>>
>>
>> 26 aug 2010 kl. 09.24 skrev Li Li:
>>
>>> I have about 70k document, the total indexed size is about 15MB(the
>>> orginal text files' size).
>>> dir=new RAMDirectory();
>>> IndexWriter write=new IndexWriter(dir,...;
>>> for(loop){
>>> writer.addDocument(doc);
>>> }
>>> writer.optimize();
>>> writer.close();
>>>
>>> IndexReader ir=IndexReader.open(dir,true);
>>> InstantiatedIndex ii=new InstantiatedIndex(ir);
>>> InstantiatedIndexReader iir=new InstantiatedIndexReader(ii);
>>> is=new IndexSearcher(ir);
>>> is2=new IndexSearcher(iir);
>>>
>>> I calculate the time by:
>>> long searchStart=System.nanoTime();
>>> TopDocs docs=is.search(bQuery,Integer.MAX_VALUE);
>>> long searchEnd=System.nanoTime();
>>>
>>> I searched 10,000 documents and the time of RAMDirectory
>>> and instantiated
>>> the time used is time1: 21s(21812978000 ns) time2:
>>> 20s(20713817000 ns)
>>> I also calulate the time including get field value:
>>> total1: 23852ms total2: 22610ms
>>> it seems instantiated is not much faster than
>>> RAMDirectory. Is there any thing wrong I used? my max memory is 4GB
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: instantiated contrib
Posted by Li Li <fa...@gmail.com>.
if I index only 7k documents, the time comparison:
time1: 7602331019 time2: 4246878035 total1: 10736 total2: 7393
it seems II is faster than RAMDirectory.
My indexed texts are all hotel names (chinese and english, litter french).
it has about 100k terms. terms such as hotel is very frequent and
hotel name is very rare(exception Hotel Chain).
so I guess it's distribution is a litter term is very frequent and
other term is very rare.
2010/8/27 Karl Wettin <ka...@gmail.com>:
> My mail client died while sending this mail.. Sorry for any duplicate.
>
> It is strange that it should take 20 second to gather fields, this is the
> only thing that really suprises me. I'd expect it to be instant compared to
> RAMDirectory. It is hard to say from the information you provided. Did you
> perhaps lazy load field values from your RAMDirectory and not retrieve them,
> or something like that?
>
> Why your queries are slow is also hard to say, there can be many reaons. 70k
> documents can be quite a few documents for II if they contain enough text.
> Here are a few questions that may or may not be helpful:
>
> What is the content of the documents? Do they contain a lot of the same
> text? Or are they all rather unique? The major thing that makes II faster
> than RAMDirectory is that it does not have to deserialize values from the
> bytestream. As the index grows binary searching for documents containing a
> given term will start consume more time than deserializing the index.
>
> What speed do you see if you only load 10% (7k)?
>
> Did you see the graphics in the package level javadocs?
> http://lucene.apache.org/java/3_0_2/api/all/org/apache/lucene/store/instantiated/package-summary.html
>
>
> karl
>
>
> 26 aug 2010 kl. 09.24 skrev Li Li:
>
>> I have about 70k document, the total indexed size is about 15MB(the
>> orginal text files' size).
>> dir=new RAMDirectory();
>> IndexWriter write=new IndexWriter(dir,...;
>> for(loop){
>> writer.addDocument(doc);
>> }
>> writer.optimize();
>> writer.close();
>>
>> IndexReader ir=IndexReader.open(dir,true);
>> InstantiatedIndex ii=new InstantiatedIndex(ir);
>> InstantiatedIndexReader iir=new InstantiatedIndexReader(ii);
>> is=new IndexSearcher(ir);
>> is2=new IndexSearcher(iir);
>>
>> I calculate the time by:
>> long searchStart=System.nanoTime();
>> TopDocs docs=is.search(bQuery,Integer.MAX_VALUE);
>> long searchEnd=System.nanoTime();
>>
>> I searched 10,000 documents and the time of RAMDirectory
>> and instantiated
>> the time used is time1: 21s(21812978000 ns) time2:
>> 20s(20713817000 ns)
>> I also calulate the time including get field value:
>> total1: 23852ms total2: 22610ms
>> it seems instantiated is not much faster than
>> RAMDirectory. Is there any thing wrong I used? my max memory is 4GB
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: instantiated contrib
Posted by Li Li <fa...@gmail.com>.
" It is strange that it should take 20 second to gather fields,"
20s including search and gather fields, it's the total time
2010/8/27 Karl Wettin <ka...@gmail.com>:
> My mail client died while sending this mail.. Sorry for any duplicate.
>
> It is strange that it should take 20 second to gather fields, this is the
> only thing that really suprises me. I'd expect it to be instant compared to
> RAMDirectory. It is hard to say from the information you provided. Did you
> perhaps lazy load field values from your RAMDirectory and not retrieve them,
> or something like that?
>
> Why your queries are slow is also hard to say, there can be many reaons. 70k
> documents can be quite a few documents for II if they contain enough text.
> Here are a few questions that may or may not be helpful:
>
> What is the content of the documents? Do they contain a lot of the same
> text? Or are they all rather unique? The major thing that makes II faster
> than RAMDirectory is that it does not have to deserialize values from the
> bytestream. As the index grows binary searching for documents containing a
> given term will start consume more time than deserializing the index.
>
> What speed do you see if you only load 10% (7k)?
>
> Did you see the graphics in the package level javadocs?
> http://lucene.apache.org/java/3_0_2/api/all/org/apache/lucene/store/instantiated/package-summary.html
>
>
> karl
>
>
> 26 aug 2010 kl. 09.24 skrev Li Li:
>
>> I have about 70k document, the total indexed size is about 15MB(the
>> orginal text files' size).
>> dir=new RAMDirectory();
>> IndexWriter write=new IndexWriter(dir,...;
>> for(loop){
>> writer.addDocument(doc);
>> }
>> writer.optimize();
>> writer.close();
>>
>> IndexReader ir=IndexReader.open(dir,true);
>> InstantiatedIndex ii=new InstantiatedIndex(ir);
>> InstantiatedIndexReader iir=new InstantiatedIndexReader(ii);
>> is=new IndexSearcher(ir);
>> is2=new IndexSearcher(iir);
>>
>> I calculate the time by:
>> long searchStart=System.nanoTime();
>> TopDocs docs=is.search(bQuery,Integer.MAX_VALUE);
>> long searchEnd=System.nanoTime();
>>
>> I searched 10,000 documents and the time of RAMDirectory
>> and instantiated
>> the time used is time1: 21s(21812978000 ns) time2:
>> 20s(20713817000 ns)
>> I also calulate the time including get field value:
>> total1: 23852ms total2: 22610ms
>> it seems instantiated is not much faster than
>> RAMDirectory. Is there any thing wrong I used? my max memory is 4GB
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: instantiated contrib
Posted by Karl Wettin <ka...@gmail.com>.
My mail client died while sending this mail.. Sorry for any duplicate.
It is strange that it should take 20 second to gather fields, this is
the only thing that really suprises me. I'd expect it to be instant
compared to RAMDirectory. It is hard to say from the information you
provided. Did you perhaps lazy load field values from your
RAMDirectory and not retrieve them, or something like that?
Why your queries are slow is also hard to say, there can be many
reaons. 70k documents can be quite a few documents for II if they
contain enough text. Here are a few questions that may or may not be
helpful:
What is the content of the documents? Do they contain a lot of the
same text? Or are they all rather unique? The major thing that makes
II faster than RAMDirectory is that it does not have to deserialize
values from the bytestream. As the index grows binary searching for
documents containing a given term will start consume more time than
deserializing the index.
What speed do you see if you only load 10% (7k)?
Did you see the graphics in the package level javadocs?
http://lucene.apache.org/java/3_0_2/api/all/org/apache/lucene/store/instantiated/package-summary.html
karl
26 aug 2010 kl. 09.24 skrev Li Li:
> I have about 70k document, the total indexed size is about 15MB(the
> orginal text files' size).
> dir=new RAMDirectory();
> IndexWriter write=new IndexWriter(dir,...;
> for(loop){
> writer.addDocument(doc);
> }
> writer.optimize();
> writer.close();
>
> IndexReader ir=IndexReader.open(dir,true);
> InstantiatedIndex ii=new InstantiatedIndex(ir);
> InstantiatedIndexReader iir=new InstantiatedIndexReader(ii);
> is=new IndexSearcher(ir);
> is2=new IndexSearcher(iir);
>
> I calculate the time by:
> long searchStart=System.nanoTime();
> TopDocs docs=is.search(bQuery,Integer.MAX_VALUE);
> long searchEnd=System.nanoTime();
>
> I searched 10,000 documents and the time of RAMDirectory
> and instantiated
> the time used is time1: 21s(21812978000 ns) time2:
> 20s(20713817000 ns)
> I also calulate the time including get field value:
> total1: 23852ms total2: 22610ms
> it seems instantiated is not much faster than
> RAMDirectory. Is there any thing wrong I used? my max memory is 4GB
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: instantiated contrib
Posted by Grant Ingersoll <gs...@apache.org>.
I think that fits with its design goals. It wasn't intended for a large number of docs like RAMDir, more for the case where one might use MemoryIndex, but has more than one doc, but not so many that you need a more compact representation.
-Grant
On Aug 26, 2010, at 4:11 PM, Earwin Burrfoot wrote:
> I meant exactly what I said. My opinion is that Instantiated puts more
> strain on GC compared to RAMDir. If that aspect is negligible for you
> (less docs, smaller heap) it quite probably runs faster, if not -
> after some point GC costs will dwarf any gains.
>
> 2010/8/26 Li Li <fa...@gmail.com>:
>> do you mean that InstantiatedIndex is not as fast as its document says?
>>
>> 2010/8/26 Earwin Burrfoot <ea...@gmail.com>:
>>> IMHO, Instantiated sucks GC-wise. Put more docs in it, do enough
>>> queries, and RAMDir eventually outperforms it.
>>>
>>> On Thu, Aug 26, 2010 at 11:24, Li Li <fa...@gmail.com> wrote:
>>>> I have about 70k document, the total indexed size is about 15MB(the
>>>> orginal text files' size).
>>>> dir=new RAMDirectory();
>>>> IndexWriter write=new IndexWriter(dir,...;
>>>> for(loop){
>>>> writer.addDocument(doc);
>>>> }
>>>> writer.optimize();
>>>> writer.close();
>>>>
>>>> IndexReader ir=IndexReader.open(dir,true);
>>>> InstantiatedIndex ii=new InstantiatedIndex(ir);
>>>> InstantiatedIndexReader iir=new InstantiatedIndexReader(ii);
>>>> is=new IndexSearcher(ir);
>>>> is2=new IndexSearcher(iir);
>>>>
>>>> I calculate the time by:
>>>> long searchStart=System.nanoTime();
>>>> TopDocs docs=is.search(bQuery,Integer.MAX_VALUE);
>>>> long searchEnd=System.nanoTime();
>>>>
>>>> I searched 10,000 documents and the time of RAMDirectory
>>>> and instantiated
>>>> the time used is time1: 21s(21812978000 ns) time2:
>>>> 20s(20713817000 ns)
>>>> I also calulate the time including get field value:
>>>> total1: 23852ms total2: 22610ms
>>>> it seems instantiated is not much faster than
>>>> RAMDirectory. Is there any thing wrong I used? my max memory is 4GB
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Kirill Zakharenko/Кирилл Захаренко (earwin@gmail.com)
>>> Phone: +7 (495) 683-567-4
>>> ICQ: 104465785
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
>
>
> --
> Kirill Zakharenko/Кирилл Захаренко (earwin@gmail.com)
> Phone: +7 (495) 683-567-4
> ICQ: 104465785
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
--------------------------
Grant Ingersoll
http://www.lucidimagination.com/
Search the Lucene ecosystem docs using Solr/Lucene:
http://www.lucidimagination.com/search
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: instantiated contrib
Posted by Earwin Burrfoot <ea...@gmail.com>.
I meant exactly what I said. My opinion is that Instantiated puts more
strain on GC compared to RAMDir. If that aspect is negligible for you
(less docs, smaller heap) it quite probably runs faster, if not -
after some point GC costs will dwarf any gains.
2010/8/26 Li Li <fa...@gmail.com>:
> do you mean that InstantiatedIndex is not as fast as its document says?
>
> 2010/8/26 Earwin Burrfoot <ea...@gmail.com>:
>> IMHO, Instantiated sucks GC-wise. Put more docs in it, do enough
>> queries, and RAMDir eventually outperforms it.
>>
>> On Thu, Aug 26, 2010 at 11:24, Li Li <fa...@gmail.com> wrote:
>>> I have about 70k document, the total indexed size is about 15MB(the
>>> orginal text files' size).
>>> dir=new RAMDirectory();
>>> IndexWriter write=new IndexWriter(dir,...;
>>> for(loop){
>>> writer.addDocument(doc);
>>> }
>>> writer.optimize();
>>> writer.close();
>>>
>>> IndexReader ir=IndexReader.open(dir,true);
>>> InstantiatedIndex ii=new InstantiatedIndex(ir);
>>> InstantiatedIndexReader iir=new InstantiatedIndexReader(ii);
>>> is=new IndexSearcher(ir);
>>> is2=new IndexSearcher(iir);
>>>
>>> I calculate the time by:
>>> long searchStart=System.nanoTime();
>>> TopDocs docs=is.search(bQuery,Integer.MAX_VALUE);
>>> long searchEnd=System.nanoTime();
>>>
>>> I searched 10,000 documents and the time of RAMDirectory
>>> and instantiated
>>> the time used is time1: 21s(21812978000 ns) time2:
>>> 20s(20713817000 ns)
>>> I also calulate the time including get field value:
>>> total1: 23852ms total2: 22610ms
>>> it seems instantiated is not much faster than
>>> RAMDirectory. Is there any thing wrong I used? my max memory is 4GB
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: dev-help@lucene.apache.org
>>>
>>>
>>
>>
>>
>> --
>> Kirill Zakharenko/Кирилл Захаренко (earwin@gmail.com)
>> Phone: +7 (495) 683-567-4
>> ICQ: 104465785
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
--
Kirill Zakharenko/Кирилл Захаренко (earwin@gmail.com)
Phone: +7 (495) 683-567-4
ICQ: 104465785
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: instantiated contrib
Posted by Li Li <fa...@gmail.com>.
do you mean that InstantiatedIndex is not as fast as its document says?
2010/8/26 Earwin Burrfoot <ea...@gmail.com>:
> IMHO, Instantiated sucks GC-wise. Put more docs in it, do enough
> queries, and RAMDir eventually outperforms it.
>
> On Thu, Aug 26, 2010 at 11:24, Li Li <fa...@gmail.com> wrote:
>> I have about 70k document, the total indexed size is about 15MB(the
>> orginal text files' size).
>> dir=new RAMDirectory();
>> IndexWriter write=new IndexWriter(dir,...;
>> for(loop){
>> writer.addDocument(doc);
>> }
>> writer.optimize();
>> writer.close();
>>
>> IndexReader ir=IndexReader.open(dir,true);
>> InstantiatedIndex ii=new InstantiatedIndex(ir);
>> InstantiatedIndexReader iir=new InstantiatedIndexReader(ii);
>> is=new IndexSearcher(ir);
>> is2=new IndexSearcher(iir);
>>
>> I calculate the time by:
>> long searchStart=System.nanoTime();
>> TopDocs docs=is.search(bQuery,Integer.MAX_VALUE);
>> long searchEnd=System.nanoTime();
>>
>> I searched 10,000 documents and the time of RAMDirectory
>> and instantiated
>> the time used is time1: 21s(21812978000 ns) time2:
>> 20s(20713817000 ns)
>> I also calulate the time including get field value:
>> total1: 23852ms total2: 22610ms
>> it seems instantiated is not much faster than
>> RAMDirectory. Is there any thing wrong I used? my max memory is 4GB
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: dev-help@lucene.apache.org
>>
>>
>
>
>
> --
> Kirill Zakharenko/Кирилл Захаренко (earwin@gmail.com)
> Phone: +7 (495) 683-567-4
> ICQ: 104465785
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org
Re: instantiated contrib
Posted by Earwin Burrfoot <ea...@gmail.com>.
IMHO, Instantiated sucks GC-wise. Put more docs in it, do enough
queries, and RAMDir eventually outperforms it.
On Thu, Aug 26, 2010 at 11:24, Li Li <fa...@gmail.com> wrote:
> I have about 70k document, the total indexed size is about 15MB(the
> orginal text files' size).
> dir=new RAMDirectory();
> IndexWriter write=new IndexWriter(dir,...;
> for(loop){
> writer.addDocument(doc);
> }
> writer.optimize();
> writer.close();
>
> IndexReader ir=IndexReader.open(dir,true);
> InstantiatedIndex ii=new InstantiatedIndex(ir);
> InstantiatedIndexReader iir=new InstantiatedIndexReader(ii);
> is=new IndexSearcher(ir);
> is2=new IndexSearcher(iir);
>
> I calculate the time by:
> long searchStart=System.nanoTime();
> TopDocs docs=is.search(bQuery,Integer.MAX_VALUE);
> long searchEnd=System.nanoTime();
>
> I searched 10,000 documents and the time of RAMDirectory
> and instantiated
> the time used is time1: 21s(21812978000 ns) time2:
> 20s(20713817000 ns)
> I also calulate the time including get field value:
> total1: 23852ms total2: 22610ms
> it seems instantiated is not much faster than
> RAMDirectory. Is there any thing wrong I used? my max memory is 4GB
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
> For additional commands, e-mail: dev-help@lucene.apache.org
>
>
--
Kirill Zakharenko/Кирилл Захаренко (earwin@gmail.com)
Phone: +7 (495) 683-567-4
ICQ: 104465785
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org