You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Manoj Kr. Sheoran" <ma...@daffodildb.com> on 2005/11/04 07:11:27 UTC
Searching + Sorting in 3 milion documents
Hi,
I am having 1,00,000 documents in a index but in near future it will be 3 million and more. I am somewhat concerned about the searhing process with this much number of document. I am giving order on some fields of documents. Could anybody tell be the expected result from lucene engine with 3 million results and order/sorted on 4 column.
Thanks,
Manoj
Re: Searching + Sorting in 3 milion documents
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On 4 Nov 2005, at 02:50, Manoj Kr. Sheoran wrote:
>> - state of the index (optimized vs. unoptimized)
>>
> Which one will be best for these sort of scenario? Optimized?
Optimized is always best for better search efficiency.
But practically speaking, how optimized an index is depends on how
you need to update it.
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Searching + Sorting in 3 milion documents
Posted by "Manoj Kr. Sheoran" <ma...@daffodildb.com>.
Dear Otis,
Thanks for prompt reply.
> - state of the index (optimized vs. unoptimized)
Which one will be best for these sort of scenario? Optimized?
> - amount of RAM (can your index fit into RAM?) and speed of disk
> - desired response time
Is there any OutOfMemory issue in case if RAM amout is not enough or it's
simple take help of o/s specific pages?
Regards,
Manoj
----- Original Message -----
From: "Otis Gospodnetic" <ot...@yahoo.com>
To: <ja...@lucene.apache.org>
Cc: "Manoj Kr. Sheoran" <ma...@daffodildb.com>
Sent: Friday, November 04, 2005 1:12 PM
Subject: Re: Searching + Sorting in 3 milion documents
> Redirecting to java-user (-dev is for developers of Lucene).
> @daffodildb.com, interesting :)
>
> It really depends on a number of factors:
> - types of Fields in your Documents
> - number of indexed Fields
> - types of queries
> - state of the index (optimized vs. unoptimized)
> - amount of RAM (can your index fit into RAM?) and speed of disk
> - desired response time
> - etc.
>
> So, the best way is to create a fake index with 3M documents on your
> server and see what happens.
>
>
> Otis
>
>
> --- "Manoj Kr. Sheoran" <ma...@daffodildb.com> wrote:
>
> > Hi,
> > I am having 1,00,000 documents in a index but in near future it will
> > be 3 million and more. I am somewhat concerned about the searhing
> > process with this much number of document. I am giving order on some
> > fields of documents. Could anybody tell be the expected result from
> > lucene engine with 3 million results and order/sorted on 4 column.
> >
> > Thanks,
> > Manoj
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org
Re: Searching + Sorting in 3 milion documents
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Redirecting to java-user (-dev is for developers of Lucene).
@daffodildb.com, interesting :)
It really depends on a number of factors:
- types of Fields in your Documents
- number of indexed Fields
- types of queries
- state of the index (optimized vs. unoptimized)
- amount of RAM (can your index fit into RAM?) and speed of disk
- desired response time
- etc.
So, the best way is to create a fake index with 3M documents on your
server and see what happens.
Otis
--- "Manoj Kr. Sheoran" <ma...@daffodildb.com> wrote:
> Hi,
> I am having 1,00,000 documents in a index but in near future it will
> be 3 million and more. I am somewhat concerned about the searhing
> process with this much number of document. I am giving order on some
> fields of documents. Could anybody tell be the expected result from
> lucene engine with 3 million results and order/sorted on 4 column.
>
> Thanks,
> Manoj
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org