You are viewing a plain text version of this content. The canonical link for it is here.
Posted to java-user@lucene.apache.org by Marcus Falck <ma...@observer.se> on 2006/05/21 17:31:51 UTC
Date first best practises
Hi,
I will use Lucene to index 200 million documents (doc size 2kb -> 20 kb).
With the following requirements:
IndexSearcher needs to be created atleast every 5 minute.
The ranking/scoring/sorting will need to reply the hits ordered by date desc.
Will the sorting be good enough on a machine with 4GB of RAM? Since the cache will be destroyed every time I open a new IndexSercher i need to know how long it will take to rebuild it (the index is stored on a SAN).
Or do I have to change the scoring so I wont need to access the date field?
If i will have to change the scoring does anybody have any tips on where to begin?
/
Regards
Marcus
Re: Date first best practises
Posted by Erik Hatcher <er...@ehatchersolutions.com>.
On May 21, 2006, at 11:31 AM, Marcus Falck wrote:
> I will use Lucene to index 200 million documents (doc size 2kb ->
> 20 kb).
> With the following requirements:
> IndexSearcher needs to be created atleast every 5 minute.
> The ranking/scoring/sorting will need to reply the hits ordered by
> date desc.
>
> Will the sorting be good enough on a machine with 4GB of RAM? Since
> the cache will be destroyed every time I open a new IndexSercher i
> need to know how long it will take to rebuild it (the index is
> stored on a SAN).
My hunch is that sorting will be fine, but you'll want to warm up the
caches before putting a new IndexSearcher into production use. Solr
does this by running a configurable set of queries against an index
before swapping out the old IndexSearcher with a new warmed up one.
> Or do I have to change the scoring so I wont need to access the
> date field?
>
> If i will have to change the scoring does anybody have any tips on
> where to begin?
Solr's FunctionQuery (isn't it time we move this to the core of
Lucene?! :) might be able to help you out in this situation, but it
sounds like sorting is what you want to use, not scoring tweaks.
Erik
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org