Using multiple index searchers.

[Henrik Axelsson <sk...@gmail.com>]

Hi all,

After reading the FAQ I have a question regarding the use of multiple
indexes and thus IndexSearches on the one server.

I work on ecommerce websites and am looking at replacing our current method
of full text searching product descriptions and names with a Lucene
implementation. I envisaged creating a separate index file for each of the
sites running on our main webserver (about 10 sites, each with different
product listings). However this would mean that I would need to have many
instances of IndexSearcher open and potentially come across file handle
limit problems (as outlined in the FAQ) as well as consuming lots of memory.
Is this a valid concern? Would I be able to use Lucene this way?

An alternative would be to combine all the sites data into one index, and
have a field identifiying which site each product entry belonged to. However
I would rather not mix the data together.

Thanks,
Henrik

Re: Using multiple index searchers.

[Henrik Axelsson <sk...@gmail.com>]

Thanks for the quick reply, time to get to work on a prototype!

On Mon, Nov 24, 2008 at 2:12 PM, Yonik Seeley <yo...@apache.org> wrote:
> If the data is unrelated, separate indexes will lead to the best performance.
> Memory usage should be less or equal to one big index.
> File descriptor usage can be minimized by either calling optimize
> before opening a new IndexSearcher (depends on how often you want to
> see updates), lowering the merge factor, or using the compound file
> format.
>
> -Yonik
>
>
> On Sun, Nov 23, 2008 at 9:32 PM, Henrik Axelsson <sk...@gmail.com> wrote:
>> Hi all,
>>
>> After reading the FAQ I have a question regarding the use of multiple
>> indexes and thus IndexSearches on the one server.
>>
>> I work on ecommerce websites and am looking at replacing our current method
>> of full text searching product descriptions and names with a Lucene
>> implementation. I envisaged creating a separate index file for each of the
>> sites running on our main webserver (about 10 sites, each with different
>> product listings). However this would mean that I would need to have many
>> instances of IndexSearcher open and potentially come across file handle
>> limit problems (as outlined in the FAQ) as well as consuming lots of memory.
>> Is this a valid concern? Would I be able to use Lucene this way?
>>
>> An alternative would be to combine all the sites data into one index, and
>> have a field identifiying which site each product entry belonged to. However
>> I would rather not mix the data together.
>>
>> Thanks,
>> Henrik
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Using multiple index searchers.

[Yonik Seeley <yo...@apache.org>]

If the data is unrelated, separate indexes will lead to the best performance.
Memory usage should be less or equal to one big index.
File descriptor usage can be minimized by either calling optimize
before opening a new IndexSearcher (depends on how often you want to
see updates), lowering the merge factor, or using the compound file
format.

-Yonik


On Sun, Nov 23, 2008 at 9:32 PM, Henrik Axelsson <sk...@gmail.com> wrote:
> Hi all,
>
> After reading the FAQ I have a question regarding the use of multiple
> indexes and thus IndexSearches on the one server.
>
> I work on ecommerce websites and am looking at replacing our current method
> of full text searching product descriptions and names with a Lucene
> implementation. I envisaged creating a separate index file for each of the
> sites running on our main webserver (about 10 sites, each with different
> product listings). However this would mean that I would need to have many
> instances of IndexSearcher open and potentially come across file handle
> limit problems (as outlined in the FAQ) as well as consuming lots of memory.
> Is this a valid concern? Would I be able to use Lucene this way?
>
> An alternative would be to combine all the sites data into one index, and
> have a field identifiying which site each product entry belonged to. However
> I would rather not mix the data together.
>
> Thanks,
> Henrik
>

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org

Re: Using multiple index searchers.

[Henrik Axelsson <sk...@gmail.com>]

Hi all,

After reading the FAQ I have a question regarding the use of multiple
indexes and thus IndexSearches on the one server.

I work on ecommerce websites and am looking at replacing our current
method of full text searching product descriptions and names with a
Lucene implementation. I envisaged creating a separate index file for
each of the sites running on our main webserver (about 10 sites, each
with different product listings). However this would mean that I would
need to have many instances of IndexSearcher open and potentially come
across file handle limit problems (as outlined in the FAQ) as well as
consuming lots of memory. Is this a valid concern? Would I be able to
use Lucene this way?

An alternative would be to combine all the sites data into one index,
and have a field identifiying which site each product entry belonged
to. However I would rather not mix the data together.

Thanks,
Henrik

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org