Which server parameters to tweak in Solr if I expect heavy writes and light reads?

[Archon810 <ar...@gmail.com>]

I am facing scalability issues designing a new Solr cluster and I need to
master to be able to handle a relatively high rate of updates with almost no
reads - they can be done via slaves.

My existing Solr instance is occupying a huge amount of RAM, in fact it
started swapping at only 4.5mil docs. I am interested in making the
footprint as little as possible in RAM, even if it affects search
performance.

So, which Solr config values can I tweak in order to accomplish this?

Thank you.

P.S. Cross posted to
http://stackoverflow.com/questions/1280447/which-server-parameters-to-tweak-in-solr-if-i-expect-heavy-writes-and-light-reads
for additional help.
-- 
View this message in context: http://www.nabble.com/Which-server-parameters-to-tweak-in-Solr-if-I-expect-heavy-writes-and-light-reads--tp24979526p24979526.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Which server parameters to tweak in Solr if I expect heavy writes and light reads?

[Shalin Shekhar Mangar <sh...@gmail.com>]

On Sat, Aug 15, 2009 at 3:51 AM, Archon810 <ar...@gmail.com> wrote:

>
> I am facing scalability issues designing a new Solr cluster and I need to
> master to be able to handle a relatively high rate of updates with almost
> no
> reads - they can be done via slaves.
>

Is it ok for the new writes to be available at a later time to the
searchers?


>
> My existing Solr instance is occupying a huge amount of RAM, in fact it
> started swapping at only 4.5mil docs. I am interested in making the
> footprint as little as possible in RAM, even if it affects search
> performance.
>
> So, which Solr config values can I tweak in order to accomplish this?
>

I'm assuming you are using Solr 1.3

It has been reported that a higher ramBufferSizeMB will help speeding up
updates. You may not need to go  as high as mentioned in the thread below:

http://www.lucidimagination.com/search/document/9bd1af395b5eeedb/performance_tuning_segment_merge_index_update_5_1_timing#5fa698179d561922

Since there will be no reads on the master, you can disable caches (no
autowarming is needed). Remove any queries specified on
newSearcher/firstSearcher events. These will reduce the memory requirements.

Commit as late as possible (disable auto commit).

-- 
Regards,
Shalin Shekhar Mangar.

RE: Which server parameters to tweak in Solr if I expect heavy writes and light reads?

[Fuad Efendi <fu...@efendi.ca>]

In my personal experience: ramBufferSizeMB=8192 helps to keep many things in
RAM and to delay Index Merge forever (I have single segment 10G with almost
100 mlns docs after 24 hours)

Heavy I/O was a problem before, and I solved it


-----Original Message-----
From: Archon810 [mailto:archon810@gmail.com] 
Sent: August-14-09 6:21 PM
To: solr-user@lucene.apache.org
Subject: Which server parameters to tweak in Solr if I expect heavy writes
and light reads?


I am facing scalability issues designing a new Solr cluster and I need to
master to be able to handle a relatively high rate of updates with almost no
reads - they can be done via slaves.

My existing Solr instance is occupying a huge amount of RAM, in fact it
started swapping at only 4.5mil docs. I am interested in making the
footprint as little as possible in RAM, even if it affects search
performance.

So, which Solr config values can I tweak in order to accomplish this?

Thank you.

P.S. Cross posted to
http://stackoverflow.com/questions/1280447/which-server-parameters-to-tweak-
in-solr-if-i-expect-heavy-writes-and-light-reads
for additional help.
-- 
View this message in context:
http://www.nabble.com/Which-server-parameters-to-tweak-in-Solr-if-I-expect-h
eavy-writes-and-light-reads--tp24979526p24979526.html
Sent from the Solr - User mailing list archive at Nabble.com.