You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by Otis Gospodnetic <ot...@yahoo.com> on 2009/08/01 08:37:22 UTC
Re: 99.9% uptime requirement
Robi,
Solr is indeed very stable. However, it can crash and I've seen it crash. Or rather, I should say I've seen the JVM that runs Solr crash. For instance, if you have a servlet container with a number of webapps, one of which is Solr, and one of which has a memory leak, I believe all webapps will suffer and "crash". And even if you have just Solr in your servlet container, it can OOM, say if you specify overly large caches or too frequent commits, etc.
Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
----- Original Message ----
> From: Robert Petersen <ro...@buy.com>
> To: solr-user@lucene.apache.org
> Sent: Friday, July 31, 2009 12:18:55 PM
> Subject: 99.9% uptime requirement
>
> Hi all,
>
> My solr project powers almost all the pages in our site and so needs to
> be up period. My question is what can I do to ensure that happens?
> Does solr ever crash, assuming reasonable load conditions and no extreme
> index sizes?
>
> I saw some comments about running solr under daemontools in order to get
> an auto-restart on crashes. From what I have seen so far in my limited
> experience, solr is very stable and never crashes (so far). Does anyone
> else have this requirement and if so how do they deal with it? Is
> anyone else running solr under daemontools in a production site?
>
> Thanks for any input you might have,
> Robi
Re: 99.9% uptime requirement
Posted by Walter Underwood <wu...@wunderwood.org>.
For 99.9%, run three copies behind a load balancer. That allows you to
take one down for upgrade, and still be fault-tolerant.
wunder
On Aug 3, 2009, at 10:46 AM, Robert Petersen wrote:
> So then would the 'right' thing to do be to run it under something
> like
> Daemontools so it bounces back up on a crash? Do any other people use
> this approach or is there something better to make it come back up?
>
> Speaking of overly large caches, if I have solr running on a machine
> with 8GB main memory is it going to hurt to make some huge cache
> sizes?
> Are these settings reasonable? With a small index I have been getting
> some great hit-rates.
> <ramBufferSizeMB>1024</ramBufferSizeMB>
>
> <filterCache class="solr.FastLRUCache" size="350000"
> initialSize="512" autowarmCount="80"/>
> <queryResultCache class="solr.LRUCache" size="512000000"
> initialSize="512" autowarmCount="80"/>
> <documentCache class="solr.FastLRUCache" size="512000"
> initialSize="512" autowarmCount="0"/>
>
> Thanks
> Robi
>
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
> Sent: Friday, July 31, 2009 11:37 PM
> To: solr-user@lucene.apache.org
> Subject: Re: 99.9% uptime requirement
>
> Robi,
>
> Solr is indeed very stable. However, it can crash and I've seen it
> crash. Or rather, I should say I've seen the JVM that runs Solr
> crash.
> For instance, if you have a servlet container with a number of
> webapps,
> one of which is Solr, and one of which has a memory leak, I believe
> all
> webapps will suffer and "crash". And even if you have just Solr in
> your
> servlet container, it can OOM, say if you specify overly large
> caches or
> too frequent commits, etc.
>
> Otis
> --
> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
>
>
>
> ----- Original Message ----
>> From: Robert Petersen <ro...@buy.com>
>> To: solr-user@lucene.apache.org
>> Sent: Friday, July 31, 2009 12:18:55 PM
>> Subject: 99.9% uptime requirement
>>
>> Hi all,
>>
>> My solr project powers almost all the pages in our site and so needs
> to
>> be up period. My question is what can I do to ensure that happens?
>> Does solr ever crash, assuming reasonable load conditions and no
> extreme
>> index sizes?
>>
>> I saw some comments about running solr under daemontools in order to
> get
>> an auto-restart on crashes. From what I have seen so far in my
> limited
>> experience, solr is very stable and never crashes (so far). Does
> anyone
>> else have this requirement and if so how do they deal with it? Is
>> anyone else running solr under daemontools in a production site?
>>
>> Thanks for any input you might have,
>> Robi
>
Re: 99.9% uptime requirement
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Thu, Aug 6, 2009 at 4:10 AM, Robert Petersen <ro...@buy.com> wrote:
> Maintenance Questions: In a two slave one master setup where the two
> slaves are behind load balancers what happens if I have to restart solr?
> If I have to restart solr say for a schema update where I have added a
> new field then what is the recommended procedure?
>
> If I can guarantee no commits or optimizes happen on the master during
> the schema update so no new snapshots become available then can I safely
> leave rsyncd enabled? When I stop and start a slave server, should I
> first pull it out of the load balancers list or will solr gracefully
> release connections as it shuts down so no searches are lost?
>
We pull slaves out of the load balancer, wait for 15-20 seconds and then
stop the tomcat process.
>
> What do you guys do to push out updates?
>
Disable the cron job on all slaves (which calls snappuller).
Update schema on master and re-index.
For each slave: Take it out of rotation, stop tomcat, update the schema,
start tomcat, call snappuller, start cron.
This is now a piece of cake with the java based replication in Solr 1.4
which supports replicating configuration too without downtime.
--
Regards,
Shalin Shekhar Mangar.
Re: 99.9% uptime requirement
Posted by Walter Underwood <wu...@wunderwood.org>.
Design so that you can handle the load with one server down (N+1
sizing), then take one server out for any maintenance. Simple and
works fine.
wunder
On Aug 6, 2009, at 9:25 AM, Robert Petersen wrote:
> Here is another idea. With solr multicore you can dynamically spin up
> extra cores and bring them online. I'm not sure how well this would
> work for us since we have hard coded the names of the cores we are
> hitting in our config files.
>
> -----Original Message-----
> From: Brian Klippel [mailto:brian@theport.com]
> Sent: Thursday, August 06, 2009 8:38 AM
> To: solr-user@lucene.apache.org
> Subject: RE: 99.9% uptime requirement
>
> You could create a new "working" core, then call the swap command once
> it is ready. Then remove the work core and delete the appropriate
> index
> folder at your convenience.
>
>
> -----Original Message-----
> From: Robert Petersen [mailto:robertpe@buy.com]
> Sent: Wednesday, August 05, 2009 6:41 PM
> To: solr-user@lucene.apache.org
> Subject: RE: 99.9% uptime requirement
>
> Maintenance Questions: In a two slave one master setup where the two
> slaves are behind load balancers what happens if I have to restart
> solr?
> If I have to restart solr say for a schema update where I have added a
> new field then what is the recommended procedure?
>
> If I can guarantee no commits or optimizes happen on the master during
> the schema update so no new snapshots become available then can I
> safely
> leave rsyncd enabled? When I stop and start a slave server, should I
> first pull it out of the load balancers list or will solr gracefully
> release connections as it shuts down so no searches are lost?
>
> What do you guys do to push out updates?
>
> Thanks for any thoughts,
> Robi
>
>
> -----Original Message-----
> From: Walter Underwood [mailto:wunder@wunderwood.org]
> Sent: Tuesday, August 04, 2009 8:57 AM
> To: solr-user@lucene.apache.org
> Subject: Re: 99.9% uptime requirement
>
> Right. You don't get to 99.9% by assuming that an 8 hour outage is OK.
> Design for continuous uptime, with plans for how long it takes to
> patch around a single point of failure. For example, if your load
> balancer is a single point of failure, make sure that you can redirect
> the front end servers to a single Solr server in much less than 8
> hours.
>
> Also, think about your SLA. Can the search index be more than 8 hours
> stale? How quickly do you need to be able to replace a failed indexing
> server? You might be able to run indexing locally on each search
> server if they are lightly loaded.
>
> wunder
>
> On Aug 4, 2009, at 7:11 AM, Norberto Meijome wrote:
>
>> On Mon, 3 Aug 2009 13:15:44 -0700
>> "Robert Petersen" <ro...@buy.com> wrote:
>>
>>> Thanks all, I figured there would be more talk about daemontools if
>>> there
>>> were really a need. I appreciate the input and for starters we'll
>>> put two
>>> slaves behind a load balancer and grow it from there.
>>>
>>
>> Robert,
>> not taking away from daemon tools, but daemon tools won't help you
>> if your
>> whole server goes down.
>>
>> don't put all your eggs in one basket - several
>> servers, load balancer (hardware load balancers x 2, haproxy, etc)
>>
>> and sure, use daemon tools to keep your services running within each
>> server...
>>
>> B
>> _________________________
>> {Beto|Norberto|Numard} Meijome
>>
>> "Why do you sit there looking like an envelope without any address
>> on it?"
>> Mark Twain
>>
>> I speak for myself, not my employer. Contents may be hot. Slippery
>> when wet.
>> Reading disclaimers makes you go blind. Writing them is worse. You
>> have been
>> Warned.
>>
>
RE: 99.9% uptime requirement
Posted by Robert Petersen <ro...@buy.com>.
Here is another idea. With solr multicore you can dynamically spin up
extra cores and bring them online. I'm not sure how well this would
work for us since we have hard coded the names of the cores we are
hitting in our config files.
-----Original Message-----
From: Brian Klippel [mailto:brian@theport.com]
Sent: Thursday, August 06, 2009 8:38 AM
To: solr-user@lucene.apache.org
Subject: RE: 99.9% uptime requirement
You could create a new "working" core, then call the swap command once
it is ready. Then remove the work core and delete the appropriate index
folder at your convenience.
-----Original Message-----
From: Robert Petersen [mailto:robertpe@buy.com]
Sent: Wednesday, August 05, 2009 6:41 PM
To: solr-user@lucene.apache.org
Subject: RE: 99.9% uptime requirement
Maintenance Questions: In a two slave one master setup where the two
slaves are behind load balancers what happens if I have to restart solr?
If I have to restart solr say for a schema update where I have added a
new field then what is the recommended procedure?
If I can guarantee no commits or optimizes happen on the master during
the schema update so no new snapshots become available then can I safely
leave rsyncd enabled? When I stop and start a slave server, should I
first pull it out of the load balancers list or will solr gracefully
release connections as it shuts down so no searches are lost?
What do you guys do to push out updates?
Thanks for any thoughts,
Robi
-----Original Message-----
From: Walter Underwood [mailto:wunder@wunderwood.org]
Sent: Tuesday, August 04, 2009 8:57 AM
To: solr-user@lucene.apache.org
Subject: Re: 99.9% uptime requirement
Right. You don't get to 99.9% by assuming that an 8 hour outage is OK.
Design for continuous uptime, with plans for how long it takes to
patch around a single point of failure. For example, if your load
balancer is a single point of failure, make sure that you can redirect
the front end servers to a single Solr server in much less than 8 hours.
Also, think about your SLA. Can the search index be more than 8 hours
stale? How quickly do you need to be able to replace a failed indexing
server? You might be able to run indexing locally on each search
server if they are lightly loaded.
wunder
On Aug 4, 2009, at 7:11 AM, Norberto Meijome wrote:
> On Mon, 3 Aug 2009 13:15:44 -0700
> "Robert Petersen" <ro...@buy.com> wrote:
>
>> Thanks all, I figured there would be more talk about daemontools if
>> there
>> were really a need. I appreciate the input and for starters we'll
>> put two
>> slaves behind a load balancer and grow it from there.
>>
>
> Robert,
> not taking away from daemon tools, but daemon tools won't help you
> if your
> whole server goes down.
>
> don't put all your eggs in one basket - several
> servers, load balancer (hardware load balancers x 2, haproxy, etc)
>
> and sure, use daemon tools to keep your services running within each
> server...
>
> B
> _________________________
> {Beto|Norberto|Numard} Meijome
>
> "Why do you sit there looking like an envelope without any address
> on it?"
> Mark Twain
>
> I speak for myself, not my employer. Contents may be hot. Slippery
> when wet.
> Reading disclaimers makes you go blind. Writing them is worse. You
> have been
> Warned.
>
RE: 99.9% uptime requirement
Posted by Brian Klippel <br...@theport.com>.
You could create a new "working" core, then call the swap command once
it is ready. Then remove the work core and delete the appropriate index
folder at your convenience.
-----Original Message-----
From: Robert Petersen [mailto:robertpe@buy.com]
Sent: Wednesday, August 05, 2009 6:41 PM
To: solr-user@lucene.apache.org
Subject: RE: 99.9% uptime requirement
Maintenance Questions: In a two slave one master setup where the two
slaves are behind load balancers what happens if I have to restart solr?
If I have to restart solr say for a schema update where I have added a
new field then what is the recommended procedure?
If I can guarantee no commits or optimizes happen on the master during
the schema update so no new snapshots become available then can I safely
leave rsyncd enabled? When I stop and start a slave server, should I
first pull it out of the load balancers list or will solr gracefully
release connections as it shuts down so no searches are lost?
What do you guys do to push out updates?
Thanks for any thoughts,
Robi
-----Original Message-----
From: Walter Underwood [mailto:wunder@wunderwood.org]
Sent: Tuesday, August 04, 2009 8:57 AM
To: solr-user@lucene.apache.org
Subject: Re: 99.9% uptime requirement
Right. You don't get to 99.9% by assuming that an 8 hour outage is OK.
Design for continuous uptime, with plans for how long it takes to
patch around a single point of failure. For example, if your load
balancer is a single point of failure, make sure that you can redirect
the front end servers to a single Solr server in much less than 8 hours.
Also, think about your SLA. Can the search index be more than 8 hours
stale? How quickly do you need to be able to replace a failed indexing
server? You might be able to run indexing locally on each search
server if they are lightly loaded.
wunder
On Aug 4, 2009, at 7:11 AM, Norberto Meijome wrote:
> On Mon, 3 Aug 2009 13:15:44 -0700
> "Robert Petersen" <ro...@buy.com> wrote:
>
>> Thanks all, I figured there would be more talk about daemontools if
>> there
>> were really a need. I appreciate the input and for starters we'll
>> put two
>> slaves behind a load balancer and grow it from there.
>>
>
> Robert,
> not taking away from daemon tools, but daemon tools won't help you
> if your
> whole server goes down.
>
> don't put all your eggs in one basket - several
> servers, load balancer (hardware load balancers x 2, haproxy, etc)
>
> and sure, use daemon tools to keep your services running within each
> server...
>
> B
> _________________________
> {Beto|Norberto|Numard} Meijome
>
> "Why do you sit there looking like an envelope without any address
> on it?"
> Mark Twain
>
> I speak for myself, not my employer. Contents may be hot. Slippery
> when wet.
> Reading disclaimers makes you go blind. Writing them is worse. You
> have been
> Warned.
>
RE: 99.9% uptime requirement
Posted by Robert Petersen <ro...@buy.com>.
Maintenance Questions: In a two slave one master setup where the two
slaves are behind load balancers what happens if I have to restart solr?
If I have to restart solr say for a schema update where I have added a
new field then what is the recommended procedure?
If I can guarantee no commits or optimizes happen on the master during
the schema update so no new snapshots become available then can I safely
leave rsyncd enabled? When I stop and start a slave server, should I
first pull it out of the load balancers list or will solr gracefully
release connections as it shuts down so no searches are lost?
What do you guys do to push out updates?
Thanks for any thoughts,
Robi
-----Original Message-----
From: Walter Underwood [mailto:wunder@wunderwood.org]
Sent: Tuesday, August 04, 2009 8:57 AM
To: solr-user@lucene.apache.org
Subject: Re: 99.9% uptime requirement
Right. You don't get to 99.9% by assuming that an 8 hour outage is OK.
Design for continuous uptime, with plans for how long it takes to
patch around a single point of failure. For example, if your load
balancer is a single point of failure, make sure that you can redirect
the front end servers to a single Solr server in much less than 8 hours.
Also, think about your SLA. Can the search index be more than 8 hours
stale? How quickly do you need to be able to replace a failed indexing
server? You might be able to run indexing locally on each search
server if they are lightly loaded.
wunder
On Aug 4, 2009, at 7:11 AM, Norberto Meijome wrote:
> On Mon, 3 Aug 2009 13:15:44 -0700
> "Robert Petersen" <ro...@buy.com> wrote:
>
>> Thanks all, I figured there would be more talk about daemontools if
>> there
>> were really a need. I appreciate the input and for starters we'll
>> put two
>> slaves behind a load balancer and grow it from there.
>>
>
> Robert,
> not taking away from daemon tools, but daemon tools won't help you
> if your
> whole server goes down.
>
> don't put all your eggs in one basket - several
> servers, load balancer (hardware load balancers x 2, haproxy, etc)
>
> and sure, use daemon tools to keep your services running within each
> server...
>
> B
> _________________________
> {Beto|Norberto|Numard} Meijome
>
> "Why do you sit there looking like an envelope without any address
> on it?"
> Mark Twain
>
> I speak for myself, not my employer. Contents may be hot. Slippery
> when wet.
> Reading disclaimers makes you go blind. Writing them is worse. You
> have been
> Warned.
>
Re: 99.9% uptime requirement
Posted by Walter Underwood <wu...@wunderwood.org>.
Right. You don't get to 99.9% by assuming that an 8 hour outage is OK.
Design for continuous uptime, with plans for how long it takes to
patch around a single point of failure. For example, if your load
balancer is a single point of failure, make sure that you can redirect
the front end servers to a single Solr server in much less than 8 hours.
Also, think about your SLA. Can the search index be more than 8 hours
stale? How quickly do you need to be able to replace a failed indexing
server? You might be able to run indexing locally on each search
server if they are lightly loaded.
wunder
On Aug 4, 2009, at 7:11 AM, Norberto Meijome wrote:
> On Mon, 3 Aug 2009 13:15:44 -0700
> "Robert Petersen" <ro...@buy.com> wrote:
>
>> Thanks all, I figured there would be more talk about daemontools if
>> there
>> were really a need. I appreciate the input and for starters we'll
>> put two
>> slaves behind a load balancer and grow it from there.
>>
>
> Robert,
> not taking away from daemon tools, but daemon tools won't help you
> if your
> whole server goes down.
>
> don't put all your eggs in one basket - several
> servers, load balancer (hardware load balancers x 2, haproxy, etc)
>
> and sure, use daemon tools to keep your services running within each
> server...
>
> B
> _________________________
> {Beto|Norberto|Numard} Meijome
>
> "Why do you sit there looking like an envelope without any address
> on it?"
> Mark Twain
>
> I speak for myself, not my employer. Contents may be hot. Slippery
> when wet.
> Reading disclaimers makes you go blind. Writing them is worse. You
> have been
> Warned.
>
Re: 99.9% uptime requirement
Posted by Norberto Meijome <nu...@gmail.com>.
On Mon, 3 Aug 2009 13:15:44 -0700
"Robert Petersen" <ro...@buy.com> wrote:
> Thanks all, I figured there would be more talk about daemontools if there
> were really a need. I appreciate the input and for starters we'll put two
> slaves behind a load balancer and grow it from there.
>
Robert,
not taking away from daemon tools, but daemon tools won't help you if your
whole server goes down.
don't put all your eggs in one basket - several
servers, load balancer (hardware load balancers x 2, haproxy, etc)
and sure, use daemon tools to keep your services running within each server...
B
_________________________
{Beto|Norberto|Numard} Meijome
"Why do you sit there looking like an envelope without any address on it?"
Mark Twain
I speak for myself, not my employer. Contents may be hot. Slippery when wet.
Reading disclaimers makes you go blind. Writing them is worse. You have been
Warned.
RE: 99.9% uptime requirement
Posted by Robert Petersen <ro...@buy.com>.
Thanks all, I figured there would be more talk about daemontools if there were really a need. I appreciate the input and for starters we'll put two slaves behind a load balancer and grow it from there.
Lovin' Solr So Far! We were using alta vista as our search engine... it was sooo 90's! haha
Thanks again,
Robi
-----Original Message-----
From: Rafał Kuć [mailto:rafal@alud.com.pl]
Sent: Monday, August 03, 2009 11:00 AM
To: solr-user@lucene.apache.org
Subject: Re: 99.9% uptime requirement
Hello!
Robert, from my experience with Solr (since 1.2 and running few 1.4 deployments) Solr does not need any kind of mechanism to ensure it will auto start on crash, because I didn`t see it crash on it`s own fault. Just ensure, You have not one instance of Solr, and run it behind a proxy or load balancer of some kind.
--
Regards,
Rafał Kuć
> So then would the 'right' thing to do be to run it under something like
> Daemontools so it bounces back up on a crash? Do any other people use
> this approach or is there something better to make it come back up?
> Speaking of overly large caches, if I have solr running on a machine
> with 8GB main memory is it going to hurt to make some huge cache sizes?
> Are these settings reasonable? With a small index I have been getting
> some great hit-rates.
> <ramBufferSizeMB>1024</ramBufferSizeMB>
> <filterCache class="solr.FastLRUCache" size="350000"
> initialSize="512" autowarmCount="80"/>
> <queryResultCache class="solr.LRUCache" size="512000000"
> initialSize="512" autowarmCount="80"/>
> <documentCache class="solr.FastLRUCache" size="512000"
> initialSize="512" autowarmCount="0"/>
> Thanks
> Robi
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
> Sent: Friday, July 31, 2009 11:37 PM
> To: solr-user@lucene.apache.org
> Subject: Re: 99.9% uptime requirement
> Robi,
> Solr is indeed very stable. However, it can crash and I've seen it
> crash. Or rather, I should say I've seen the JVM that runs Solr crash.
> For instance, if you have a servlet container with a number of webapps,
> one of which is Solr, and one of which has a memory leak, I believe all
> webapps will suffer and "crash". And even if you have just Solr in your
> servlet container, it can OOM, say if you specify overly large caches or
> too frequent commits, etc.
> Otis
> --
> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
> ----- Original Message ----
>> From: Robert Petersen <ro...@buy.com>
>> To: solr-user@lucene.apache.org
>> Sent: Friday, July 31, 2009 12:18:55 PM
>> Subject: 99.9% uptime requirement
>> Hi all,
>> My solr project powers almost all the pages in our site and so needs
> to
>> be up period. My question is what can I do to ensure that happens?
>> Does solr ever crash, assuming reasonable load conditions and no
> extreme
>> index sizes?
>> I saw some comments about running solr under daemontools in order to
> get
>> an auto-restart on crashes. From what I have seen so far in my
> limited
>> experience, solr is very stable and never crashes (so far). Does
> anyone
>> else have this requirement and if so how do they deal with it? Is
>> anyone else running solr under daemontools in a production site?
>> Thanks for any input you might have,
>> Robi
Re: 99.9% uptime requirement
Posted by Rafał Kuć <ra...@alud.com.pl>.
Hello!
Robert, from my experience with Solr (since 1.2 and running few 1.4 deployments) Solr does not need any kind of mechanism to ensure it will auto start on crash, because I didn`t see it crash on it`s own fault. Just ensure, You have not one instance of Solr, and run it behind a proxy or load balancer of some kind.
--
Regards,
Rafał Kuć
> So then would the 'right' thing to do be to run it under something like
> Daemontools so it bounces back up on a crash? Do any other people use
> this approach or is there something better to make it come back up?
> Speaking of overly large caches, if I have solr running on a machine
> with 8GB main memory is it going to hurt to make some huge cache sizes?
> Are these settings reasonable? With a small index I have been getting
> some great hit-rates.
> <ramBufferSizeMB>1024</ramBufferSizeMB>
> <filterCache class="solr.FastLRUCache" size="350000"
> initialSize="512" autowarmCount="80"/>
> <queryResultCache class="solr.LRUCache" size="512000000"
> initialSize="512" autowarmCount="80"/>
> <documentCache class="solr.FastLRUCache" size="512000"
> initialSize="512" autowarmCount="0"/>
> Thanks
> Robi
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
> Sent: Friday, July 31, 2009 11:37 PM
> To: solr-user@lucene.apache.org
> Subject: Re: 99.9% uptime requirement
> Robi,
> Solr is indeed very stable. However, it can crash and I've seen it
> crash. Or rather, I should say I've seen the JVM that runs Solr crash.
> For instance, if you have a servlet container with a number of webapps,
> one of which is Solr, and one of which has a memory leak, I believe all
> webapps will suffer and "crash". And even if you have just Solr in your
> servlet container, it can OOM, say if you specify overly large caches or
> too frequent commits, etc.
> Otis
> --
> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
> ----- Original Message ----
>> From: Robert Petersen <ro...@buy.com>
>> To: solr-user@lucene.apache.org
>> Sent: Friday, July 31, 2009 12:18:55 PM
>> Subject: 99.9% uptime requirement
>> Hi all,
>> My solr project powers almost all the pages in our site and so needs
> to
>> be up period. My question is what can I do to ensure that happens?
>> Does solr ever crash, assuming reasonable load conditions and no
> extreme
>> index sizes?
>> I saw some comments about running solr under daemontools in order to
> get
>> an auto-restart on crashes. From what I have seen so far in my
> limited
>> experience, solr is very stable and never crashes (so far). Does
> anyone
>> else have this requirement and if so how do they deal with it? Is
>> anyone else running solr under daemontools in a production site?
>> Thanks for any input you might have,
>> Robi
Re: 99.9% uptime requirement
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Yes, daemontools or any kind of home-grown process-watching-and-restarting tool will work.
Regarding those caches - they look too large.
Also, the ramBufferSizeMB is irrelevant on search slaves.
Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
----- Original Message ----
> From: Robert Petersen <ro...@buy.com>
> To: solr-user@lucene.apache.org
> Sent: Monday, August 3, 2009 1:46:21 PM
> Subject: RE: 99.9% uptime requirement
>
> So then would the 'right' thing to do be to run it under something like
> Daemontools so it bounces back up on a crash? Do any other people use
> this approach or is there something better to make it come back up?
>
> Speaking of overly large caches, if I have solr running on a machine
> with 8GB main memory is it going to hurt to make some huge cache sizes?
> Are these settings reasonable? With a small index I have been getting
> some great hit-rates.
> 1024
>
>
> initialSize="512" autowarmCount="80"/>
>
> initialSize="512" autowarmCount="80"/>
>
> initialSize="512" autowarmCount="0"/>
>
> Thanks
> Robi
>
> -----Original Message-----
> From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
> Sent: Friday, July 31, 2009 11:37 PM
> To: solr-user@lucene.apache.org
> Subject: Re: 99.9% uptime requirement
>
> Robi,
>
> Solr is indeed very stable. However, it can crash and I've seen it
> crash. Or rather, I should say I've seen the JVM that runs Solr crash.
> For instance, if you have a servlet container with a number of webapps,
> one of which is Solr, and one of which has a memory leak, I believe all
> webapps will suffer and "crash". And even if you have just Solr in your
> servlet container, it can OOM, say if you specify overly large caches or
> too frequent commits, etc.
>
> Otis
> --
> Sematext is hiring -- http://sematext.com/about/jobs.html?mls
> Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
>
>
>
> ----- Original Message ----
> > From: Robert Petersen
> > To: solr-user@lucene.apache.org
> > Sent: Friday, July 31, 2009 12:18:55 PM
> > Subject: 99.9% uptime requirement
> >
> > Hi all,
> >
> > My solr project powers almost all the pages in our site and so needs
> to
> > be up period. My question is what can I do to ensure that happens?
> > Does solr ever crash, assuming reasonable load conditions and no
> extreme
> > index sizes?
> >
> > I saw some comments about running solr under daemontools in order to
> get
> > an auto-restart on crashes. From what I have seen so far in my
> limited
> > experience, solr is very stable and never crashes (so far). Does
> anyone
> > else have this requirement and if so how do they deal with it? Is
> > anyone else running solr under daemontools in a production site?
> >
> > Thanks for any input you might have,
> > Robi
RE: 99.9% uptime requirement
Posted by Robert Petersen <ro...@buy.com>.
So then would the 'right' thing to do be to run it under something like
Daemontools so it bounces back up on a crash? Do any other people use
this approach or is there something better to make it come back up?
Speaking of overly large caches, if I have solr running on a machine
with 8GB main memory is it going to hurt to make some huge cache sizes?
Are these settings reasonable? With a small index I have been getting
some great hit-rates.
<ramBufferSizeMB>1024</ramBufferSizeMB>
<filterCache class="solr.FastLRUCache" size="350000"
initialSize="512" autowarmCount="80"/>
<queryResultCache class="solr.LRUCache" size="512000000"
initialSize="512" autowarmCount="80"/>
<documentCache class="solr.FastLRUCache" size="512000"
initialSize="512" autowarmCount="0"/>
Thanks
Robi
-----Original Message-----
From: Otis Gospodnetic [mailto:otis_gospodnetic@yahoo.com]
Sent: Friday, July 31, 2009 11:37 PM
To: solr-user@lucene.apache.org
Subject: Re: 99.9% uptime requirement
Robi,
Solr is indeed very stable. However, it can crash and I've seen it
crash. Or rather, I should say I've seen the JVM that runs Solr crash.
For instance, if you have a servlet container with a number of webapps,
one of which is Solr, and one of which has a memory leak, I believe all
webapps will suffer and "crash". And even if you have just Solr in your
servlet container, it can OOM, say if you specify overly large caches or
too frequent commits, etc.
Otis
--
Sematext is hiring -- http://sematext.com/about/jobs.html?mls
Lucene, Solr, Nutch, Katta, Hadoop, HBase, UIMA, NLP, NER, IR
----- Original Message ----
> From: Robert Petersen <ro...@buy.com>
> To: solr-user@lucene.apache.org
> Sent: Friday, July 31, 2009 12:18:55 PM
> Subject: 99.9% uptime requirement
>
> Hi all,
>
> My solr project powers almost all the pages in our site and so needs
to
> be up period. My question is what can I do to ensure that happens?
> Does solr ever crash, assuming reasonable load conditions and no
extreme
> index sizes?
>
> I saw some comments about running solr under daemontools in order to
get
> an auto-restart on crashes. From what I have seen so far in my
limited
> experience, solr is very stable and never crashes (so far). Does
anyone
> else have this requirement and if so how do they deal with it? Is
> anyone else running solr under daemontools in a production site?
>
> Thanks for any input you might have,
> Robi