You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by gunjan_versata <gu...@gmail.com> on 2010/02/25 13:04:07 UTC
SolrJ commit options
Hi,
We are using SolrJ to handle commits to our solr server.. All runs fine..
But whenever the commit happens, the server becomes slow and stops
responding.. therby resulting in TimeOut errors on our production. We are
using the default commit with waitFlush = true, waitSearcher = true...
Can I change there values so that the requests coming to solr dont block on
recent commit?? Also, what will be the impact of changing these values??
Regards,
Gunjan
--
View this message in context: http://old.nabble.com/SolrJ-commit-options-tp27714405p27714405.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrJ commit options
Posted by Chris Hostetter <ho...@fucit.org>.
: One technique to control commit times is to do automatic commits: you
: can configure a core to commit every N seconds (really milliseconds,
: but less than 5 minutes becomes difficult) and/or every N documents.
: This promotes a more fixed amount of work per commit.
...but increaseing commit frequency only really helps you if the slowdown
you are seeing is coming from the actaul commit -- if it's coming from
really resource intensive cache warming that chews up all the CPU then it
can just make the problem worse -- likewise, if you don't have any
warming, the perception of "slow/stoped queries during/after commit" can
sometimes come from time spent initializing FieldCaches (particularly if
there is one or two fields that almost all queries sort on )
the long and short being: performance issues can be caused by a great many
differnet things, so you really need to figure out what exactly is going
on during these "slow" periods in order to dtermine the best way to deal
with it.
-Hoss
Re: SolrJ commit options
Posted by Lance Norskog <go...@gmail.com>.
One technique to control commit times is to do automatic commits: you
can configure a core to commit every N seconds (really milliseconds,
but less than 5 minutes becomes difficult) and/or every N documents.
This promotes a more fixed amount of work per commit.
Also, the maxMergeDocs parameter lets you force a maximum segment size
(in documents). This may cap the longest possible commit times.
http://www.lucidimagination.com/search/document/CDRG_ch08_8.1.2.3?q=maxMergeDocs
On Fri, Mar 5, 2010 at 2:57 PM, Otis Gospodnetic
<ot...@yahoo.com> wrote:
> Jerry,
>
> This is why people often do index modifications on one server (master) and replicate the read-only index to 1+ different servers (slaves).
> If you do that, does the problem go away?
>
> Otis
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
> Hadoop ecosystem search :: http://search-hadoop.com/
>
>
>
> ----- Original Message ----
>> From: Jerome L Quinn <jl...@us.ibm.com>
>> To: solr-user@lucene.apache.org
>> Sent: Fri, March 5, 2010 10:13:03 AM
>> Subject: Re: SolrJ commit options
>>
>> Shalin Shekhar Mangar wrote on 02/25/2010 07:38:39
>> AM:
>>
>> > On Thu, Feb 25, 2010 at 5:34 PM, gunjan_versata
>> wrote:
>> >
>> > >
>> > > We are using SolrJ to handle commits to our solr server.. All runs
>> fine..
>> > > But whenever the commit happens, the server becomes slow and stops
>> > > responding.. therby resulting in TimeOut errors on our production. We
>> are
>> > > using the default commit with waitFlush = true, waitSearcher = true...
>> > >
>> > > Can I change there values so that the requests coming to solr dont
>> block on
>> > > recent commit?? Also, what will be the impact of changing these
>> values??
>> > >
>> >
>> > Solr does not block reads during a commit/optimize. Write operations are
>> > queued up but they are still accepted. Are you using the same Solr server
>> > for reads as well as writes?
>>
>> I've seen similar things with Solr 1.3 (not using SolrJ). If I try to
>> optimize the
>> index, queries will take much longer - easily a minute or more, resulting
>> in timeouts.
>>
>> Jerry
>
>
--
Lance Norskog
goksron@gmail.com
Re: SolrJ commit options
Posted by Otis Gospodnetic <ot...@yahoo.com>.
Jerry,
This is why people often do index modifications on one server (master) and replicate the read-only index to 1+ different servers (slaves).
If you do that, does the problem go away?
Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
Hadoop ecosystem search :: http://search-hadoop.com/
----- Original Message ----
> From: Jerome L Quinn <jl...@us.ibm.com>
> To: solr-user@lucene.apache.org
> Sent: Fri, March 5, 2010 10:13:03 AM
> Subject: Re: SolrJ commit options
>
> Shalin Shekhar Mangar wrote on 02/25/2010 07:38:39
> AM:
>
> > On Thu, Feb 25, 2010 at 5:34 PM, gunjan_versata
> wrote:
> >
> > >
> > > We are using SolrJ to handle commits to our solr server.. All runs
> fine..
> > > But whenever the commit happens, the server becomes slow and stops
> > > responding.. therby resulting in TimeOut errors on our production. We
> are
> > > using the default commit with waitFlush = true, waitSearcher = true...
> > >
> > > Can I change there values so that the requests coming to solr dont
> block on
> > > recent commit?? Also, what will be the impact of changing these
> values??
> > >
> >
> > Solr does not block reads during a commit/optimize. Write operations are
> > queued up but they are still accepted. Are you using the same Solr server
> > for reads as well as writes?
>
> I've seen similar things with Solr 1.3 (not using SolrJ). If I try to
> optimize the
> index, queries will take much longer - easily a minute or more, resulting
> in timeouts.
>
> Jerry
Re: SolrJ commit options
Posted by Jerome L Quinn <jl...@us.ibm.com>.
Shalin Shekhar Mangar <sh...@gmail.com> wrote on 02/25/2010 07:38:39
AM:
> On Thu, Feb 25, 2010 at 5:34 PM, gunjan_versata
<gu...@gmail.com>wrote:
>
> >
> > We are using SolrJ to handle commits to our solr server.. All runs
fine..
> > But whenever the commit happens, the server becomes slow and stops
> > responding.. therby resulting in TimeOut errors on our production. We
are
> > using the default commit with waitFlush = true, waitSearcher = true...
> >
> > Can I change there values so that the requests coming to solr dont
block on
> > recent commit?? Also, what will be the impact of changing these
values??
> >
>
> Solr does not block reads during a commit/optimize. Write operations are
> queued up but they are still accepted. Are you using the same Solr server
> for reads as well as writes?
I've seen similar things with Solr 1.3 (not using SolrJ). If I try to
optimize the
index, queries will take much longer - easily a minute or more, resulting
in timeouts.
Jerry
Re: SolrJ commit options
Posted by Shalin Shekhar Mangar <sh...@gmail.com>.
On Thu, Feb 25, 2010 at 5:34 PM, gunjan_versata <gu...@gmail.com>wrote:
>
> We are using SolrJ to handle commits to our solr server.. All runs fine..
> But whenever the commit happens, the server becomes slow and stops
> responding.. therby resulting in TimeOut errors on our production. We are
> using the default commit with waitFlush = true, waitSearcher = true...
>
> Can I change there values so that the requests coming to solr dont block on
> recent commit?? Also, what will be the impact of changing these values??
>
Solr does not block reads during a commit/optimize. Write operations are
queued up but they are still accepted. Are you using the same Solr server
for reads as well as writes?
--
Regards,
Shalin Shekhar Mangar.
Re: SolrJ commit options
Posted by Lance Norskog <go...@gmail.com>.
waitFlush=true means that the commit HTTP call waits until everything
is sent to disk before it returns.
waitSearcher=true means that the commit HTTP call waits until Solr has
reloaded the index and is ready to search against it. (For more, study
Solr warming up.)
Both of these mean that the HTTP call (or curl program or Solrj
program) that started the commit, waits until it is done. Other
processes doing searches against the index are not blocked. However,
the commit may have so much disk activity that the other searches do
not proceeed very fast. They are not completely blocked.
The commit will take as long as it takes, and your results will appear
after that. If you want to time that, use
waitFlush=true&waitSearcher=true.
On Fri, Mar 5, 2010 at 9:39 PM, gunjan_versata <gu...@gmail.com> wrote:
>
> But can anyone explain me the use of these parameters.. I have read upon it..
> what i could not understand was.. if can i set both the params to false,
> after how much time will my changes start reflecting?
>
> --
> View this message in context: http://old.nabble.com/SolrJ-commit-options-tp27714405p27802041.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
--
Lance Norskog
goksron@gmail.com
Re: SolrJ commit options
Posted by gunjan_versata <gu...@gmail.com>.
But can anyone explain me the use of these parameters.. I have read upon it..
what i could not understand was.. if can i set both the params to false,
after how much time will my changes start reflecting?
--
View this message in context: http://old.nabble.com/SolrJ-commit-options-tp27714405p27802041.html
Sent from the Solr - User mailing list archive at Nabble.com.