Realtime updates solrcloud

[roySolr <ro...@gmail.com>]

Hello Guys,

I want to use the realtime updates mechanism of solrcloud. My setup is as
follow:

3 solr engines,
3 zookeeper instances(ensemble)

The setup works great, recovery, leader election etc.

The problem is the realtime updates, it's slow after the servers gets some
traffic.

I try to explain it:
I test the realtime update with the following command:

*curl http://SOLRURL:SOLRPORT/solr/update -H "Content-Type: text/xml"
--data-binary '<add><doc><field name="id">3504811</field><field
name="website">http://www.google.nl</add></doc>'*

I see this in logs of solr server:

*Mar 29, 2013 12:38:51 PM
org.apache.solr.update.processor.LogUpdateProcessor finish
INFO: [collection1] webapp=/solr path=/update params={} {add=[3504811
(1430841858290876416)]} 0 35 *

The other solr servers get the following lines in the log:

*INFO: [collection1] webapp=/solr path=/update
params={distrib.from=http://SOLRIP:SOLRPORT/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2}
{add=[3504811 (1430844456234385408)]} 0 14*

This looks good, the doc is added and the leader send this doc to the other
solr servers.

First times it takes 1 sec to make the update visible:)

When i send some traffic to the server(200q/s), the update takes +- 30 sec
to make it visible.
I stopped the traffic it's still takes 30 sec's to make the update visible.
How is it possible? The solrconfig parts:

*<autoCommit> 
     <maxTime>600000</maxTime> 
     <openSearcher>false</openSearcher> 
</autoCommit>

<autoSoftCommit> 
    <maxTime>2000</maxTime> 
</autoSoftCommit>*

Did i miss something?

Best Regards,
Roy



--
View this message in context: http://lucene.472066.n3.nabble.com/Realtime-updates-solrcloud-tp4052370.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Realtime updates solrcloud

[Erick Erickson <er...@gmail.com>]

Does the 30 second interval persist for a long time after you stop
your queries? It's possible that your requests are queueing up and you
have a bunch of search in the queue in front of the update....

Best
Erick

On Fri, Mar 29, 2013 at 8:33 AM, roySolr <ro...@gmail.com> wrote:
> Hello Guys,
>
> I want to use the realtime updates mechanism of solrcloud. My setup is as
> follow:
>
> 3 solr engines,
> 3 zookeeper instances(ensemble)
>
> The setup works great, recovery, leader election etc.
>
> The problem is the realtime updates, it's slow after the servers gets some
> traffic.
>
> I try to explain it:
> I test the realtime update with the following command:
>
> *curl http://SOLRURL:SOLRPORT/solr/update -H "Content-Type: text/xml"
> --data-binary '<add><doc><field name="id">3504811</field><field
> name="website">http://www.google.nl</add></doc>'*
>
> I see this in logs of solr server:
>
> *Mar 29, 2013 12:38:51 PM
> org.apache.solr.update.processor.LogUpdateProcessor finish
> INFO: [collection1] webapp=/solr path=/update params={} {add=[3504811
> (1430841858290876416)]} 0 35 *
>
> The other solr servers get the following lines in the log:
>
> *INFO: [collection1] webapp=/solr path=/update
> params={distrib.from=http://SOLRIP:SOLRPORT/solr/collection1/&update.distrib=FROMLEADER&wt=javabin&version=2}
> {add=[3504811 (1430844456234385408)]} 0 14*
>
> This looks good, the doc is added and the leader send this doc to the other
> solr servers.
>
> First times it takes 1 sec to make the update visible:)
>
> When i send some traffic to the server(200q/s), the update takes +- 30 sec
> to make it visible.
> I stopped the traffic it's still takes 30 sec's to make the update visible.
> How is it possible? The solrconfig parts:
>
> *<autoCommit>
>      <maxTime>600000</maxTime>
>      <openSearcher>false</openSearcher>
> </autoCommit>
>
> <autoSoftCommit>
>     <maxTime>2000</maxTime>
> </autoSoftCommit>*
>
> Did i miss something?
>
> Best Regards,
> Roy
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/Realtime-updates-solrcloud-tp4052370.html
> Sent from the Solr - User mailing list archive at Nabble.com.