You are viewing a plain text version of this content. The canonical link for it is here.
Posted to solr-user@lucene.apache.org by stephon <st...@gmail.com> on 2014/11/28 09:03:51 UTC
SolrCloud replica always fully resync index from leader node
I have an SolrCloud core with 4 shards, and replication factor is 1.
mentioned below: * coreA_shard1_replica1 * coreA_shard2_replica1 *
coreA_shard3_replica1 * coreA_shard4_replica1
After added the new replica of coreA_shard1, i.e.: coreA_shard1_replica2. it
will do fully resync from the leader node (coreA_shard1_replica1) every 2
days.
In the solrconfir.xml of coreA, autocommit has set to 30 secs
<autoCommit>
<maxTime>30000</maxTime>
<openSearcher>true</openSearcher>
</autoCommit>
and setting replicateAfter:commit
How do I prevent coreA_shard1_replica2 from always fully resyncing from
coreA_shard1_replica1 ?
Thanks a lot.
stephon
--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-replica-always-fully-resync-index-from-leader-node-tp4171403.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud replica always fully resync index from leader node
Posted by stephon <st...@gmail.com>.
Hello Ludovic,
Before then, I have comment out AutoCommit, the old-style configuration. But
not works, the replica is fully resync every 2 days.
What is the correct way to make a replica to prevent this issue, or what
config I missed?
Thanks a lot
--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-replica-always-fully-resync-index-from-leader-node-tp4171403p4172026.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud replica always fully resync index from leader node
Posted by lboutros <bo...@gmail.com>.
Hi Stephon,
nothing obvious to me. But it is early in the morning for a saturday :D
Did you comment out the old-style replication configuration since your first
message ?
Do you always see the same behavior ?
Ludovic.
-----
Jouve
France.
--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-replica-always-fully-resync-index-from-leader-node-tp4171403p4171497.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud replica always fully resync index from leader node
Posted by stephon <st...@gmail.com>.
Hello Erick,
My solrconfig.xml is in attachment.
solrconfig.xml
<http://lucene.472066.n3.nabble.com/file/n4171487/solrconfig.xml>
It is running with a Debian server with 64GB RAM.
And the full replication evidence is coreA_shard1_replica2 is in recovering
state.
Since in this state, solr/coreA_shard1_replica2/ has a index.TIMESTAMP
directory which is full resyncing from the leader node, and runs out of my
rest disk space :/.
What is the correct way to make a replica in SolrCloud 4.5 ?
Thanks a lot.
--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-replica-always-fully-resync-index-from-leader-node-tp4171403p4171487.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud replica always fully resync index from leader node
Posted by Erick Erickson <er...@gmail.com>.
Stephon:
Not quite sure what's going on, but you're hinting
that you're mixing old-style replication with SolrCloud,
the two are orthogonal.
This, for instance, is irrelevant for SolrCloud:
and setting replicateAfter:commit
So let's see the relevant configuration from solrconfig.xml.
Also, what is your evidence that a full replication is happening?
Showing us what you see will offer some more clues.
Best,
Erick
On Fri, Nov 28, 2014 at 1:10 AM, stephon <st...@gmail.com> wrote:
> Hello Ludovic,
>
> Zookeeper timeout errors not found in log file
>
> Here is my SolrCloud environment information.
> * Solr 4.5.1 used
> * Index size : ~270G
> * Index update: every 30 secs, each update will contain 30000~40000 index
> version changes
> * example:
> * old index version number: 1417165450218
> * new index version number: 1417165480450
> * omitHeader has been set to true
>
> If additional information needed, please let me know.
>
> Thanks a lot
> ---
> stephon
>
>
>
>
> --
> View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-replica-always-fully-resync-index-from-leader-node-tp4171403p4171409.html
> Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud replica always fully resync index from leader node
Posted by stephon <st...@gmail.com>.
Hello Ludovic,
Zookeeper timeout errors not found in log file
Here is my SolrCloud environment information.
* Solr 4.5.1 used
* Index size : ~270G
* Index update: every 30 secs, each update will contain 30000~40000 index
version changes
* example:
* old index version number: 1417165450218
* new index version number: 1417165480450
* omitHeader has been set to true
If additional information needed, please let me know.
Thanks a lot
---
stephon
--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-replica-always-fully-resync-index-from-leader-node-tp4171403p4171409.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: SolrCloud replica always fully resync index from leader node
Posted by lboutros <bo...@gmail.com>.
Hi Stephon,
do you see Zookeeper timeout errors in your log files ?
Could you please give us additional informations like :
How often is your index updated ? Which version of Solr do you use ? What is
the size of your index ?
Make sure you have this handler in your solr configuration file :
<requestHandler name="/get" class="solr.RealTimeGetHandler">
<lst name="defaults">
<str name="omitHeader">true</str>
</lst>
</requestHandler>
Ludovic.
-----
Jouve
France.
--
View this message in context: http://lucene.472066.n3.nabble.com/SolrCloud-replica-always-fully-resync-index-from-leader-node-tp4171403p4171407.html
Sent from the Solr - User mailing list archive at Nabble.com.