You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Hoss Man (JIRA)" <ji...@apache.org> on 2013/06/14 00:43:21 UTC
[jira] [Commented] (SOLR-4923) Replica index is one version behind
sending the commit to non-leader instance
[ https://issues.apache.org/jira/browse/SOLR-4923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13682853#comment-13682853 ]
Hoss Man commented on SOLR-4923:
--------------------------------
Hmmm...
I can't explain it -- but i can reproduce this fairly trivially on trunk using the example configs...
* Spin up two ports using numShards=1...
{noformat}
$ java -Dbootstrap_confdir=./solr/collection1/conf -Dcollection.configName=myconf -DzkRun -DnumShards=1 -jar start.jar
$ java -Djetty.port=7574 -DzkHost=localhost:9983 -jar start.jar
{noformat}
* use UI to confirm that 8983 is the leader and 7574 is it's replica
* send a doc to the 7574 replica and observe the results of querying against the two distinct replicas...
{noformat}
$ curl "http://localhost:8983/solr/collection1/select?q=id:HOSS&wt=json&omitHeader=true"
{"response":{"numFound":0,"start":0,"docs":[]}}
$ curl "http://localhost:7574/solr/collection1/select?q=id:HOSS&wt=json&omitHeader=true"
{"response":{"numFound":0,"start":0,"docs":[]}}
$ curl "http://localhost:7574/solr/collection1/update?commit=true&wt=json&omitHeader=true" -H 'Content-Type: text/xml' --data-binary '<add><doc><field name="id">HOSS</field><field name="cat_s">1</field></doc></add>'
{}
$ curl "http://localhost:7574/solr/collection1/select?q=id:HOSS&wt=json&omitHeader=true"
{"response":{"numFound":0,"start":0,"docs":[]}}
$ curl "http://localhost:8983/solr/collection1/select?q=id:HOSS&wt=json&omitHeader=true"
{"response":{"numFound":1,"start":0,"docs":[{"id":"HOSS","cat_s":"1","_version_":1437767149083951104}]}}
{noformat}
* wait a bit and retry in case there is some delay in replication...
{noformat}
$ curl "http://localhost:7574/solr/collection1/select?q=id:HOSS&wt=json&omitHeader=true"
{"response":{"numFound":0,"start":0,"docs":[]}}
$ curl "http://localhost:7574/solr/collection1/select?q=id:HOSS&wt=json&omitHeader=true"
{"response":{"numFound":0,"start":0,"docs":[]}}
$ curl "http://localhost:7574/solr/collection1/select?q=id:HOSS&wt=json&omitHeader=true"
{"response":{"numFound":0,"start":0,"docs":[]}}
$ curl "http://localhost:7574/solr/collection1/select?q=id:HOSS&wt=json&omitHeader=true"
{"response":{"numFound":0,"start":0,"docs":[]}}
$ curl "http://localhost:7574/solr/collection1/select?q=id:HOSS&wt=json&omitHeader=true"
{"response":{"numFound":0,"start":0,"docs":[]}}
$ curl "http://localhost:7574/solr/collection1/select?q=id:HOSS&wt=json&omitHeader=true"
{"response":{"numFound":0,"start":0,"docs":[]}}
$ curl "http://localhost:7574/solr/collection1/select?q=id:HOSS&wt=json&omitHeader=true"
{"response":{"numFound":0,"start":0,"docs":[]}}
{noformat}
* send a new version of the doc to the 7574 replica and observe the results of querying against the two distinct replicas...
{noformat}
$ curl "http://localhost:7574/solr/collection1/update?commit=true&wt=json&omitHeader=true" -H 'Content-Type: text/xml' --data-binary '<add><doc><field name="id">HOSS</field><field name="cat_s">2</field></doc></add>'
{}
$ curl "http://localhost:7574/solr/collection1/select?q=id:HOSS&wt=json&omitHeader=true"
{"response":{"numFound":1,"start":0,"docs":[{"id":"HOSS","cat_s":"1","_version_":1437767149083951104}]}}
$ curl "http://localhost:8983/solr/collection1/select?q=id:HOSS&wt=json&omitHeader=true"
{"response":{"numFound":1,"start":0,"docs":[{"id":"HOSS","cat_s":"2","_version_":1437767405646381056}]}}
{noformat}
I'll see if i can work up a JUnit test.
> Replica index is one version behind sending the commit to non-leader instance
> -----------------------------------------------------------------------------
>
> Key: SOLR-4923
> URL: https://issues.apache.org/jira/browse/SOLR-4923
> Project: Solr
> Issue Type: Bug
> Components: replication (java)
> Affects Versions: 4.2
> Environment: Solr 4.2.1
> OS X 10.8.3
> Reporter: Ricardo Merizalde
> Priority: Critical
>
> I was actually trying to debug an issue we experiencing in production where the replica version is ahead from the leader when I noticed this problem.
> For my tests I'm running two Solr instances with distributed updates (SolrCloud). ZK runs embedded within one of the instances.
> The test consists on updating one field in single document. If I send an update to the leader the index is replicated correctly. However, if I run the update against the follower replica only the leader is updated correctly. I can reproduce this using both hard and soft commits. Here is the command I'm running:
> curl "http://localhost:8999/solr/rulePreview/update?commit=true&softCommit=true" -H "Content-Type: text/xml" --data-binary '<add>...</add>
> If I execute a second commit against the follower the leader will have the most recent update and the follower will be update from the first commit.
> For example, my field is named category and initially it contains the value cat_1. If update the value to cat_2 the leader sees the change but the follower doesn't. If a second commit updates the field to cat_3 the leader will return cat_3 but the follower return cat_2.
> Reloading the core in the follower fixes the problem.
> The logs seem to confirm the follower gets the latest index version. However, the version in the logs doesn't matches the on in the Core Admin UI nor Luke. Here are some logs from the leader:
> Jun 12, 2013 10:34:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
> INFO: [rulePreview_en] webapp=/solr path=/update params={distrib.from=http://192.168.1.106:8998/solr/rulePreview_en/&update.distrib=TOLEADER&wt=javabin&version=2} {add=[importedRedirect1 (1437700518392627200)]} 0 11
> Jun 12, 2013 10:34:19 PM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> Jun 12, 2013 10:34:19 PM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@47e4e06c main
> Jun 12, 2013 10:34:19 PM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: end_commit_flush
> Jun 12, 2013 10:34:19 PM org.apache.solr.core.QuerySenderListener newSearcher
> INFO: QuerySenderListener sending requests to Searcher@47e4e06c main{StandardDirectoryReader(segments_3g:467:nrt _2a(4.2.1):C134/1 _3c(4.2.1):C1)}
> Jun 12, 2013 10:34:19 PM org.apache.solr.core.QuerySenderListener newSearcher
> INFO: QuerySenderListener done.
> Jun 12, 2013 10:34:19 PM org.apache.solr.core.SolrCore registerSearcher
> INFO: [rulePreview_en] Registered new searcher Searcher@47e4e06c main{StandardDirectoryReader(segments_3g:467:nrt _2a(4.2.1):C134/1 _3c(4.2.1):C1)}
> Jun 12, 2013 10:34:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
> INFO: [rulePreview_en] webapp=/solr path=/update params={waitSearcher=true&commit=true&wt=javabin&expungeDeletes=false&commit_end_point=true&version=2&softCommit=true} {commit=} 0 12
> And the logs from the follower:
> Jun 12, 2013 10:34:19 PM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: start commit{,optimize=false,openSearcher=true,waitSearcher=true,expungeDeletes=false,softCommit=true,prepareCommit=false}
> Jun 12, 2013 10:34:19 PM org.apache.solr.search.SolrIndexSearcher <init>
> INFO: Opening Searcher@1e23cfc main
> Jun 12, 2013 10:34:19 PM org.apache.solr.update.DirectUpdateHandler2 commit
> INFO: end_commit_flush
> Jun 12, 2013 10:34:19 PM org.apache.solr.core.QuerySenderListener newSearcher
> INFO: QuerySenderListener sending requests to Searcher@1e23cfc main{StandardDirectoryReader(segments_3i:463:nrt _2a(4.2.1):C134/1 _3b(4.2.1):C1)}
> Jun 12, 2013 10:34:19 PM org.apache.solr.core.QuerySenderListener newSearcher
> INFO: QuerySenderListener done.
> Jun 12, 2013 10:34:19 PM org.apache.solr.core.SolrCore registerSearcher
> INFO: [rulePreview_en] Registered new searcher Searcher@1e23cfc main{StandardDirectoryReader(segments_3i:463:nrt _2a(4.2.1):C134/1 _3b(4.2.1):C1)}
> Jun 12, 2013 10:34:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
> INFO: [rulePreview_en] webapp=/solr path=/update params={distrib.from=http://192.168.1.106:8999/solr/rulePreview_en/&update.distrib=FROMLEADER&wt=javabin&version=2} {add=[importedRedirect1 (1437700518392627200)]} 0 4
> Jun 12, 2013 10:34:19 PM org.apache.solr.update.SolrCmdDistributor distribCommit
> INFO: Distrib commit to:[StdNode: http://192.168.1.106:8999/solr/rulePreview_en/] params:commit_end_point=true&commit=true&softCommit=true&waitSearcher=true&expungeDeletes=false
> Jun 12, 2013 10:34:19 PM org.apache.solr.update.processor.LogUpdateProcessor finish
> INFO: [rulePreview_en] webapp=/solr path=/update params={softCommit=true} {add=[importedRedirect1],commit=} 0 41
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org