You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Varun Thacker (JIRA)" <ji...@apache.org> on 2017/10/17 20:05:00 UTC
[jira] [Commented] (SOLR-11326) CDCR bootstrap should not download
tlog's from source
[ https://issues.apache.org/jira/browse/SOLR-11326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16208245#comment-16208245 ]
Varun Thacker commented on SOLR-11326:
--------------------------------------
Is there any scenario where we need to download the tlogs? I don't see any use-case in master-slave / solrcloud or in cdcr . So why even have the option?
Today in CDCR we explicitly ask the replication handler not to download the tlogs by doing this
{[solrParams.set(ReplicationHandler.TLOG_FILES, false);}}
But the param is not respected. The patch actually fixes the issue. But why even keep such an option when we don't need it?
Does someone know the motivation of this in {{ReplicationHander::getFileList}}
{code}
// fetch list of tlog files only if cdcr is activated
if (solrParams.getBool(TLOG_FILES, true) && core.getUpdateHandler().getUpdateLog() != null
&& core.getUpdateHandler().getUpdateLog() instanceof CdcrUpdateLog) {
{code}
> CDCR bootstrap should not download tlog's from source
> -----------------------------------------------------
>
> Key: SOLR-11326
> URL: https://issues.apache.org/jira/browse/SOLR-11326
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Reporter: Varun Thacker
> Attachments: SOLR-11326.patch, SOLR-11326.patch
>
>
> While analyzing two separate fails on SOLR-11278 I see that during bootstrap the tlog's from the source is getting download
> snippet1:
> {code}
> [junit4] 2> 42931 INFO (qtp1525032019-69) [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor
> [junit4] 2> 42934 INFO (cdcr-bootstrap-status-32-thread-1-processing-n:127.0.0.1:53178_solr x:cdcr-source_shard1_replica1 s:shard1 c:cdcr-source r:core_node1) [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target shard: shard1 leader: http://127.0.0.1:53170/solr/cdcr-target_shard1_replica1/
> [junit4] 2> 43003 INFO (qtp1525032019-69) [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.c.S.Request [cdcr-source_shard1_replica1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0
> [junit4] 2> 43004 INFO (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's generation: 12
> [junit4] 2> 43004 INFO (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Master's version: 1503514968639
> [junit4] 2> 43004 INFO (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's generation: 1
> [junit4] 2> 43004 INFO (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Slave's version: 0
> [junit4] 2> 43004 INFO (recoveryExecutor-6-thread-1-processing-n:127.0.0.1:53170_solr x:cdcr-target_shard1_replica1 s:shard1 c:cdcr-target r:core_node1) [n:127.0.0.1:53170_solr c:cdcr-target s:shard1 r:core_node1 x:cdcr-target_shard1_replica1] o.a.s.h.IndexFetcher Starting replication process
> [junit4] 2> 43041 INFO (qtp1525032019-71) [n:127.0.0.1:53178_solr c:cdcr-source s:shard1 r:core_node1 x:cdcr-source_shard1_replica1] o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4649, name=tlog.0000000000000000000.1576549701811961856}, {size=4770, name=tlog.0000000000000000001.1576549702515556352}, {size=4770, name=tlog.0000000000000000002.1576549702628802560}, {size=4770, name=tlog.0000000000000000003.1576549702720028672}, {size=4770, name=tlog.0000000000000000004.1576549702799720448}, {size=4770, name=tlog.0000000000000000005.1576549702894092288}, {size=4770, name=tlog.0000000000000000006.1576549703029358592}, {size=4770, name=tlog.0000000000000000007.1576549703126876160}, {size=4770, name=tlog.0000000000000000008.1576549703208665088}, {size=4770, name=tlog.0000000000000000009.1576549703295696896}
> {code}
> snippet2:
> {code}
> 17070 [junit4] 2> 677606 INFO (qtp22544544-5725) [ ] o.a.s.h.CdcrReplicatorManager Attempting to bootstrap target collection: cdcr-target, shard: shard1^M
> 17071 [junit4] 2> 677608 INFO (qtp22544544-5725) [ ] o.a.s.h.CdcrReplicatorManager Submitting bootstrap task to executor^M
> 17091 [junit4] 2> 677627 INFO (qtp22544544-5724) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={qt=/replication&wt=javabin&version=2&command=indexversion} status=0 QTime=0^M
> 17092 [junit4] 2> 677628 INFO (recoveryExecutor-1024-thread-1) [ ] o.a.s.h.IndexFetcher Master's generation: 12^M
> 17093 [junit4] 2> 677628 INFO (recoveryExecutor-1024-thread-1) [ ] o.a.s.h.IndexFetcher Master's version: 1504127155366^M
> 17094 [junit4] 2> 677628 INFO (recoveryExecutor-1024-thread-1) [ ] o.a.s.h.IndexFetcher Slave's generation: 1^M
> 17095 [junit4] 2> 677628 INFO (recoveryExecutor-1024-thread-1) [ ] o.a.s.h.IndexFetcher Slave's version: 0^M
> 17096 [junit4] 2> 677628 INFO (recoveryExecutor-1024-thread-1) [ ] o.a.s.h.IndexFetcher Starting replication process^M
> 17097 [junit4] 2> 677654 INFO (qtp22544544-5725) [ ] o.a.s.h.ReplicationHandler Adding tlog files to list: [{size=4773, name=tlog.0000000000000000009}]^M
> 17098 [junit4] 2> 677655 INFO (qtp22544544-5725) [ ] o.a.s.c.S.Request [cdcr-source_shard1_replica_n1] webapp=/solr path=/replication params={generation=12&qt=/replication&wt=javabin&version=2&command=filelist} status=0 QTime=26^M
> 17099 [junit4] 2> 677656 INFO (recoveryExecutor-1024-thread-1) [ ] o.a.s.h.IndexFetcher Number of files in latest index in master: 23^M
> 17100 [junit4] 2> 677656 INFO (recoveryExecutor-1024-thread-1) [ ] o.a.s.h.IndexFetcher Number of tlog files in master: 1^M
> {code}
> Also here's the code comment which also says we don't need to download the tlog's
> {code:title=CdcrRequestHandler.java|borderStyle=solid}
> ModifiableSolrParams solrParams = new ModifiableSolrParams();
> solrParams.set(ReplicationHandler.MASTER_URL, masterUrl);
> // we do not want the raw tlog files from the source
> solrParams.set(ReplicationHandler.TLOG_FILES, false);
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org