You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@lucene.apache.org by "Amrit Sarkar (JIRA)" <ji...@apache.org> on 2018/03/08 09:05:00 UTC
[jira] [Commented] (SOLR-12063) Fix tlog entry indexes in UpdateLog
for CDCR to work smoothly.
[ https://issues.apache.org/jira/browse/SOLR-12063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390945#comment-16390945 ]
Amrit Sarkar commented on SOLR-12063:
-------------------------------------
Issue:
The tlog entry for 'DeleteById' has 3 entries before 7.2: 1. operation 2. flags 3. payload.
and 4 after the CDCR bidirectional patch with 4th one: boolean denoting it is CDCR or not.
UpdateLog.RecentUpdates are utilized very vividly in Recovery, PeerSync and in CDCR CollectionCheckpoint API.
In the RecentUpdates creation, update() method, DeleteByIdList gets created for replay where the the payload is loaded into the list by referring the last index of tlog entry (entry.size() - 1).
With Cdcr bidirectional patch, the payload is at the index [2] of entry, instead of last, and hence the Recovery, PeerSync and Cdcr CollectionCheckpoint, when CDCR enabled, will have serious implications and issues.
Fix:
Throughout the project, the payload for DeleteById and DeleteByQuery is on index [2] and that was the motivation behind attaching the Cdcr boolean to the back. If we change the tlog entry for payload for DeleteById to index [2], everything gets in order.
Verification:
I have attached log outputs for {{TestStressRecovery}} and {{PeerSyncTest}} where the above fix is validated and the tests are running successfully, while printing "DELETE LIST triggred here: [payload]" to make sure we are reading from the right index.
> Fix tlog entry indexes in UpdateLog for CDCR to work smoothly.
> --------------------------------------------------------------
>
> Key: SOLR-12063
> URL: https://issues.apache.org/jira/browse/SOLR-12063
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: CDCR
> Affects Versions: 7.2
> Reporter: Amrit Sarkar
> Assignee: Varun Thacker
> Priority: Major
> Attachments: SOLR-12063.patch, SOLR-12063.patch, test-report-PeerSyncTest, test-report-TestStressRecovery
>
>
> In *UpdateLog*, {{RecentUpdates}} reads the entry of tlogs, and throughout the project the entry indexes for various operations are consistent, but odd in this part. As we included new entry in TransactionLog for CDCR, read operations in {{update()}} method of {{RecentUpdates}} throw error rightfully as elements are read from wrong indexes of tlog entry. The entry indexes of llog should be consistent throughout.
> {code}
> [beaster] 2> 27394 WARN (qtp97093533-72) [n:127.0.0.1:44658_solr c:cdcr-cluster1 s:shard1 r:core_node3 x:cdcr-cluster1_shard1_replica_n1] o.a.s.u.UpdateLog Unexpected log entry or corrupt log. Entry=[2, -1594312216007409664, [B@28e6859c, true]
> [beaster] 2> java.lang.ClassCastException: java.lang.Boolean cannot be cast to [B
> [beaster] 2> at org.apache.solr.update.UpdateLog$RecentUpdates.update(UpdateLog.java:1443)
> [beaster] 2> at org.apache.solr.update.UpdateLog$RecentUpdates.<init>(UpdateLog.java:1340)
> [beaster] 2> at org.apache.solr.update.UpdateLog.getRecentUpdates(UpdateLog.java:1513)
> [beaster] 2> at org.apache.solr.handler.CdcrRequestHandler.handleShardCheckpointAction(CdcrRequestHandler.java:448)
> [beaster] 2> at org.apache.solr.handler.CdcrRequestHandler.handleRequestBody(CdcrRequestHandler.java:198)
> [beaster] 2> at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
> [beaster] 2> at org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
> [beaster] 2> at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)
> [beaster] 2> at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:517)
> [beaster] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)
> [beaster] 2> at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
> [beaster] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> [beaster] 2> at org.apache.solr.client.solrj.embedded.JettySolrRunner$DebugFilter.doFilter(JettySolrRunner.java:139)
> [beaster] 2> at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1637)
> [beaster] 2> at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
> [beaster] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
> [beaster] 2> at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
> [beaster] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
> [beaster] 2> at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
> [beaster] 2> at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
> [beaster] 2> at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
> [beaster] 2> at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org