[jira] [Updated] (KUDU-567) Remove 'update_lock_' from consensus

["Mike Percy (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/KUDU-567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Mike Percy updated KUDU-567:
----------------------------
    Parent: KUDU-560

> Remove 'update_lock_' from consensus
> ------------------------------------
>
>                 Key: KUDU-567
>                 URL: https://issues.apache.org/jira/browse/KUDU-567
>             Project: Kudu
>          Issue Type: Sub-task
>          Components: consensus
>    Affects Versions: M4.5
>            Reporter: David Alves
>
> We should remove 'update_lock_' from the consensus implementation, but in order to do so we need to use two watermarks.
> The last received watermark should reflect what has been received from the leader, the replica updates it when new messages arrive and the leader uses it to select which messages to send, but doesn't update global watermarks based on it.
> The last replicated watermark reflects exclusively what has been made durable at the replica, the leader uses this watermark to advance global watermarks.
> Removing 'update_lock_' presently would create a subtle bug. Here's the scenario:
> 1- Leader sends messages 1-10 to replica.
> 2- Replica takes all messages, prepares them and appends them to the log.
> 3- The log append takes a while, the rpc times out at the leader and the leader resends messages 1-10.
> 4- Messages are deduplicated at the replica making the leader's message "status-only", i.e. without anything for the replica to do, the replica would reply immediately with the last received watermark = 10 even though the log append from step 2 might not have completed yet.
> Having the two watermarks would allow the 2nd message to reply immediately to the leader without having the leader advance the watermarks more than it should.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)