[jira] [Commented] (HBASE-26890) Make the WAL interface async so sync replication can be built up on the WAL interface

["Duo Zhang (Jira)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/HBASE-26890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17733142#comment-17733142 ] 

Duo Zhang commented on HBASE-26890:
-----------------------------------

After investigating,  I do not think this is the correct way to implement sync replication.

In the WAL implementation, we have also down sequence id accounting, and it is highly related to how we roll the WAL, so it is impossible for us to have two WAL instances for the same region, as we can not have two different sequence id accountings...

So the current abstraction layer is correct, and also for new WAL implementations, they should just implement a new WAL.AsyncWriter(and a WAL.Writer maybe), then they could just reuse all the code in AsyncFSWAL. We could use URI instead of Path to describe a WAL entry, it will be more general.

And for sync replication, the only thing is that, it can only work with AsyncWriter, maybe we could make it more general so we do not need to hack into AsyncFSWAL, just pass something like an AsyncWriterFactory is fine. The only thing for now is that, while reopening a region, we need to write some markers out so we have done some hacks in AsyncFSWAL to only allow marker edits out. But I do not think this is necessary, we will only reopen regions when transiting state from ACTIVE to DOWNGRADE_ACTIVE, where we know that we will not be in sync with the peer cluster, so it is OK for us to also write data edits out to the source cluster, next time when we want to convert back to ACTIVE, the async replication will replicate the out to the peer cluster, so there is no problem.

Let me file an issue to see if I can make the implementation more general.

Thanks.

> Make the WAL interface async so sync replication can be built up on the WAL interface
> -------------------------------------------------------------------------------------
>
>                 Key: HBASE-26890
>                 URL: https://issues.apache.org/jira/browse/HBASE-26890
>             Project: HBase
>          Issue Type: Improvement
>          Components: Replication, wal
>            Reporter: Duo Zhang
>            Assignee: Duo Zhang
>            Priority: Major
>
> Instead of hacking into the WAL implementation.
> This could make the implementation more general if later we want to change the WAL implementation.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)