You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@hbase.apache.org by "chenglei (Jira)" <ji...@apache.org> on 2022/11/16 03:22:00 UTC
[jira] [Updated] (HBASE-27358) Avoid synchronization in AsyncFSWAL
[ https://issues.apache.org/jira/browse/HBASE-27358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
chenglei updated HBASE-27358:
-----------------------------
Description:
{{AsyncFSWAL}} uses {{consumeLock }}to synchronize operations between {{AsyncFSWAL.consumeExecutor}} and {{AsyncFSWAL.doReplaceWriter}} (also {{AsyncFSWAL.doShutdown}}), I think we could further use the event-driven approach again to eliminate the consumeLock:
* For {{AsyncFSWAL.doReplaceWriter}} ,it could just publish the request for replacing writer event to {{AsyncFSWAL}} and wait for the event to complete.
* For {{AsyncFSWAL.consumeExecutor}} ,it could detect the replacing writer event at its safe point (i.e. writer is broken or no {{FSWALEntry}} to write) and executes replacing writer event by itself, and after it completes , it notifies {{LogRoller}} thread that the event is completed. Replacing writer is a very lightweight operation and only replacing writer is completed, {{AsyncFSWAL.consumeExecutor}} could continue to write WAL, so it is harmless for {{AsyncFSWAL.consumeExecutor}} and could avoid using consumeLock.
The main difference between the PR and the current implementation is {{AsyncFSWAL.consumeExecutor}} replacing the writer by itself.
was:{{AsyncFSWAL}} uses {{consumeLock}} to synchronize operations between {{AsyncFSWAL.consumeExecutor}} and {{AsyncFSWAL.doReplaceWriter}}/{{AsyncFSWAL.doShutdown}}. If we try to make {{AsyncFSWAL.consumeExecutor}} to roll WAL at the safe point by itself, and {{AsyncFSWAL.doReplaceWriter}} just publishs the roll wal task and waits for the roll operation to complete, then we could avoid the synchronize operations in {{AsyncFSWAL}}.
> Avoid synchronization in AsyncFSWAL
> -----------------------------------
>
> Key: HBASE-27358
> URL: https://issues.apache.org/jira/browse/HBASE-27358
> Project: HBase
> Issue Type: Improvement
> Components: wal
> Affects Versions: 3.0.0-alpha-3
> Reporter: chenglei
> Assignee: chenglei
> Priority: Major
>
> {{AsyncFSWAL}} uses {{consumeLock }}to synchronize operations between {{AsyncFSWAL.consumeExecutor}} and {{AsyncFSWAL.doReplaceWriter}} (also {{AsyncFSWAL.doShutdown}}), I think we could further use the event-driven approach again to eliminate the consumeLock:
> * For {{AsyncFSWAL.doReplaceWriter}} ,it could just publish the request for replacing writer event to {{AsyncFSWAL}} and wait for the event to complete.
> * For {{AsyncFSWAL.consumeExecutor}} ,it could detect the replacing writer event at its safe point (i.e. writer is broken or no {{FSWALEntry}} to write) and executes replacing writer event by itself, and after it completes , it notifies {{LogRoller}} thread that the event is completed. Replacing writer is a very lightweight operation and only replacing writer is completed, {{AsyncFSWAL.consumeExecutor}} could continue to write WAL, so it is harmless for {{AsyncFSWAL.consumeExecutor}} and could avoid using consumeLock.
> The main difference between the PR and the current implementation is {{AsyncFSWAL.consumeExecutor}} replacing the writer by itself.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)