[jira] [Resolved] (HDFS-16910) Fix incorrectly initializing RandomAccessFile caused flush performance decreased for JN

["ZanderXu (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-16910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

ZanderXu resolved HDFS-16910.
-----------------------------
    Fix Version/s: 3.4.0
     Hadoop Flags: Reviewed
       Resolution: Fixed

> Fix incorrectly initializing RandomAccessFile caused flush performance decreased for JN
> ---------------------------------------------------------------------------------------
>
>                 Key: HDFS-16910
>                 URL: https://issues.apache.org/jira/browse/HDFS-16910
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Haiyang Hu
>            Assignee: Haiyang Hu
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 3.4.0
>
>
> At present, after our cluster backport patch HDFS-15882, 
> when set shouldSyncWritesAndSkipFsync to false, there will be flush performance degradation caused by JN.
> *Root Cause*：
> when setting shouldSyncWritesAndSkipFsync to false, the mode of init RandomAccessFile will be `rws`. 
> even if fc.force(false) is executed when flushAndSync is executed (hopefully, only requires updates to the file's content to be written to storage and the metadata is not update), 
> but since the mode of RandomAccessFile is `rws`, It will requires updates to both the file's content and its metadata to be written, 
> there will be flush performance degradation caused by JN.
> *Fix:*
> Need to update RandomAccessFile's mode from `rws` to `rwd`:
> rwd: Open for reading and writing, as with "rw", and also require that every update to the file's content be written synchronously to the underlying storage device.
> {code:java}
> if (shouldSyncWritesAndSkipFsync) {
> rp = new RandomAccessFile(name, "rwd");
> } else {
> rp = new RandomAccessFile(name, "rw");
> }
> {code}
> In this way, when flushAndSync is executed, 
> if shouldSyncWritesAndSkipFsync is false and the mode of RandomAccessFile is 'rw', it will call fc.force(false) to execute, 
> otherwise should use `rwd` to perform the operation.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org