[jira] [Work logged] (HDFS-16531) Avoid setReplication logging an edit record if old replication equals the new value

["ASF GitHub Bot (Jira)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/HDFS-16531?focusedWorklogId=754070&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-754070 ]

ASF GitHub Bot logged work on HDFS-16531:
-----------------------------------------

                Author: ASF GitHub Bot
            Created on: 07/Apr/22 13:34
            Start Date: 07/Apr/22 13:34
    Worklog Time Spent: 10m 
      Work Description: sodonnel opened a new pull request, #4148:
URL: https://github.com/apache/hadoop/pull/4148

   ### Description of PR
   
   I recently came across a NN log where about 800k setRep calls were made, setting the replication from 3 to 3 - ie leaving it unchanged. Obviously the application should be fixed, but we could have an optimisation for this.
   
   When the replication is unchanged in a case like this, we log an edit record, an audit log, and perform some quota checks etc. I believe we should still log an audit in these sort of cases, but we can skip all the checks and avoid writing an edit.
   
   ### How was this patch tested?
   
   Added a new test and validated the code paths were correct around writing and syncing edits or not with some log messages I then removed.
   
   




Issue Time Tracking
-------------------

            Worklog Id:     (was: 754070)
    Remaining Estimate: 0h
            Time Spent: 10m

> Avoid setReplication logging an edit record if old replication equals the new value
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-16531
>                 URL: https://issues.apache.org/jira/browse/HDFS-16531
>             Project: Hadoop HDFS
>          Issue Type: Improvement
>          Components: namenode
>            Reporter: Stephen O'Donnell
>            Assignee: Stephen O'Donnell
>            Priority: Major
>          Time Spent: 10m
>  Remaining Estimate: 0h
>
> I recently came across a NN log where about 800k setRep calls were made, setting the replication from 3 to 3 - ie leaving it unchanged.
> Even in a case like this, we log an edit record, an audit log, and perform some quota checks etc.
> I believe it should be possible to avoid some of the work if we check for oldRep == newRep and jump out of the method early.



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org