[GitHub] [hadoop] virajjasani commented on a change in pull request #4082: HDFS-16507. [SBN read] Avoid purging edit log which is in progress

[GitBox <gi...@apache.org>]

virajjasani commented on a change in pull request #4082:
URL: https://github.com/apache/hadoop/pull/4082#discussion_r836245302



##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
##########
@@ -1509,13 +1509,18 @@ synchronized void abortCurrentLogSegment() {
    * effect.
    */
   @Override
-  public synchronized void purgeLogsOlderThan(final long minTxIdToKeep) {
+  public synchronized void purgeLogsOlderThan(long minTxIdToKeep) {

Review comment:
       Once done, we can also revert this.

##########
File path: hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
##########
@@ -1509,13 +1509,18 @@ synchronized void abortCurrentLogSegment() {
    * effect.
    */
   @Override
-  public synchronized void purgeLogsOlderThan(final long minTxIdToKeep) {
+  public synchronized void purgeLogsOlderThan(long minTxIdToKeep) {
     // Should not purge logs unless they are open for write.
     // This prevents the SBN from purging logs on shared storage, for example.
     if (!isOpenForWrite()) {
       return;
     }
-    
+
+    // Reset purgeLogsFrom to avoid purging edit log which is in progress.
+    if (isSegmentOpen()) {
+      minTxIdToKeep = minTxIdToKeep > curSegmentTxId ? curSegmentTxId : minTxIdToKeep;

Review comment:
       @tomscut I agree that assert alone is not a good idea because not all prod systems have it enabled. I believe we should replace assert here with `Preconditions.checkArgument()`, then we don't need this condition here.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org