You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-issues@hadoop.apache.org by GitBox <gi...@apache.org> on 2022/08/02 20:10:51 UTC

[GitHub] [hadoop] KevinWikant commented on pull request #4568: HDFS-16664. Use correct GenerationStamp when invalidating corrupt block replicas

KevinWikant commented on PR #4568:
URL: https://github.com/apache/hadoop/pull/4568#issuecomment-1203167177

   I have confirmed that the change in unit test behavior that uncovered this bug is related to a change in [this condition](https://github.com/apache/hadoop/blob/a5adc27c99ffa0ee4f274276afd38f0e7367b9d2/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java#L1979) which was made as part of: https://issues.apache.org/jira/browse/HDFS-15200
   
   In trunk the block invalidation does not get postponed because "dfs.namenode.corrupt.block.delete.immediately.enabled" defaults to true
   
   I have added some additional test coverage for this change
   
   - "testDeleteCorruptReplicaForUnderReplicatedBlock" test without postponing (trunk behavior)
   
   ```
   > mvn test -Dtest=org.apache.hadoop.hdfs.TestDecommission#testDeleteCorruptReplicaForUnderReplicatedBlock
   
   > cat target/surefire-reports/org.apache.hadoop.hdfs.TestDecommission-output.txt | grep XXX
   XXX invalidateBlock? dn=127.0.0.1:63741 , blk=1073741825_1001 , replicasOnStaleNodes=1
   XXX addToInvalidates
   XXX invalidateBlock? dn=127.0.0.1:63745 , blk=1073741825_1003 , replicasOnStaleNodes=0
   XXX addToInvalidates
   ```
   
   - "testDeleteCorruptReplicaForUnderReplicatedBlock" test with postponing (branch-3.2.1 behavior)
   
   ```
   > mvn test -Dtest=org.apache.hadoop.hdfs.TestDecommission#testDeleteCorruptReplicaForUnderReplicatedBlockWithInvalidationPostponed
   
   > cat target/surefire-reports/org.apache.hadoop.hdfs.TestDecommission-output.txt | grep XXX
   XXX invalidateBlock? dn=127.0.0.1:63927 , blk=1073741825_1001 , replicasOnStaleNodes=1
   XXX postponeBlock
   XXX invalidateBlock? dn=127.0.0.1:63931 , blk=1073741825_1003 , replicasOnStaleNodes=0
   XXX addToInvalidates
   XXX invalidateBlock? dn=127.0.0.1:63927 , blk=1073741825_1001 , replicasOnStaleNodes=0
   XXX addToInvalidates
   ```
   
   - CorruptReplicasMap test
   
   ```
   > mvn test -Dtest=org.apache.hadoop.hdfs.server.blockmanagement.TestCorruptReplicaInfo#testGetCorruptReplicaGenerationStamp
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org