You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Konstantin Shvachko (JIRA)" <ji...@apache.org> on 2012/05/25 08:44:24 UTC
[jira] [Created] (HDFS-3463) DFSTestUtil.waitCorruptReplicas()
should not use file reading time as a timeout measure.
Konstantin Shvachko created HDFS-3463:
-----------------------------------------
Summary: DFSTestUtil.waitCorruptReplicas() should not use file reading time as a timeout measure.
Key: HDFS-3463
URL: https://issues.apache.org/jira/browse/HDFS-3463
Project: Hadoop HDFS
Issue Type: Bug
Components: test
Affects Versions: 3.0.0
Reporter: Konstantin Shvachko
Tests fail because DFSTestUtil.waitCorruptReplicas() does not wait long enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3463) DFSTestUtil.waitCorruptReplicas()
should not use file reading time as a timeout measure.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285463#comment-13285463 ]
Konstantin Shvachko commented on HDFS-3463:
-------------------------------------------
Agreed, these are the same. Thanks for pointing out. Feel free to close one.
> DFSTestUtil.waitCorruptReplicas() should not use file reading time as a timeout measure.
> ----------------------------------------------------------------------------------------
>
> Key: HDFS-3463
> URL: https://issues.apache.org/jira/browse/HDFS-3463
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 3.0.0
> Reporter: Konstantin Shvachko
> Attachments: testBlockCorruptionRecoveryPolicy1.log.htm
>
>
> Tests fail because DFSTestUtil.waitCorruptReplicas() does not wait long enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3463) DFSTestUtil.waitCorruptReplicas()
should not use file reading time as a timeout measure.
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13285090#comment-13285090 ]
Aaron T. Myers commented on HDFS-3463:
--------------------------------------
Hey Konstantin, this seems like a duplicate of HDFS-3266. Do you agree?
> DFSTestUtil.waitCorruptReplicas() should not use file reading time as a timeout measure.
> ----------------------------------------------------------------------------------------
>
> Key: HDFS-3463
> URL: https://issues.apache.org/jira/browse/HDFS-3463
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 3.0.0
> Reporter: Konstantin Shvachko
> Attachments: testBlockCorruptionRecoveryPolicy1.log.htm
>
>
> Tests fail because DFSTestUtil.waitCorruptReplicas() does not wait long enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (HDFS-3463) DFSTestUtil.waitCorruptReplicas()
should not use file reading time as a timeout measure.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13283184#comment-13283184 ]
Konstantin Shvachko commented on HDFS-3463:
-------------------------------------------
TestDatanodeBlockScanner.testBlockCorruptionRecoveryPolicy1 failed because DFSTestUtil.waitCorruptReplicas() reads some file 50 times and checks if the corruption is detected after each read. In the run that time was enough for the DN to restart, but not enough for NN to detect the corruption.
I see fifty "Waiting for 1 corrupt replicas", which means all 50 reads have been done, but don't see "NameSystem.addToCorruptReplicasMap:" in the log indicating that corruption was not detected.
> DFSTestUtil.waitCorruptReplicas() should not use file reading time as a timeout measure.
> ----------------------------------------------------------------------------------------
>
> Key: HDFS-3463
> URL: https://issues.apache.org/jira/browse/HDFS-3463
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 3.0.0
> Reporter: Konstantin Shvachko
> Attachments: testBlockCorruptionRecoveryPolicy1.log.htm
>
>
> Tests fail because DFSTestUtil.waitCorruptReplicas() does not wait long enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (HDFS-3463) DFSTestUtil.waitCorruptReplicas()
should not use file reading time as a timeout measure.
Posted by "Aaron T. Myers (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Aaron T. Myers resolved HDFS-3463.
----------------------------------
Resolution: Duplicate
> DFSTestUtil.waitCorruptReplicas() should not use file reading time as a timeout measure.
> ----------------------------------------------------------------------------------------
>
> Key: HDFS-3463
> URL: https://issues.apache.org/jira/browse/HDFS-3463
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 3.0.0
> Reporter: Konstantin Shvachko
> Attachments: testBlockCorruptionRecoveryPolicy1.log.htm
>
>
> Tests fail because DFSTestUtil.waitCorruptReplicas() does not wait long enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (HDFS-3463) DFSTestUtil.waitCorruptReplicas()
should not use file reading time as a timeout measure.
Posted by "Konstantin Shvachko (JIRA)" <ji...@apache.org>.
[ https://issues.apache.org/jira/browse/HDFS-3463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Konstantin Shvachko updated HDFS-3463:
--------------------------------------
Attachment: testBlockCorruptionRecoveryPolicy1.log.htm
Attaching log from Jenkins build for HDFS-3368 where two tests failed because of that:
TestDatanodeBlockScanner.testBlockCorruptionRecoveryPolicy1
TestBlocksWithNotEnoughRacks.testCorruptBlockRereplicatedAcrossRacks
> DFSTestUtil.waitCorruptReplicas() should not use file reading time as a timeout measure.
> ----------------------------------------------------------------------------------------
>
> Key: HDFS-3463
> URL: https://issues.apache.org/jira/browse/HDFS-3463
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: test
> Affects Versions: 3.0.0
> Reporter: Konstantin Shvachko
> Attachments: testBlockCorruptionRecoveryPolicy1.log.htm
>
>
> Tests fail because DFSTestUtil.waitCorruptReplicas() does not wait long enough.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira