You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Zhe Zhang (JIRA)" <ji...@apache.org> on 2016/09/10 00:56:20 UTC
[jira] [Created] (HDFS-10854) Remove createStripedFile and
addBlockToFile by creating real EC files
Zhe Zhang created HDFS-10854:
--------------------------------
Summary: Remove createStripedFile and addBlockToFile by creating real EC files
Key: HDFS-10854
URL: https://issues.apache.org/jira/browse/HDFS-10854
Project: Hadoop HDFS
Issue Type: Bug
Components: erasure-coding, test
Affects Versions: 3.0.0-alpha2
Reporter: Zhe Zhang
{{DFSTestUtil#createStripedFile}} and {{addBlockToFile}} were developed before we completed EC client. They were used to test the {{NameNode}} EC logic when the client was unable to really create/read/write EC files.
They are causing confusions in other issues about {{NameNode}}. For example, in one of the patches under {{HDFS-10301}}, {{testProcessOverReplicatedAndMissingStripedBlock}} fails because in the test we fake a block report from a DN, with a randomly generated storage ID. The DN itself is never aware of that storage. This is not possible in a real production environment.
{code}
DatanodeStorage storage = new DatanodeStorage(UUID.randomUUID().toString());
StorageReceivedDeletedBlocks[] reports = DFSTestUtil
.makeReportForReceivedBlock(block,
ReceivedDeletedBlockInfo.BlockStatus.RECEIVING_BLOCK, storage);
for (StorageReceivedDeletedBlocks report : reports) {
ns.processIncrementalBlockReport(dn.getDatanodeId(), report);
}
{code}
Now that we have a fully functional EC client, we should remove the old testing logic and use similar logic as non-EC tests (creating real files and emulate blocks missing / being corrupt).
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org