You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Jing Zhao (JIRA)" <ji...@apache.org> on 2016/01/13 23:49:40 UTC
[jira] [Commented] (HDFS-9646) ErasureCodingWorker may fail when
recovering data blocks with length less than the first internal block
[ https://issues.apache.org/jira/browse/HDFS-9646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097200#comment-15097200 ]
Jing Zhao commented on HDFS-9646:
---------------------------------
The failure can also be reproduced with the following change on {{TestRecoverStripedFile}}:
{code}
--- a/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRecoverStripedFile.java
+++ b/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/TestRecoverStripedFile.java
@@ -212,7 +212,7 @@ private void assertFileBlocksRecovery(String fileName, int fileLen,
int[] toDead = new int[toRecoverBlockNum];
int n = 0;
- for (int i = 0; i < indices.length; i++) {
+ for (int i = indices.length - 1; i >= 0; i--) {
if (n < toRecoverBlockNum) {
if (recovery == 0) {
if (indices[i] >= dataBlkNum) {
{code}
> ErasureCodingWorker may fail when recovering data blocks with length less than the first internal block
> -------------------------------------------------------------------------------------------------------
>
> Key: HDFS-9646
> URL: https://issues.apache.org/jira/browse/HDFS-9646
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: erasure-coding
> Affects Versions: 3.0.0
> Reporter: Takuya Fukudome
> Assignee: Jing Zhao
> Priority: Critical
> Attachments: test-reconstruct-stripe-file.patch
>
>
> This is reported by [~tfukudom]: ErasureCodingWorker may fail with the following exception when recovering a non-full internal block.
> {code}
> 2016-01-06 11:14:44,740 WARN datanode.DataNode (ErasureCodingWorker.java:run(467)) - Failed to recover striped block: BP-987302662-172.29.4.13-1450757377698:blk_-92233720368
> 54322288_29751
> java.io.IOException: Transfer failed for all targets.
> at org.apache.hadoop.hdfs.server.datanode.erasurecode.ErasureCodingWorker$ReconstructAndTransferBlock.run(ErasureCodingWorker.java:455)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)