You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-issues@hadoop.apache.org by "Tsz Wo (Nicholas), SZE (Commented) (JIRA)" <ji...@apache.org> on 2012/03/15 21:32:38 UTC
[jira] [Commented] (HDFS-3100) failed to append data using webhdfs
[ https://issues.apache.org/jira/browse/HDFS-3100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13230519#comment-13230519 ]
Tsz Wo (Nicholas), SZE commented on HDFS-3100:
----------------------------------------------
It looks like that the BlockPoolSliceScanner incorrectly makes the replica as corrupted.
{noformat}
2012-03-15 13:26:27,317 WARN datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(419)) - First Verification failed for BP-426067686-10.10.10.105-1331843180223:blk_-951537730291424878_1083
java.io.IOException: Stream closed
at java.io.BufferedInputStream.getInIfOpen(BufferedInputStream.java:134)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
at java.io.DataInputStream.readShort(DataInputStream.java:295)
at org.apache.hadoop.hdfs.server.datanode.BlockMetadataHeader.readHeader(BlockMetadataHeader.java:78)
at org.apache.hadoop.hdfs.server.datanode.BlockSender.<init>(BlockSender.java:228)
at org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner.verifyBlock(BlockPoolSliceScanner.java:378)
at org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner.verifyFirstBlock(BlockPoolSliceScanner.java:463)
at org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner.scan(BlockPoolSliceScanner.java:594)
at org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner.scanBlockPoolSlice(BlockPoolSliceScanner.java:570)
at org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.run(DataBlockScanner.java:95)
at java.lang.Thread.run(Thread.java:680)
2012-03-15 13:26:27,320 WARN datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:verifyBlock(419)) - Second Verification failed for BP-426067686-10.10.10.105-1331843180223:blk_-951537730291424878_1083
java.io.IOException: Stream closed
at java.io.BufferedInputStream.getInIfOpen(BufferedInputStream.java:134)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
at java.io.DataInputStream.readShort(DataInputStream.java:295)
at org.apache.hadoop.hdfs.server.datanode.BlockMetadataHeader.readHeader(BlockMetadataHeader.java:78)
at org.apache.hadoop.hdfs.server.datanode.BlockSender.<init>(BlockSender.java:228)
at org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner.verifyBlock(BlockPoolSliceScanner.java:378)
at org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner.verifyFirstBlock(BlockPoolSliceScanner.java:463)
at org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner.scan(BlockPoolSliceScanner.java:594)
at org.apache.hadoop.hdfs.server.datanode.BlockPoolSliceScanner.scanBlockPoolSlice(BlockPoolSliceScanner.java:570)
at org.apache.hadoop.hdfs.server.datanode.DataBlockScanner.run(DataBlockScanner.java:95)
at java.lang.Thread.run(Thread.java:680)
2012-03-15 13:26:27,320 WARN datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:addBlock(234)) - Adding an already existing block BP-426067686-10.10.10.105-1331843180223:blk_-951537730291424878_1084
2012-03-15 13:26:27,320 INFO datanode.BlockPoolSliceScanner (BlockPoolSliceScanner.java:handleScanFailure(301)) - Reporting bad block BP-426067686-10.10.10.105-1331843180223:blk_-951537730291424878_1083
2012-03-15 13:26:27,321 INFO DataNode.clienttrace (BlockReceiver.java:run(1062)) - src: /127.0.0.1:54170, dest: /127.0.0.1:54083, bytes: 84992, op: HDFS_WRITE, cliID: DFSClient_NONMAPREDUCE_2012624116_1, offset: 0, srvID: DS-600201831-10.10.10.105-54083-1331843
{noformat}
> failed to append data using webhdfs
> -----------------------------------
>
> Key: HDFS-3100
> URL: https://issues.apache.org/jira/browse/HDFS-3100
> Project: Hadoop HDFS
> Issue Type: Bug
> Components: data-node
> Affects Versions: 0.23.1
> Reporter: Zhanwei.Wang
> Assignee: Tsz Wo (Nicholas), SZE
> Attachments: hadoop-wangzw-datanode-ubuntu.log, hadoop-wangzw-namenode-ubuntu.log, test.sh, testAppend.patch
>
>
> STEP：
> 1, deploy a single node hdfs 0.23.1 cluster and configure hdfs as:
> A) enable webhdfs
> B) enable append
> C) disable permissions
> 2, start hdfs
> 3, run the test script as attached
> RESULT:
> expected: a file named testFile should be created and populated with 32K * 5000 zeros, HDFS should be OK.
> I got: script cannot be finished, file has been created but not be populated as expected, actually append operation failed.
> Datanode log shows that, blockscaner report a bad replica and nanenode decide to delete it. Since it is a single node cluster, append fail. It makes no sense that the script failed every time.
> Datanode and Namenode logs are attached.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira