You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Xiao Chen (JIRA)" <ji...@apache.org> on 2018/05/30 18:08:00 UTC
[jira] [Created] (HDFS-13642) Creating a file with block size
smaller than EC policy's cell size should throw
Xiao Chen created HDFS-13642:
--------------------------------
Summary: Creating a file with block size smaller than EC policy's cell size should throw
Key: HDFS-13642
URL: https://issues.apache.org/jira/browse/HDFS-13642
Project: Hadoop HDFS
Issue Type: Bug
Components: erasure-coding
Affects Versions: 3.0.0
Reporter: Xiao Chen
Assignee: Xiao Chen
Attachments: HDFS-13642.01.patch
The following command causes an exception:
{noformat}
hadoop fs -Ddfs.block.size=349696 -put -f lineitem_sixblocks.parquet /test-warehouse/tmp123ec
{noformat}
{noformat}
18/05/25 16:00:59 WARN hdfs.DataStreamer: DataStreamer Exception
java.io.IOException: BlockSize 349696 < lastByteOffsetInBlock, #0: blk_-9223372036854574256_14634, packet seqno: 7 offsetInBlock: 349696 lastPacketInBlock: false lastByteOffsetInBlock: 350208
at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:729)
at org.apache.hadoop.hdfs.StripedDataStreamer.run(StripedDataStreamer.java:46)
18/05/25 16:00:59 WARN hdfs.DFSOutputStream: Failed: offset=4096, length=512, DFSStripedOutputStream:#0: failed, blk_-9223372036854574256_14634
java.io.IOException: BlockSize 349696 < lastByteOffsetInBlock, #0: blk_-9223372036854574256_14634, packet seqno: 7 offsetInBlock: 349696 lastPacketInBlock: false lastByteOffsetInBlock: 350208
at org.apache.hadoop.hdfs.DataStreamer.run(DataStreamer.java:729)
at org.apache.hadoop.hdfs.StripedDataStreamer.run(StripedDataStreamer.java:46)
{noformat}
Then the streamer is confused and hangs.
The local file is under 6MB, the hdfs file has a RS-3-2-1024k EC policy.
Credit to [~tarasbob] for reporting this issue.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org