You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by "Andrian Jardan (JIRA)" <ji...@apache.org> on 2016/08/17 16:20:20 UTC
[jira] [Created] (HDFS-10771) Error while reading block
java.io.IOException: Need xx bytes, but only yy bytes available
Andrian Jardan created HDFS-10771:
-------------------------------------
Summary: Error while reading block java.io.IOException: Need xx bytes, but only yy bytes available
Key: HDFS-10771
URL: https://issues.apache.org/jira/browse/HDFS-10771
Project: Hadoop HDFS
Issue Type: Bug
Components: datanode
Affects Versions: 2.6.0
Environment: Hadoop 2.6.0-cdh5.7.0
Subversion http://github.com/cloudera/hadoop -r c00978c67b0d3fe9f3b896b5030741bd40bf541a
Compiled by jenkins on 2016-03-23T18:36Z
Compiled with protoc 2.5.0
From source with checksum b2eabfa328e763c88cb14168f9b372
This command was run using /opt/cloudera/parcels/CDH-5.7.0-1.cdh5.7.0.p0.45/jars/hadoop-common-2.6.0-cdh5.7.0.jar
Reporter: Andrian Jardan
Priority: Minor
Got an error every time we try to "distcp" a file from a cluster (cp works just fine). Here is what I found in the log on the data node distcp tries to copy from
{code}
2016-08-17 18:02:49,073 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: opReadBlock BP-626139917-127.0.0.1-1438009948483:blk_1152071533_78503164 received exception java.io.IOException: Need 21925420 bytes, but only 16682940 bytes available
2016-08-17 18:02:49,075 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(127.0.0.1, datanodeUuid=b6c35b7e-9ab7-4b1b-9258-69988346142b, infoPort=50075, infoSecurePort=0, ipcPort=50020, storageInfo=lv=-56;cid=cluster6;nsid=895831559;c=0):Got exception while serving BP-626139917-127.0.0.1-1438009948483:blk_1152071533_78503164 to /127.0.0.2:43758
java.io.IOException: Need 21925420 bytes, but only 16682940 bytes available
at org.apache.hadoop.hdfs.server.datanode.BlockSender.waitForMinLength(BlockSender.java:473)
at org.apache.hadoop.hdfs.server.datanode.BlockSender.<init>(BlockSender.java:241)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:531)
at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:148)
at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:246)
at java.lang.Thread.run(Thread.java:745)
2016-08-17 18:02:49,075 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: dn:50010:DataXceiver error processing READ_BLOCK operation src: /127.0.0.2:43758 dst: /127.0.0.1:50010
java.io.IOException: Need 21925420 bytes, but only 16682940 bytes available
at org.apache.hadoop.hdfs.server.datanode.BlockSender.waitForMinLength(BlockSender.java:473)
at org.apache.hadoop.hdfs.server.datanode.BlockSender.<init>(BlockSender.java:241)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.readBlock(DataXceiver.java:531)
at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opReadBlock(Receiver.java:148)
at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:103)
at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:246)
at java.lang.Thread.run(Thread.java:745)
{code}
I suppose it is talking about RAM (heap) ?
Why doesn't it retry from another DataNode (replication factor is 3 for this file)?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org