You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@hbase.apache.org by Arati Patro <rt...@gmail.com> on 2012/11/29 07:28:35 UTC
DataXceiver java.io.EOFException
Hi,
I'm using hbase version 0.94.1 and hadoop version 1.0.3
I'm running HBase + HDFS on a 4 node cluster (48 GB RAM, 12TB DiskSpace on
each node).
1 HMaster + NameNode and
3 HRegionServer + DataNode
Replication is set to 2
Running 6 MapReduce jobs (two of which run concurrently)
When MapReduce jobs are triggered the datanode log shows exceptions like
this:
2012-11-26 17:37:38,672 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: writeBlock
blk_-4043001352486758862_3090 received exception java.io.EOFException
2012-11-26 17:37:38,673 ERROR
org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(
10.63.63.249:50010,
storageID=DS-778870342-10.63.63.249-50010-1353922061110, infoPort=50075,
ipcPort=50020):DataXceiver
java.io.EOFException
at java.io.DataInputStream.readShort(DataInputStream.java:298)
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:351)
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:107)
at java.lang.Thread.run(Thread.java:619)
2012-11-26 17:37:38,675 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
blk_5001084339060873354_3090 src: /10.63.63.249:37109 dest: /
10.63.63.249:50010
Xciever value is set as below in hdfs-site.xml
<property>
<name>dfs.datanode.max.xcievers</name>
<value>16384</value>
</property>
Could anyone give some more light on why this is happening.
Thanks,
Arati Patro
Re: DataXceiver java.io.EOFException
Posted by Arati Patro <ar...@gmail.com>.
Hi,
Tried changing dfs.datanode.handler.count to 6. No improvement seen over 24
hours of run. Same amount of DataXceiver exceptions seen.
Any help?
Thanks,
Arati Patro
Re: DataXceiver java.io.EOFException
Posted by Arati Patro <ar...@gmail.com>.
Thanks for your replies!
Yes the dfs.datanode.handler.count Will increase as suggested.
Also, the dfs.namenode.handler.count is also configured to the default 10.
Should this be increased? If yes, what benefit would it give?
No there aren't any checkDisk exceptions in the logs.
Thanks again,
Arati Patro
Re: DataXceiver java.io.EOFException
Posted by Inder Pall <in...@gmail.com>.
do you see lots of checkDisk exceptions in datanode logs
- inder
On Thu, Nov 29, 2012 at 2:02 PM, Viral Bajaria <vi...@gmail.com>wrote:
> xcievers. I would recommend increasing that to 6 and see if the error goes
>
--
- Inder
"You are average of the 5 people you spend the most time with"
Re: DataXceiver java.io.EOFException
Posted by Viral Bajaria <vi...@gmail.com>.
Hi,
Is your dfs.datanode.handler.count set to the default value of 3 ? I think
I bumped it up when I got these exceptions and the issue wasn't due to
xcievers. I would recommend increasing that to 6 and see if the error goes
away or the frequency of the error decreases.
Thanks,
Viral
On Wed, Nov 28, 2012 at 10:32 PM, Arati Patro <ar...@gmail.com> wrote:
> Hi,
>
> I'm using hbase version 0.94.1 and hadoop version 1.0.3
>
> I'm running HBase + HDFS on a 4 node cluster (48 GB RAM, 12TB DiskSpace on
> each node).
>
> 1 HMaster + NameNode and
> 3 HRegionServer + DataNode
>
> Replication is set to 2
>
> Running 6 MapReduce jobs (two of which run concurrently)
>
>
> When MapReduce jobs are triggered the datanode log shows exceptions like
> this:
>
> 2012-11-26 17:37:38,672 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: writeBlock
> blk_-4043001352486758862_3090 received exception java.io.EOFException
>
> 2012-11-26 17:37:38,673 ERROR
> org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(
> 10.63.63.249:50010,
> storageID=DS-778870342-10.63.63.249-50010-1353922061110, infoPort=50075,
> ipcPort=50020):DataXceiver
>
> java.io.EOFException
>
> at java.io.DataInputStream.readShort(DataInputStream.java:298)
>
> at
>
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:351)
>
> at
>
> org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:107)
>
> at java.lang.Thread.run(Thread.java:619)
>
> 2012-11-26 17:37:38,675 INFO
> org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
> blk_5001084339060873354_3090 src: /10.63.63.249:37109 dest: /
> 10.63.63.249:50010
>
>
> Xciever value is set as below in hdfs-site.xml
>
> <property>
> <name>dfs.datanode.max.xcievers</name>
> <value>16384</value>
> </property>
>
>
> Could anyone give some more light on why this is happening.
>
>
> Thanks,
> Arati Patro
>
DataXceiver java.io.EOFException
Posted by Arati Patro <ar...@gmail.com>.
Hi,
I'm using hbase version 0.94.1 and hadoop version 1.0.3
I'm running HBase + HDFS on a 4 node cluster (48 GB RAM, 12TB DiskSpace on
each node).
1 HMaster + NameNode and
3 HRegionServer + DataNode
Replication is set to 2
Running 6 MapReduce jobs (two of which run concurrently)
When MapReduce jobs are triggered the datanode log shows exceptions like
this:
2012-11-26 17:37:38,672 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: writeBlock
blk_-4043001352486758862_3090 received exception java.io.EOFException
2012-11-26 17:37:38,673 ERROR
org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration(
10.63.63.249:50010,
storageID=DS-778870342-10.63.63.249-50010-1353922061110, infoPort=50075,
ipcPort=50020):DataXceiver
java.io.EOFException
at java.io.DataInputStream.readShort(DataInputStream.java:298)
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.writeBlock(DataXceiver.java:351)
at
org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:107)
at java.lang.Thread.run(Thread.java:619)
2012-11-26 17:37:38,675 INFO
org.apache.hadoop.hdfs.server.datanode.DataNode: Receiving block
blk_5001084339060873354_3090 src: /10.63.63.249:37109 dest: /
10.63.63.249:50010
Xciever value is set as below in hdfs-site.xml
<property>
<name>dfs.datanode.max.xcievers</name>
<value>16384</value>
</property>
Could anyone give some more light on why this is happening.
Thanks,
Arati Patro