You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-user@hadoop.apache.org by Dhanasekaran Anbalagan <bu...@gmail.com> on 2013/03/17 14:12:56 UTC
Understand dfs.datanode.max.xcievers
Hi Guys,
We are having few data nodes in an inconsistent state. frequently goes
dead state. Because of DataXceiver Error
In console we seen the Error.
*INFO hdfs.DFSClient: Could not obtain block*
blk_XXXXXXXXXXXXXXXXXXXXXX_YYYYYYYY from any node: java.io.IOException:
currently we have this value.
dfs.datanode.max.xcievers, dfs.datanode.max.transfer.threads = 4096
currently monitering our cluster JVM monitoring using openNMS,
http://i.imgur.com/E1u0fev.png
Not at all hit 4096 value this value for this week . But frequently
the particular node goes dead state why? log says DataXceiver Error.
dfs.datanode.max.xcievers value set to the particular node or across the
cluster Please guide me.
-Dhanasekaran.
Did I learn something today? If not, I wasted it.
Re: Understand dfs.datanode.max.xcievers
Posted by Yanbo Liang <ya...@gmail.com>.
dfs.datanode.max.xcievers value should set across the cluster rather than
particular DataNode.
It means the upper bound on the number of files that the DataNode will
serve at any one time.
2013/3/17 Dhanasekaran Anbalagan <bu...@gmail.com>
> Hi Guys,
>
> We are having few data nodes in an inconsistent state. frequently goes
> dead state. Because of DataXceiver Error
>
> In console we seen the Error.
> *INFO hdfs.DFSClient: Could not obtain block*
> blk_XXXXXXXXXXXXXXXXXXXXXX_YYYYYYYY from any node: java.io.IOException:
>
> currently we have this value.
> dfs.datanode.max.xcievers, dfs.datanode.max.transfer.threads = 4096
>
> currently monitering our cluster JVM monitoring using openNMS,
> http://i.imgur.com/E1u0fev.png
>
> Not at all hit 4096 value this value for this week . But frequently
> the particular node goes dead state why? log says DataXceiver Error.
>
>
> dfs.datanode.max.xcievers value set to the particular node or across the
> cluster Please guide me.
>
> -Dhanasekaran.
>
>
>
>
> Did I learn something today? If not, I wasted it.
>
Re: Understand dfs.datanode.max.xcievers
Posted by Yanbo Liang <ya...@gmail.com>.
dfs.datanode.max.xcievers value should set across the cluster rather than
particular DataNode.
It means the upper bound on the number of files that the DataNode will
serve at any one time.
2013/3/17 Dhanasekaran Anbalagan <bu...@gmail.com>
> Hi Guys,
>
> We are having few data nodes in an inconsistent state. frequently goes
> dead state. Because of DataXceiver Error
>
> In console we seen the Error.
> *INFO hdfs.DFSClient: Could not obtain block*
> blk_XXXXXXXXXXXXXXXXXXXXXX_YYYYYYYY from any node: java.io.IOException:
>
> currently we have this value.
> dfs.datanode.max.xcievers, dfs.datanode.max.transfer.threads = 4096
>
> currently monitering our cluster JVM monitoring using openNMS,
> http://i.imgur.com/E1u0fev.png
>
> Not at all hit 4096 value this value for this week . But frequently
> the particular node goes dead state why? log says DataXceiver Error.
>
>
> dfs.datanode.max.xcievers value set to the particular node or across the
> cluster Please guide me.
>
> -Dhanasekaran.
>
>
>
>
> Did I learn something today? If not, I wasted it.
>
Re: Understand dfs.datanode.max.xcievers
Posted by Yanbo Liang <ya...@gmail.com>.
dfs.datanode.max.xcievers value should set across the cluster rather than
particular DataNode.
It means the upper bound on the number of files that the DataNode will
serve at any one time.
2013/3/17 Dhanasekaran Anbalagan <bu...@gmail.com>
> Hi Guys,
>
> We are having few data nodes in an inconsistent state. frequently goes
> dead state. Because of DataXceiver Error
>
> In console we seen the Error.
> *INFO hdfs.DFSClient: Could not obtain block*
> blk_XXXXXXXXXXXXXXXXXXXXXX_YYYYYYYY from any node: java.io.IOException:
>
> currently we have this value.
> dfs.datanode.max.xcievers, dfs.datanode.max.transfer.threads = 4096
>
> currently monitering our cluster JVM monitoring using openNMS,
> http://i.imgur.com/E1u0fev.png
>
> Not at all hit 4096 value this value for this week . But frequently
> the particular node goes dead state why? log says DataXceiver Error.
>
>
> dfs.datanode.max.xcievers value set to the particular node or across the
> cluster Please guide me.
>
> -Dhanasekaran.
>
>
>
>
> Did I learn something today? If not, I wasted it.
>
Re: Understand dfs.datanode.max.xcievers
Posted by Yanbo Liang <ya...@gmail.com>.
dfs.datanode.max.xcievers value should set across the cluster rather than
particular DataNode.
It means the upper bound on the number of files that the DataNode will
serve at any one time.
2013/3/17 Dhanasekaran Anbalagan <bu...@gmail.com>
> Hi Guys,
>
> We are having few data nodes in an inconsistent state. frequently goes
> dead state. Because of DataXceiver Error
>
> In console we seen the Error.
> *INFO hdfs.DFSClient: Could not obtain block*
> blk_XXXXXXXXXXXXXXXXXXXXXX_YYYYYYYY from any node: java.io.IOException:
>
> currently we have this value.
> dfs.datanode.max.xcievers, dfs.datanode.max.transfer.threads = 4096
>
> currently monitering our cluster JVM monitoring using openNMS,
> http://i.imgur.com/E1u0fev.png
>
> Not at all hit 4096 value this value for this week . But frequently
> the particular node goes dead state why? log says DataXceiver Error.
>
>
> dfs.datanode.max.xcievers value set to the particular node or across the
> cluster Please guide me.
>
> -Dhanasekaran.
>
>
>
>
> Did I learn something today? If not, I wasted it.
>