You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by zhoushuaifeng <zh...@gmail.com> on 2013/04/18 17:01:36 UTC
setting hdfs balancer bandwidth doesn't work
Hi,
I set the hdfs balance bandwidth from 1048576 to 104857600, but it doesn't work, what's wrong?
Does anyone encounter the same problem?
Thanks a lot.
<property>
<name>dfs.balance.bandwidthPerSec</name>
<value>104857600</value>
<description>
Specifies the maximum amount of bandwidth that each datanode
can utilize for the balancing purpose in term of
the number of bytes per second.
</description>
</property>
zhoushuaifeng
Re: Re: setting hdfs balancer bandwidth doesn't work
Posted by zhoushuaifeng <zh...@gmail.com>.
Yes, I restart my cluster.
It may be the OS problem. My cluster has 5 nodes, 4 are redhat, and the last added one is suse11. The bandwidth setting works on the redhat nodes, but not on the suse one. My be it doesn't work well when the cluster are composed by different systems.
Is it a bug?
Thanks
zhoushuaifeng
发件人： yuchen xie
发送时间： 2013-04-19 14:50
收件人： user
主题： Re: setting hdfs balancer bandwidth doesn't work
have you try to restart your cluster or execute "$HADOOP_HOME/bin/hadoop dfsadmin -setBalanacerBandwidth 104857600" ?
2013/4/19 周帅锋 <zh...@gmail.com>
I set bandwidthPerSec = 104857600, but when I add a new data node, and run hadoop balancer, the bandwidth is only 1MB/s, and the datanode log shows that:
org.apache.hadoop.hdfs.server.
datanode.DataNode: Balancing bandwith is 1048576 bytes/s
My hadoop core version is 1.0.3
Thanks
2013/4/19 Thanh Do <th...@cs.wisc.edu>
What do you mean by "doesn't work"?
On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com> wrote:
Hi,
I set the hdfs balance bandwidth from 1048576 to 104857600, but it doesn't work, what's wrong?
Does anyone encounter the same problem?
Thanks a lot.
<property>
<name>dfs.balance.bandwidthPerSec</name>
<value>104857600</value>
<description>
Specifies the maximum amount of bandwidth that each datanode
can utilize for the balancing purpose in term of
the number of bytes per second.
</description>
</property>
zhoushuaifeng
Re: Re: setting hdfs balancer bandwidth doesn't work
Posted by zhoushuaifeng <zh...@gmail.com>.
Yes, I restart my cluster.
It may be the OS problem. My cluster has 5 nodes, 4 are redhat, and the last added one is suse11. The bandwidth setting works on the redhat nodes, but not on the suse one. My be it doesn't work well when the cluster are composed by different systems.
Is it a bug?
Thanks
zhoushuaifeng
发件人： yuchen xie
发送时间： 2013-04-19 14:50
收件人： user
主题： Re: setting hdfs balancer bandwidth doesn't work
have you try to restart your cluster or execute "$HADOOP_HOME/bin/hadoop dfsadmin -setBalanacerBandwidth 104857600" ?
2013/4/19 周帅锋 <zh...@gmail.com>
I set bandwidthPerSec = 104857600, but when I add a new data node, and run hadoop balancer, the bandwidth is only 1MB/s, and the datanode log shows that:
org.apache.hadoop.hdfs.server.
datanode.DataNode: Balancing bandwith is 1048576 bytes/s
My hadoop core version is 1.0.3
Thanks
2013/4/19 Thanh Do <th...@cs.wisc.edu>
What do you mean by "doesn't work"?
On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com> wrote:
Hi,
I set the hdfs balance bandwidth from 1048576 to 104857600, but it doesn't work, what's wrong?
Does anyone encounter the same problem?
Thanks a lot.
<property>
<name>dfs.balance.bandwidthPerSec</name>
<value>104857600</value>
<description>
Specifies the maximum amount of bandwidth that each datanode
can utilize for the balancing purpose in term of
the number of bytes per second.
</description>
</property>
zhoushuaifeng
Re: Re: setting hdfs balancer bandwidth doesn't work
Posted by zhoushuaifeng <zh...@gmail.com>.
Yes, I restart my cluster.
It may be the OS problem. My cluster has 5 nodes, 4 are redhat, and the last added one is suse11. The bandwidth setting works on the redhat nodes, but not on the suse one. My be it doesn't work well when the cluster are composed by different systems.
Is it a bug?
Thanks
zhoushuaifeng
发件人： yuchen xie
发送时间： 2013-04-19 14:50
收件人： user
主题： Re: setting hdfs balancer bandwidth doesn't work
have you try to restart your cluster or execute "$HADOOP_HOME/bin/hadoop dfsadmin -setBalanacerBandwidth 104857600" ?
2013/4/19 周帅锋 <zh...@gmail.com>
I set bandwidthPerSec = 104857600, but when I add a new data node, and run hadoop balancer, the bandwidth is only 1MB/s, and the datanode log shows that:
org.apache.hadoop.hdfs.server.
datanode.DataNode: Balancing bandwith is 1048576 bytes/s
My hadoop core version is 1.0.3
Thanks
2013/4/19 Thanh Do <th...@cs.wisc.edu>
What do you mean by "doesn't work"?
On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com> wrote:
Hi,
I set the hdfs balance bandwidth from 1048576 to 104857600, but it doesn't work, what's wrong?
Does anyone encounter the same problem?
Thanks a lot.
<property>
<name>dfs.balance.bandwidthPerSec</name>
<value>104857600</value>
<description>
Specifies the maximum amount of bandwidth that each datanode
can utilize for the balancing purpose in term of
the number of bytes per second.
</description>
</property>
zhoushuaifeng
Re: Re: setting hdfs balancer bandwidth doesn't work
Posted by zhoushuaifeng <zh...@gmail.com>.
Yes, I restart my cluster.
It may be the OS problem. My cluster has 5 nodes, 4 are redhat, and the last added one is suse11. The bandwidth setting works on the redhat nodes, but not on the suse one. My be it doesn't work well when the cluster are composed by different systems.
Is it a bug?
Thanks
zhoushuaifeng
发件人： yuchen xie
发送时间： 2013-04-19 14:50
收件人： user
主题： Re: setting hdfs balancer bandwidth doesn't work
have you try to restart your cluster or execute "$HADOOP_HOME/bin/hadoop dfsadmin -setBalanacerBandwidth 104857600" ?
2013/4/19 周帅锋 <zh...@gmail.com>
I set bandwidthPerSec = 104857600, but when I add a new data node, and run hadoop balancer, the bandwidth is only 1MB/s, and the datanode log shows that:
org.apache.hadoop.hdfs.server.
datanode.DataNode: Balancing bandwith is 1048576 bytes/s
My hadoop core version is 1.0.3
Thanks
2013/4/19 Thanh Do <th...@cs.wisc.edu>
What do you mean by "doesn't work"?
On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com> wrote:
Hi,
I set the hdfs balance bandwidth from 1048576 to 104857600, but it doesn't work, what's wrong?
Does anyone encounter the same problem?
Thanks a lot.
<property>
<name>dfs.balance.bandwidthPerSec</name>
<value>104857600</value>
<description>
Specifies the maximum amount of bandwidth that each datanode
can utilize for the balancing purpose in term of
the number of bytes per second.
</description>
</property>
zhoushuaifeng
Re: setting hdfs balancer bandwidth doesn't work
Posted by yuchen xie <xi...@gmail.com>.
have you try to restart your cluster or execute "$HADOOP_HOME/bin/hadoop
dfsadmin -setBalanacerBandwidth 104857600" ?
2013/4/19 周帅锋 <zh...@gmail.com>
> I set bandwidthPerSec = 104857600, but when I add a new data node, and run
> hadoop balancer, the bandwidth is only 1MB/s, and the datanode log shows
> that:
> org.apache.hadoop.hdfs.server.
> datanode.DataNode: Balancing bandwith is 1048576 bytes/s
>
> My hadoop core version is 1.0.3
>
> Thanks
>
>
> 2013/4/19 Thanh Do <th...@cs.wisc.edu>
>
>> What do you mean by "doesn't work"?
>>
>>
>> On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com>wrote:
>>
>>> **
>>> Hi,
>>> I set the hdfs balance bandwidth from 1048576 to 104857600, but it
>>> doesn't work, what's wrong?
>>> Does anyone encounter the same problem?
>>> Thanks a lot.
>>>
>>> <property>
>>> <name>dfs.balance.bandwidthPerSec</name>
>>> <value>104857600</value>
>>> <description>
>>> Specifies the maximum amount of bandwidth that each datanode
>>> can utilize for the balancing purpose in term of
>>> the number of bytes per second.
>>> </description>
>>> </property>
>>>
>>> ------------------------------
>>> zhoushuaifeng
>>>
>>
>>
>
Re: setting hdfs balancer bandwidth doesn't work
Posted by yuchen xie <xi...@gmail.com>.
have you try to restart your cluster or execute "$HADOOP_HOME/bin/hadoop
dfsadmin -setBalanacerBandwidth 104857600" ?
2013/4/19 周帅锋 <zh...@gmail.com>
> I set bandwidthPerSec = 104857600, but when I add a new data node, and run
> hadoop balancer, the bandwidth is only 1MB/s, and the datanode log shows
> that:
> org.apache.hadoop.hdfs.server.
> datanode.DataNode: Balancing bandwith is 1048576 bytes/s
>
> My hadoop core version is 1.0.3
>
> Thanks
>
>
> 2013/4/19 Thanh Do <th...@cs.wisc.edu>
>
>> What do you mean by "doesn't work"?
>>
>>
>> On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com>wrote:
>>
>>> **
>>> Hi,
>>> I set the hdfs balance bandwidth from 1048576 to 104857600, but it
>>> doesn't work, what's wrong?
>>> Does anyone encounter the same problem?
>>> Thanks a lot.
>>>
>>> <property>
>>> <name>dfs.balance.bandwidthPerSec</name>
>>> <value>104857600</value>
>>> <description>
>>> Specifies the maximum amount of bandwidth that each datanode
>>> can utilize for the balancing purpose in term of
>>> the number of bytes per second.
>>> </description>
>>> </property>
>>>
>>> ------------------------------
>>> zhoushuaifeng
>>>
>>
>>
>
Re: setting hdfs balancer bandwidth doesn't work
Posted by yuchen xie <xi...@gmail.com>.
have you try to restart your cluster or execute "$HADOOP_HOME/bin/hadoop
dfsadmin -setBalanacerBandwidth 104857600" ?
2013/4/19 周帅锋 <zh...@gmail.com>
> I set bandwidthPerSec = 104857600, but when I add a new data node, and run
> hadoop balancer, the bandwidth is only 1MB/s, and the datanode log shows
> that:
> org.apache.hadoop.hdfs.server.
> datanode.DataNode: Balancing bandwith is 1048576 bytes/s
>
> My hadoop core version is 1.0.3
>
> Thanks
>
>
> 2013/4/19 Thanh Do <th...@cs.wisc.edu>
>
>> What do you mean by "doesn't work"?
>>
>>
>> On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com>wrote:
>>
>>> **
>>> Hi,
>>> I set the hdfs balance bandwidth from 1048576 to 104857600, but it
>>> doesn't work, what's wrong?
>>> Does anyone encounter the same problem?
>>> Thanks a lot.
>>>
>>> <property>
>>> <name>dfs.balance.bandwidthPerSec</name>
>>> <value>104857600</value>
>>> <description>
>>> Specifies the maximum amount of bandwidth that each datanode
>>> can utilize for the balancing purpose in term of
>>> the number of bytes per second.
>>> </description>
>>> </property>
>>>
>>> ------------------------------
>>> zhoushuaifeng
>>>
>>
>>
>
Re: setting hdfs balancer bandwidth doesn't work
Posted by yuchen xie <xi...@gmail.com>.
have you try to restart your cluster or execute "$HADOOP_HOME/bin/hadoop
dfsadmin -setBalanacerBandwidth 104857600" ?
2013/4/19 周帅锋 <zh...@gmail.com>
> I set bandwidthPerSec = 104857600, but when I add a new data node, and run
> hadoop balancer, the bandwidth is only 1MB/s, and the datanode log shows
> that:
> org.apache.hadoop.hdfs.server.
> datanode.DataNode: Balancing bandwith is 1048576 bytes/s
>
> My hadoop core version is 1.0.3
>
> Thanks
>
>
> 2013/4/19 Thanh Do <th...@cs.wisc.edu>
>
>> What do you mean by "doesn't work"?
>>
>>
>> On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com>wrote:
>>
>>> **
>>> Hi,
>>> I set the hdfs balance bandwidth from 1048576 to 104857600, but it
>>> doesn't work, what's wrong?
>>> Does anyone encounter the same problem?
>>> Thanks a lot.
>>>
>>> <property>
>>> <name>dfs.balance.bandwidthPerSec</name>
>>> <value>104857600</value>
>>> <description>
>>> Specifies the maximum amount of bandwidth that each datanode
>>> can utilize for the balancing purpose in term of
>>> the number of bytes per second.
>>> </description>
>>> </property>
>>>
>>> ------------------------------
>>> zhoushuaifeng
>>>
>>
>>
>
Re: setting hdfs balancer bandwidth doesn't work
Posted by 周帅锋 <zh...@gmail.com>.
I set bandwidthPerSec = 104857600, but when I add a new data node, and run
hadoop balancer, the bandwidth is only 1MB/s, and the datanode log shows
that:
org.apache.hadoop.hdfs.server.
datanode.DataNode: Balancing bandwith is 1048576 bytes/s
My hadoop core version is 1.0.3
Thanks
2013/4/19 Thanh Do <th...@cs.wisc.edu>
> What do you mean by "doesn't work"?
>
>
> On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com>wrote:
>
>> **
>> Hi,
>> I set the hdfs balance bandwidth from 1048576 to 104857600, but it
>> doesn't work, what's wrong?
>> Does anyone encounter the same problem?
>> Thanks a lot.
>>
>> <property>
>> <name>dfs.balance.bandwidthPerSec</name>
>> <value>104857600</value>
>> <description>
>> Specifies the maximum amount of bandwidth that each datanode
>> can utilize for the balancing purpose in term of
>> the number of bytes per second.
>> </description>
>> </property>
>>
>> ------------------------------
>> zhoushuaifeng
>>
>
>
Re: setting hdfs balancer bandwidth doesn't work
Posted by 周帅锋 <zh...@gmail.com>.
I set bandwidthPerSec = 104857600, but when I add a new data node, and run
hadoop balancer, the bandwidth is only 1MB/s, and the datanode log shows
that:
org.apache.hadoop.hdfs.server.
datanode.DataNode: Balancing bandwith is 1048576 bytes/s
My hadoop core version is 1.0.3
Thanks
2013/4/19 Thanh Do <th...@cs.wisc.edu>
> What do you mean by "doesn't work"?
>
>
> On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com>wrote:
>
>> **
>> Hi,
>> I set the hdfs balance bandwidth from 1048576 to 104857600, but it
>> doesn't work, what's wrong?
>> Does anyone encounter the same problem?
>> Thanks a lot.
>>
>> <property>
>> <name>dfs.balance.bandwidthPerSec</name>
>> <value>104857600</value>
>> <description>
>> Specifies the maximum amount of bandwidth that each datanode
>> can utilize for the balancing purpose in term of
>> the number of bytes per second.
>> </description>
>> </property>
>>
>> ------------------------------
>> zhoushuaifeng
>>
>
>
Re: setting hdfs balancer bandwidth doesn't work
Posted by 周帅锋 <zh...@gmail.com>.
I set bandwidthPerSec = 104857600, but when I add a new data node, and run
hadoop balancer, the bandwidth is only 1MB/s, and the datanode log shows
that:
org.apache.hadoop.hdfs.server.
datanode.DataNode: Balancing bandwith is 1048576 bytes/s
My hadoop core version is 1.0.3
Thanks
2013/4/19 Thanh Do <th...@cs.wisc.edu>
> What do you mean by "doesn't work"?
>
>
> On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com>wrote:
>
>> **
>> Hi,
>> I set the hdfs balance bandwidth from 1048576 to 104857600, but it
>> doesn't work, what's wrong?
>> Does anyone encounter the same problem?
>> Thanks a lot.
>>
>> <property>
>> <name>dfs.balance.bandwidthPerSec</name>
>> <value>104857600</value>
>> <description>
>> Specifies the maximum amount of bandwidth that each datanode
>> can utilize for the balancing purpose in term of
>> the number of bytes per second.
>> </description>
>> </property>
>>
>> ------------------------------
>> zhoushuaifeng
>>
>
>
Re: setting hdfs balancer bandwidth doesn't work
Posted by 周帅锋 <zh...@gmail.com>.
I set bandwidthPerSec = 104857600, but when I add a new data node, and run
hadoop balancer, the bandwidth is only 1MB/s, and the datanode log shows
that:
org.apache.hadoop.hdfs.server.
datanode.DataNode: Balancing bandwith is 1048576 bytes/s
My hadoop core version is 1.0.3
Thanks
2013/4/19 Thanh Do <th...@cs.wisc.edu>
> What do you mean by "doesn't work"?
>
>
> On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com>wrote:
>
>> **
>> Hi,
>> I set the hdfs balance bandwidth from 1048576 to 104857600, but it
>> doesn't work, what's wrong?
>> Does anyone encounter the same problem?
>> Thanks a lot.
>>
>> <property>
>> <name>dfs.balance.bandwidthPerSec</name>
>> <value>104857600</value>
>> <description>
>> Specifies the maximum amount of bandwidth that each datanode
>> can utilize for the balancing purpose in term of
>> the number of bytes per second.
>> </description>
>> </property>
>>
>> ------------------------------
>> zhoushuaifeng
>>
>
>
Re: setting hdfs balancer bandwidth doesn't work
Posted by Thanh Do <th...@cs.wisc.edu>.
What do you mean by "doesn't work"?
On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com>wrote:
> **
> Hi,
> I set the hdfs balance bandwidth from 1048576 to 104857600, but it doesn't
> work, what's wrong?
> Does anyone encounter the same problem?
> Thanks a lot.
>
> <property>
> <name>dfs.balance.bandwidthPerSec</name>
> <value>104857600</value>
> <description>
> Specifies the maximum amount of bandwidth that each datanode
> can utilize for the balancing purpose in term of
> the number of bytes per second.
> </description>
> </property>
>
> ------------------------------
> zhoushuaifeng
>
Re: setting hdfs balancer bandwidth doesn't work
Posted by Thanh Do <th...@cs.wisc.edu>.
What do you mean by "doesn't work"?
On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com>wrote:
> **
> Hi,
> I set the hdfs balance bandwidth from 1048576 to 104857600, but it doesn't
> work, what's wrong?
> Does anyone encounter the same problem?
> Thanks a lot.
>
> <property>
> <name>dfs.balance.bandwidthPerSec</name>
> <value>104857600</value>
> <description>
> Specifies the maximum amount of bandwidth that each datanode
> can utilize for the balancing purpose in term of
> the number of bytes per second.
> </description>
> </property>
>
> ------------------------------
> zhoushuaifeng
>
Re: setting hdfs balancer bandwidth doesn't work
Posted by Thanh Do <th...@cs.wisc.edu>.
What do you mean by "doesn't work"?
On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com>wrote:
> **
> Hi,
> I set the hdfs balance bandwidth from 1048576 to 104857600, but it doesn't
> work, what's wrong?
> Does anyone encounter the same problem?
> Thanks a lot.
>
> <property>
> <name>dfs.balance.bandwidthPerSec</name>
> <value>104857600</value>
> <description>
> Specifies the maximum amount of bandwidth that each datanode
> can utilize for the balancing purpose in term of
> the number of bytes per second.
> </description>
> </property>
>
> ------------------------------
> zhoushuaifeng
>
Re: setting hdfs balancer bandwidth doesn't work
Posted by Thanh Do <th...@cs.wisc.edu>.
What do you mean by "doesn't work"?
On Thu, Apr 18, 2013 at 10:01 AM, zhoushuaifeng <zh...@gmail.com>wrote:
> **
> Hi,
> I set the hdfs balance bandwidth from 1048576 to 104857600, but it doesn't
> work, what's wrong?
> Does anyone encounter the same problem?
> Thanks a lot.
>
> <property>
> <name>dfs.balance.bandwidthPerSec</name>
> <value>104857600</value>
> <description>
> Specifies the maximum amount of bandwidth that each datanode
> can utilize for the balancing purpose in term of
> the number of bytes per second.
> </description>
> </property>
>
> ------------------------------
> zhoushuaifeng
>