You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by ch huang <ju...@gmail.com> on 2014/11/17 10:03:15 UTC
question about speed up the hdfs balancer
hi,maillist:
i want to balancer for my HDFS cluster ,but the default speed is
1M/s , i want to set the option dfs.balance.bandwidthPerSec to 20M/s
,surpose i issue balancer command on node A ,i do not know if i just set
the option on node A's hdfs-site.xml ,or i need to set the option on All
node of my HDFS cluster? thanks a lot!
Re: question about speed up the hdfs balancer
Posted by Benoy Antony <ba...@gmail.com>.
You can invoke the following command to change the BalanacerBandwidth at
runtime.
hadoop dfsadmin -setBalanacerBandwidth newbandwidth
This command is submitted to the namenode which in turn instructs all the
datanodes. No need to restart any servers for this change,
For more details, see https://issues.apache.org/jira/browse/HDFS-2202
thanks and regards,
Benoy Antony
On Mon, Nov 17, 2014 at 2:10 AM, Serge Blazhievsky <ha...@gmail.com>
wrote:
> If you set bandwidth on one of the nodes, it takes effect on the whole
> cluster.
>
>
>
> On Mon, Nov 17, 2014 at 1:56 AM, Gordon Wang <gw...@pivotal.io> wrote:
>
>> You have to push the conf to all the datanodes and restart the datanodes.
>>
>> On Mon, Nov 17, 2014 at 5:03 PM, ch huang <ju...@gmail.com> wrote:
>>
>>> hi,maillist:
>>> i want to balancer for my HDFS cluster ,but the default speed
>>> is 1M/s , i want to set the option dfs.balance.bandwidthPerSec to 20M/s
>>> ,surpose i issue balancer command on node A ,i do not know if i just set
>>> the option on node A's hdfs-site.xml ,or i need to set the option on All
>>> node of my HDFS cluster? thanks a lot!
>>>
>>
>>
>>
>> --
>> Regards
>> Gordon Wang
>>
>
>
Re: question about speed up the hdfs balancer
Posted by Benoy Antony <ba...@gmail.com>.
You can invoke the following command to change the BalanacerBandwidth at
runtime.
hadoop dfsadmin -setBalanacerBandwidth newbandwidth
This command is submitted to the namenode which in turn instructs all the
datanodes. No need to restart any servers for this change,
For more details, see https://issues.apache.org/jira/browse/HDFS-2202
thanks and regards,
Benoy Antony
On Mon, Nov 17, 2014 at 2:10 AM, Serge Blazhievsky <ha...@gmail.com>
wrote:
> If you set bandwidth on one of the nodes, it takes effect on the whole
> cluster.
>
>
>
> On Mon, Nov 17, 2014 at 1:56 AM, Gordon Wang <gw...@pivotal.io> wrote:
>
>> You have to push the conf to all the datanodes and restart the datanodes.
>>
>> On Mon, Nov 17, 2014 at 5:03 PM, ch huang <ju...@gmail.com> wrote:
>>
>>> hi,maillist:
>>> i want to balancer for my HDFS cluster ,but the default speed
>>> is 1M/s , i want to set the option dfs.balance.bandwidthPerSec to 20M/s
>>> ,surpose i issue balancer command on node A ,i do not know if i just set
>>> the option on node A's hdfs-site.xml ,or i need to set the option on All
>>> node of my HDFS cluster? thanks a lot!
>>>
>>
>>
>>
>> --
>> Regards
>> Gordon Wang
>>
>
>
Re: question about speed up the hdfs balancer
Posted by Benoy Antony <ba...@gmail.com>.
You can invoke the following command to change the BalanacerBandwidth at
runtime.
hadoop dfsadmin -setBalanacerBandwidth newbandwidth
This command is submitted to the namenode which in turn instructs all the
datanodes. No need to restart any servers for this change,
For more details, see https://issues.apache.org/jira/browse/HDFS-2202
thanks and regards,
Benoy Antony
On Mon, Nov 17, 2014 at 2:10 AM, Serge Blazhievsky <ha...@gmail.com>
wrote:
> If you set bandwidth on one of the nodes, it takes effect on the whole
> cluster.
>
>
>
> On Mon, Nov 17, 2014 at 1:56 AM, Gordon Wang <gw...@pivotal.io> wrote:
>
>> You have to push the conf to all the datanodes and restart the datanodes.
>>
>> On Mon, Nov 17, 2014 at 5:03 PM, ch huang <ju...@gmail.com> wrote:
>>
>>> hi,maillist:
>>> i want to balancer for my HDFS cluster ,but the default speed
>>> is 1M/s , i want to set the option dfs.balance.bandwidthPerSec to 20M/s
>>> ,surpose i issue balancer command on node A ,i do not know if i just set
>>> the option on node A's hdfs-site.xml ,or i need to set the option on All
>>> node of my HDFS cluster? thanks a lot!
>>>
>>
>>
>>
>> --
>> Regards
>> Gordon Wang
>>
>
>
Re: question about speed up the hdfs balancer
Posted by Benoy Antony <ba...@gmail.com>.
You can invoke the following command to change the BalanacerBandwidth at
runtime.
hadoop dfsadmin -setBalanacerBandwidth newbandwidth
This command is submitted to the namenode which in turn instructs all the
datanodes. No need to restart any servers for this change,
For more details, see https://issues.apache.org/jira/browse/HDFS-2202
thanks and regards,
Benoy Antony
On Mon, Nov 17, 2014 at 2:10 AM, Serge Blazhievsky <ha...@gmail.com>
wrote:
> If you set bandwidth on one of the nodes, it takes effect on the whole
> cluster.
>
>
>
> On Mon, Nov 17, 2014 at 1:56 AM, Gordon Wang <gw...@pivotal.io> wrote:
>
>> You have to push the conf to all the datanodes and restart the datanodes.
>>
>> On Mon, Nov 17, 2014 at 5:03 PM, ch huang <ju...@gmail.com> wrote:
>>
>>> hi,maillist:
>>> i want to balancer for my HDFS cluster ,but the default speed
>>> is 1M/s , i want to set the option dfs.balance.bandwidthPerSec to 20M/s
>>> ,surpose i issue balancer command on node A ,i do not know if i just set
>>> the option on node A's hdfs-site.xml ,or i need to set the option on All
>>> node of my HDFS cluster? thanks a lot!
>>>
>>
>>
>>
>> --
>> Regards
>> Gordon Wang
>>
>
>
Re: question about speed up the hdfs balancer
Posted by Serge Blazhievsky <ha...@gmail.com>.
If you set bandwidth on one of the nodes, it takes effect on the whole
cluster.
On Mon, Nov 17, 2014 at 1:56 AM, Gordon Wang <gw...@pivotal.io> wrote:
> You have to push the conf to all the datanodes and restart the datanodes.
>
> On Mon, Nov 17, 2014 at 5:03 PM, ch huang <ju...@gmail.com> wrote:
>
>> hi,maillist:
>> i want to balancer for my HDFS cluster ,but the default speed
>> is 1M/s , i want to set the option dfs.balance.bandwidthPerSec to 20M/s
>> ,surpose i issue balancer command on node A ,i do not know if i just set
>> the option on node A's hdfs-site.xml ,or i need to set the option on All
>> node of my HDFS cluster? thanks a lot!
>>
>
>
>
> --
> Regards
> Gordon Wang
>
Re: question about speed up the hdfs balancer
Posted by Serge Blazhievsky <ha...@gmail.com>.
If you set bandwidth on one of the nodes, it takes effect on the whole
cluster.
On Mon, Nov 17, 2014 at 1:56 AM, Gordon Wang <gw...@pivotal.io> wrote:
> You have to push the conf to all the datanodes and restart the datanodes.
>
> On Mon, Nov 17, 2014 at 5:03 PM, ch huang <ju...@gmail.com> wrote:
>
>> hi,maillist:
>> i want to balancer for my HDFS cluster ,but the default speed
>> is 1M/s , i want to set the option dfs.balance.bandwidthPerSec to 20M/s
>> ,surpose i issue balancer command on node A ,i do not know if i just set
>> the option on node A's hdfs-site.xml ,or i need to set the option on All
>> node of my HDFS cluster? thanks a lot!
>>
>
>
>
> --
> Regards
> Gordon Wang
>
Re: question about speed up the hdfs balancer
Posted by Serge Blazhievsky <ha...@gmail.com>.
If you set bandwidth on one of the nodes, it takes effect on the whole
cluster.
On Mon, Nov 17, 2014 at 1:56 AM, Gordon Wang <gw...@pivotal.io> wrote:
> You have to push the conf to all the datanodes and restart the datanodes.
>
> On Mon, Nov 17, 2014 at 5:03 PM, ch huang <ju...@gmail.com> wrote:
>
>> hi,maillist:
>> i want to balancer for my HDFS cluster ,but the default speed
>> is 1M/s , i want to set the option dfs.balance.bandwidthPerSec to 20M/s
>> ,surpose i issue balancer command on node A ,i do not know if i just set
>> the option on node A's hdfs-site.xml ,or i need to set the option on All
>> node of my HDFS cluster? thanks a lot!
>>
>
>
>
> --
> Regards
> Gordon Wang
>
Re: question about speed up the hdfs balancer
Posted by Serge Blazhievsky <ha...@gmail.com>.
If you set bandwidth on one of the nodes, it takes effect on the whole
cluster.
On Mon, Nov 17, 2014 at 1:56 AM, Gordon Wang <gw...@pivotal.io> wrote:
> You have to push the conf to all the datanodes and restart the datanodes.
>
> On Mon, Nov 17, 2014 at 5:03 PM, ch huang <ju...@gmail.com> wrote:
>
>> hi,maillist:
>> i want to balancer for my HDFS cluster ,but the default speed
>> is 1M/s , i want to set the option dfs.balance.bandwidthPerSec to 20M/s
>> ,surpose i issue balancer command on node A ,i do not know if i just set
>> the option on node A's hdfs-site.xml ,or i need to set the option on All
>> node of my HDFS cluster? thanks a lot!
>>
>
>
>
> --
> Regards
> Gordon Wang
>
Re: question about speed up the hdfs balancer
Posted by Gordon Wang <gw...@pivotal.io>.
You have to push the conf to all the datanodes and restart the datanodes.
On Mon, Nov 17, 2014 at 5:03 PM, ch huang <ju...@gmail.com> wrote:
> hi,maillist:
> i want to balancer for my HDFS cluster ,but the default speed is
> 1M/s , i want to set the option dfs.balance.bandwidthPerSec to 20M/s
> ,surpose i issue balancer command on node A ,i do not know if i just set
> the option on node A's hdfs-site.xml ,or i need to set the option on All
> node of my HDFS cluster? thanks a lot!
>
--
Regards
Gordon Wang
Re: question about speed up the hdfs balancer
Posted by Gordon Wang <gw...@pivotal.io>.
You have to push the conf to all the datanodes and restart the datanodes.
On Mon, Nov 17, 2014 at 5:03 PM, ch huang <ju...@gmail.com> wrote:
> hi,maillist:
> i want to balancer for my HDFS cluster ,but the default speed is
> 1M/s , i want to set the option dfs.balance.bandwidthPerSec to 20M/s
> ,surpose i issue balancer command on node A ,i do not know if i just set
> the option on node A's hdfs-site.xml ,or i need to set the option on All
> node of my HDFS cluster? thanks a lot!
>
--
Regards
Gordon Wang
Re: question about speed up the hdfs balancer
Posted by Gordon Wang <gw...@pivotal.io>.
You have to push the conf to all the datanodes and restart the datanodes.
On Mon, Nov 17, 2014 at 5:03 PM, ch huang <ju...@gmail.com> wrote:
> hi,maillist:
> i want to balancer for my HDFS cluster ,but the default speed is
> 1M/s , i want to set the option dfs.balance.bandwidthPerSec to 20M/s
> ,surpose i issue balancer command on node A ,i do not know if i just set
> the option on node A's hdfs-site.xml ,or i need to set the option on All
> node of my HDFS cluster? thanks a lot!
>
--
Regards
Gordon Wang
Re: question about speed up the hdfs balancer
Posted by Gordon Wang <gw...@pivotal.io>.
You have to push the conf to all the datanodes and restart the datanodes.
On Mon, Nov 17, 2014 at 5:03 PM, ch huang <ju...@gmail.com> wrote:
> hi,maillist:
> i want to balancer for my HDFS cluster ,but the default speed is
> 1M/s , i want to set the option dfs.balance.bandwidthPerSec to 20M/s
> ,surpose i issue balancer command on node A ,i do not know if i just set
> the option on node A's hdfs-site.xml ,or i need to set the option on All
> node of my HDFS cluster? thanks a lot!
>
--
Regards
Gordon Wang