You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by asif md <as...@gmail.com> on 2009/06/24 23:50:08 UTC
Add new Datnodes : Is redistribution of previous data required?
hello everyone,
I have added 7 nodes to my 3 node cluster. I followed the following steps to
do this
1. added the node's ip to conf/slaves at master
2. ran bin/start-balance.sh at each node
As i loaded the data when the size of the cluster was three which is now
TEN. Can i do anything to redistribute the data among all the nodes?
Any ideas appreciated.
Thanks and Regards
Asif.
Re: Add new Datnodes : Is redistribution of previous data required?
Posted by asif md <as...@gmail.com>.
@Konstantin
I'll try those . Thanks.
More comments are welcome.
On Wed, Jun 24, 2009 at 7:27 PM, Konstantin Shvachko <sh...@yahoo-inc.com>wrote:
> These links should help you to rebalance the nodes:
>
> http://developer.yahoo.com/hadoop/tutorial/module2.html#rebalancing
> http://hadoop.apache.org/core/docs/current/hdfs_user_guide.html#Rebalancer
> http://hadoop.apache.org/core/docs/current/commands_manual.html#balancer
>
> http://issues.apache.org/jira/secure/attachment/12368261/RebalanceDesign6.pdf
>
> --Konstantin
>
>
> asif md wrote:
>
>> @Alex
>>
>> Thanks.
>>
>> http://wiki.apache.org/hadoop/FAQ#6
>>
>> has anyone any experience with this?
>>
>> Please suggest.
>>
>> On Wed, Jun 24, 2009 at 5:44 PM, Alex Loddengaard <al...@cloudera.com>
>> wrote:
>>
>> Hi,
>>>
>>> Running the rebalancer script (by the way, you only need to run it once)
>>> redistributes all of your data for you. That is, after you've run the
>>> rebalancer, your data should be stored evenly among your 10 nodes.
>>>
>>> Alex
>>>
>>> On Wed, Jun 24, 2009 at 2:50 PM, asif md <as...@gmail.com> wrote:
>>>
>>> hello everyone,
>>>>
>>>> I have added 7 nodes to my 3 node cluster. I followed the following
>>>> steps
>>>> to
>>>> do this
>>>>
>>>> 1. added the node's ip to conf/slaves at master
>>>> 2. ran bin/start-balance.sh at each node
>>>>
>>>> As i loaded the data when the size of the cluster was three which is now
>>>> TEN. Can i do anything to redistribute the data among all the nodes?
>>>>
>>>> Any ideas appreciated.
>>>>
>>>> Thanks and Regards
>>>>
>>>> Asif.
>>>>
>>>>
>>
Re: Add new Datnodes : Is redistribution of previous data required?
Posted by Konstantin Shvachko <sh...@yahoo-inc.com>.
These links should help you to rebalance the nodes:
http://developer.yahoo.com/hadoop/tutorial/module2.html#rebalancing
http://hadoop.apache.org/core/docs/current/hdfs_user_guide.html#Rebalancer
http://hadoop.apache.org/core/docs/current/commands_manual.html#balancer
http://issues.apache.org/jira/secure/attachment/12368261/RebalanceDesign6.pdf
--Konstantin
asif md wrote:
> @Alex
>
> Thanks.
>
> http://wiki.apache.org/hadoop/FAQ#6
>
> has anyone any experience with this?
>
> Please suggest.
>
> On Wed, Jun 24, 2009 at 5:44 PM, Alex Loddengaard <al...@cloudera.com> wrote:
>
>> Hi,
>>
>> Running the rebalancer script (by the way, you only need to run it once)
>> redistributes all of your data for you. That is, after you've run the
>> rebalancer, your data should be stored evenly among your 10 nodes.
>>
>> Alex
>>
>> On Wed, Jun 24, 2009 at 2:50 PM, asif md <as...@gmail.com> wrote:
>>
>>> hello everyone,
>>>
>>> I have added 7 nodes to my 3 node cluster. I followed the following steps
>>> to
>>> do this
>>>
>>> 1. added the node's ip to conf/slaves at master
>>> 2. ran bin/start-balance.sh at each node
>>>
>>> As i loaded the data when the size of the cluster was three which is now
>>> TEN. Can i do anything to redistribute the data among all the nodes?
>>>
>>> Any ideas appreciated.
>>>
>>> Thanks and Regards
>>>
>>> Asif.
>>>
>
Re: Add new Datnodes : Is redistribution of previous data required?
Posted by asif md <as...@gmail.com>.
@Alex
Thanks.
http://wiki.apache.org/hadoop/FAQ#6
has anyone any experience with this?
Please suggest.
On Wed, Jun 24, 2009 at 5:44 PM, Alex Loddengaard <al...@cloudera.com> wrote:
> Hi,
>
> Running the rebalancer script (by the way, you only need to run it once)
> redistributes all of your data for you. That is, after you've run the
> rebalancer, your data should be stored evenly among your 10 nodes.
>
> Alex
>
> On Wed, Jun 24, 2009 at 2:50 PM, asif md <as...@gmail.com> wrote:
>
> > hello everyone,
> >
> > I have added 7 nodes to my 3 node cluster. I followed the following steps
> > to
> > do this
> >
> > 1. added the node's ip to conf/slaves at master
> > 2. ran bin/start-balance.sh at each node
> >
> > As i loaded the data when the size of the cluster was three which is now
> > TEN. Can i do anything to redistribute the data among all the nodes?
> >
> > Any ideas appreciated.
> >
> > Thanks and Regards
> >
> > Asif.
> >
>
Re: Add new Datnodes : Is redistribution of previous data required?
Posted by Alex Loddengaard <al...@cloudera.com>.
Hi,
Running the rebalancer script (by the way, you only need to run it once)
redistributes all of your data for you. That is, after you've run the
rebalancer, your data should be stored evenly among your 10 nodes.
Alex
On Wed, Jun 24, 2009 at 2:50 PM, asif md <as...@gmail.com> wrote:
> hello everyone,
>
> I have added 7 nodes to my 3 node cluster. I followed the following steps
> to
> do this
>
> 1. added the node's ip to conf/slaves at master
> 2. ran bin/start-balance.sh at each node
>
> As i loaded the data when the size of the cluster was three which is now
> TEN. Can i do anything to redistribute the data among all the nodes?
>
> Any ideas appreciated.
>
> Thanks and Regards
>
> Asif.
>