Moving master and namenode...

[Jean-Marc Spaggiari <je...@spaggiari.org>]

Hi,

I have added one new server in my cluster and I want to move the
master and the namenode into this new server, and install the previous
master as a datanode/RS.

In the existing namenode, I will need to backup the NameNode storage
directories and restore them on the new namenode, and then reconfigure
all the servers to use this new namenode. Is that the right way to
proceed? Any risks for the data?

Regarding the master, I simply need to configure all the region
servers to point to this new master? Nothing I need to transfert from
the existing master to the new one?

I just want to make sure I don't miss something and lose everything.

Thanks,

JM

Re: Moving master and namenode...

[varun kumar <va...@gmail.com>]

Hi Jean,

Steps need to followed during migration of namenode.

1.Make New Server with same hostname.
2.Install hadoop
3.copy the metadata from old server and paste it in new server.
4.Make sure all the datanodes are down.
5.Stop old namenode
6.Start New namenode with old metadata.
7.if it come up properly,start all datanodes.

On Sat, Feb 9, 2013 at 9:32 PM, Jean-Marc Spaggiari <jean-marc@spaggiari.org
> wrote:

> Hi,
>
> I have added one new server in my cluster and I want to move the
> master and the namenode into this new server, and install the previous
> master as a datanode/RS.
>
> In the existing namenode, I will need to backup the NameNode storage
> directories and restore them on the new namenode, and then reconfigure
> all the servers to use this new namenode. Is that the right way to
> proceed? Any risks for the data?
>
> Regarding the master, I simply need to configure all the region
> servers to point to this new master? Nothing I need to transfert from
> the existing master to the new one?
>
> I just want to make sure I don't miss something and lose everything.
>
> Thanks,
>
> JM
>



-- 
Regards,
Varun Kumar.P

Re: Moving master and namenode...

[Jean-Marc Spaggiari <je...@spaggiari.org>]

Thanks Harsh and Varun for your replies.

Finally, to keep is simple I kept the same hostname and ave upgraded the
server instead of moving it under another hostname...

At some point I will most probably have to move it, but for now, it's
enought.

I will keep your advices in mind for when I will have to do it.

JM

2013/2/10 Harsh J <ha...@cloudera.com>

> The master does not have any local storage in a fully distributed
> setup, so the transfer can also be as easy as starting the new master
> on the new host and failing it over (by killing the original one).
>
> The NameNode move part is the one that gets tricky. HBase may store NN
> URLs in its ZK transient storage for purposes such as distributed log
> splitting, etc.. (correct me if this has been addressed), and if the
> NN move is ending up with a change of its hostname, then before you
> start HBase, you may want to scour the /hbase znodes in ZK to remove
> the znodes with a reference to the older NN URL in its data. Most just
> prefer to erase the whole /hbase znode, but it would work either way.
> The goal being to not have the new master or restarted slaves run into
> issues when it picks up a URL it can no longer fetch for processing.
>
> On Sat, Feb 9, 2013 at 9:32 PM, Jean-Marc Spaggiari
> <je...@spaggiari.org> wrote:
> > Hi,
> >
> > I have added one new server in my cluster and I want to move the
> > master and the namenode into this new server, and install the previous
> > master as a datanode/RS.
> >
> > In the existing namenode, I will need to backup the NameNode storage
> > directories and restore them on the new namenode, and then reconfigure
> > all the servers to use this new namenode. Is that the right way to
> > proceed? Any risks for the data?
> >
> > Regarding the master, I simply need to configure all the region
> > servers to point to this new master? Nothing I need to transfert from
> > the existing master to the new one?
> >
> > I just want to make sure I don't miss something and lose everything.
> >
> > Thanks,
> >
> > JM
>
>
>
> --
> Harsh J
>

Re: Moving master and namenode...

[Harsh J <ha...@cloudera.com>]

The master does not have any local storage in a fully distributed
setup, so the transfer can also be as easy as starting the new master
on the new host and failing it over (by killing the original one).

The NameNode move part is the one that gets tricky. HBase may store NN
URLs in its ZK transient storage for purposes such as distributed log
splitting, etc.. (correct me if this has been addressed), and if the
NN move is ending up with a change of its hostname, then before you
start HBase, you may want to scour the /hbase znodes in ZK to remove
the znodes with a reference to the older NN URL in its data. Most just
prefer to erase the whole /hbase znode, but it would work either way.
The goal being to not have the new master or restarted slaves run into
issues when it picks up a URL it can no longer fetch for processing.

On Sat, Feb 9, 2013 at 9:32 PM, Jean-Marc Spaggiari
<je...@spaggiari.org> wrote:
> Hi,
>
> I have added one new server in my cluster and I want to move the
> master and the namenode into this new server, and install the previous
> master as a datanode/RS.
>
> In the existing namenode, I will need to backup the NameNode storage
> directories and restore them on the new namenode, and then reconfigure
> all the servers to use this new namenode. Is that the right way to
> proceed? Any risks for the data?
>
> Regarding the master, I simply need to configure all the region
> servers to point to this new master? Nothing I need to transfert from
> the existing master to the new one?
>
> I just want to make sure I don't miss something and lose everything.
>
> Thanks,
>
> JM



--
Harsh J