How to rebuild NameNode from DataNode.

[Saburo Fujioka <fu...@do-it.co.jp>]

Hello,

I do a tentative plan of operative trouble countermeasures
of a system currently now.

If when NameNode has been lost, but are investigating the
means to rebuild the remaining NameNode from DataNode,
I don't know at the moment.

Were consistent with those of the DataNode is the namespaceID
of "dfs/name/current/ VERSION" as confirmation,
"fsimage" are not rebuilt, the list did not display anything
in the "hadoop dfs-ls".

The risk of loss for NameNode  because that is protected by
"Corosync + Pacemaker + DRBD" is low.
Because of the rare case, it is necessary to clarify the means
to reconfigure the NameNode from DataNode.

Do you know how to?


I am using hadoop 1.0.1.

Thank you very much,

Re: How to rebuild NameNode from DataNode.

[Harsh J <ha...@cloudera.com>]

This isn't possible to do. DN holds no metadata about what file the
block belongs to.

Redundant copies (2-3, at least one off-machine) of dfs.name.dir,
aside from has not proven bad in recovery situations in my experience
yet. You should be just fine with adequate redundancy and suitably
periodic checkpoints.

On Thu, Apr 19, 2012 at 10:18 AM, Saburo Fujioka <fu...@do-it.co.jp> wrote:
> Hello,
>
> I do a tentative plan of operative trouble countermeasures
> of a system currently now.
>
> If when NameNode has been lost, but are investigating the
> means to rebuild the remaining NameNode from DataNode,
> I don't know at the moment.
>
> Were consistent with those of the DataNode is the namespaceID
> of "dfs/name/current/ VERSION" as confirmation,
> "fsimage" are not rebuilt, the list did not display anything
> in the "hadoop dfs-ls".
>
> The risk of loss for NameNode  because that is protected by
> "Corosync + Pacemaker + DRBD" is low.
> Because of the rare case, it is necessary to clarify the means
> to reconfigure the NameNode from DataNode.
>
> Do you know how to?
>
>
> I am using hadoop 1.0.1.
>
> Thank you very much,
>



-- 
Harsh J

Re: How to rebuild NameNode from DataNode.

[Michel Segel <mi...@hotmail.com>]

Switch to MapR M5?
:-)

Just kidding. 
Simple way of solving this pre CDH4...
NFS mount a directory from your SN and add it to your list of checkpoint directories.
You may lose some data, but you should be able rebuild. 

There is more to this, but its the basic idea on how to get a copy of your meta data.


Sent from a remote device. Please excuse any typos...

Mike Segel

On Apr 18, 2012, at 11:48 PM, Saburo Fujioka <fu...@do-it.co.jp> wrote:

> Hello,
> 
> I do a tentative plan of operative trouble countermeasures
> of a system currently now.
> 
> If when NameNode has been lost, but are investigating the
> means to rebuild the remaining NameNode from DataNode,
> I don't know at the moment.
> 
> Were consistent with those of the DataNode is the namespaceID
> of "dfs/name/current/ VERSION" as confirmation,
> "fsimage" are not rebuilt, the list did not display anything
> in the "hadoop dfs-ls".
> 
> The risk of loss for NameNode  because that is protected by
> "Corosync + Pacemaker + DRBD" is low.
> Because of the rare case, it is necessary to clarify the means
> to reconfigure the NameNode from DataNode.
> 
> Do you know how to?
> 
> 
> I am using hadoop 1.0.1.
> 
> Thank you very much,
> 
>