You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by Jeff Zhang <zj...@gmail.com> on 2010/05/13 11:01:22 UTC
How to recover hadoop cluster using SecondaryNameNode ?
Hi all,
I wonder is it enough for recovering hadoop cluster by just coping the
meta data from SecondaryNameNode to the new master node ? Do I any
need do any other stuffs ?
Thanks for any help.
--
Best Regards
Jeff Zhang
Re: How to recover hadoop cluster using SecondaryNameNode ?
Posted by Allen Wittenauer <aw...@linkedin.com>.
This is a good time to remind folks that the namenode can write to multiple directories, including one over a network filesystem or SAN so that you always have a fresh copy. :)
On May 13, 2010, at 8:05 AM, Eric Sammer wrote:
> You can use the copy of fsimage and the editlog from the SNN to
> recover. Remember that it will be (roughly) an hour old. The process
> for recovery is to copy the fsimage and editlog to a new machine,
> place them in the dfs.name.dir/current directory, and start all the
> daemons. It's worth practicing this type of procedure before trying it
> on a production cluster. More importantly, it's worth practicing this
> *before* you need it on a production cluster.
>
> On Thu, May 13, 2010 at 5:01 AM, Jeff Zhang <zj...@gmail.com> wrote:
>> Hi all,
>>
>> I wonder is it enough for recovering hadoop cluster by just coping the
>> meta data from SecondaryNameNode to the new master node ? Do I any
>> need do any other stuffs ?
>> Thanks for any help.
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>
>
>
> --
> Eric Sammer
> phone: +1-917-287-2675
> twitter: esammer
> data: www.cloudera.com
Re: How to recover hadoop cluster using SecondaryNameNode ?
Posted by Eric Sammer <es...@cloudera.com>.
You can use the copy of fsimage and the editlog from the SNN to
recover. Remember that it will be (roughly) an hour old. The process
for recovery is to copy the fsimage and editlog to a new machine,
place them in the dfs.name.dir/current directory, and start all the
daemons. It's worth practicing this type of procedure before trying it
on a production cluster. More importantly, it's worth practicing this
*before* you need it on a production cluster.
On Thu, May 13, 2010 at 5:01 AM, Jeff Zhang <zj...@gmail.com> wrote:
> Hi all,
>
> I wonder is it enough for recovering hadoop cluster by just coping the
> meta data from SecondaryNameNode to the new master node ? Do I any
> need do any other stuffs ?
> Thanks for any help.
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>
--
Eric Sammer
phone: +1-917-287-2675
twitter: esammer
data: www.cloudera.com
Re: How to recover hadoop cluster using SecondaryNameNode ?
Posted by Jeff Zhang <zj...@gmail.com>.
Allen,
Do you mean HDFS-976 <https://issues.apache.org/jira/browse/HDFS-976> and
HDFS-966 <https://issues.apache.org/jira/browse/HDFS-966> ?
On Fri, May 14, 2010 at 1:32 AM, Allen Wittenauer <aw...@linkedin.com>
wrote:
>
> This code hasn't been committed to any branch yet and doesn't appear to
have undergone any review outside of Facebook. :(
>
> On May 13, 2010, at 6:18 AM, Ted Yu wrote:
>
>> I suggest you take a look at AvatarNode which runs standby avatar that
>> can be activated if primary avatar fails.
>>
>>
>> On Thursday, May 13, 2010, Jeff Zhang <zj...@gmail.com> wrote:
>>> Hi all,
>>>
>>> I wonder is it enough for recovering hadoop cluster by just coping the
>>> meta data from SecondaryNameNode to the new master node ? Do I any
>>> need do any other stuffs ?
>>> Thanks for any help.
>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>
>
--
Best Regards
Jeff Zhang
Re: How to recover hadoop cluster using SecondaryNameNode ?
Posted by Edward Capriolo <ed...@gmail.com>.
On Thu, May 13, 2010 at 1:32 PM, Allen Wittenauer
<aw...@linkedin.com>wrote:
>
> This code hasn't been committed to any branch yet and doesn't appear to
> have undergone any review outside of Facebook. :(
>
> On May 13, 2010, at 6:18 AM, Ted Yu wrote:
>
> > I suggest you take a look at AvatarNode which runs standby avatar that
> > can be activated if primary avatar fails.
> >
> >
> > On Thursday, May 13, 2010, Jeff Zhang <zj...@gmail.com> wrote:
> >> Hi all,
> >>
> >> I wonder is it enough for recovering hadoop cluster by just coping the
> >> meta data from SecondaryNameNode to the new master node ? Do I any
> >> need do any other stuffs ?
> >> Thanks for any help.
> >>
> >>
> >>
> >> --
> >> Best Regards
> >>
> >> Jeff Zhang
> >>
>
> Wow that AvatarNode was really flying below the Radar! First I heard of it.
Awesome!
Re: How to recover hadoop cluster using SecondaryNameNode ?
Posted by Allen Wittenauer <aw...@linkedin.com>.
This code hasn't been committed to any branch yet and doesn't appear to have undergone any review outside of Facebook. :(
On May 13, 2010, at 6:18 AM, Ted Yu wrote:
> I suggest you take a look at AvatarNode which runs standby avatar that
> can be activated if primary avatar fails.
>
>
> On Thursday, May 13, 2010, Jeff Zhang <zj...@gmail.com> wrote:
>> Hi all,
>>
>> I wonder is it enough for recovering hadoop cluster by just coping the
>> meta data from SecondaryNameNode to the new master node ? Do I any
>> need do any other stuffs ?
>> Thanks for any help.
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
Re: How to recover hadoop cluster using SecondaryNameNode ?
Posted by Ted Yu <yu...@gmail.com>.
I suggest you take a look at AvatarNode which runs standby avatar that
can be activated if primary avatar fails.
On Thursday, May 13, 2010, Jeff Zhang <zj...@gmail.com> wrote:
> Hi all,
>
> I wonder is it enough for recovering hadoop cluster by just coping the
> meta data from SecondaryNameNode to the new master node ? Do I any
> need do any other stuffs ?
> Thanks for any help.
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>