You are viewing a plain text version of this content. The canonical link for it is here.
Posted to hdfs-dev@hadoop.apache.org by Stanley Shi <ss...@gopivotal.com> on 2014/01/14 10:38:58 UTC
NameNode be formated while namenode is running
Hi,
I have encountered this case in my environment:
1. NameNode is actively running without any problem;
2. someone accidentally formatted the namenode (even if there's one
instance running);
3. The running namenode continues to write edit log starting from some big
txid;
This caused the namenode now in an in-consistent state:
It has edit log and fsimage but if you restart the namenode, it will say
the namenode cannot be started because it expect some txid 1 ;
I think this is a bug. At least when formating the namenode, it should
check if there's an namenode instance already running.
Regards,
*Stanley Shi,*
Re: NameNode be formated while namenode is running
Posted by Stanley Shi <ss...@gopivotal.com>.
Yes, this is it! Thanks Uma!
Regards,
*Stanley Shi,*
On Wed, Jan 15, 2014 at 11:47 AM, Uma Maheswara Rao G
<ha...@gmail.com>wrote:
> Seems like you are hitting https://issues.apache.org/jira/browse/HDFS-1690?
>
>
> On Wed, Jan 15, 2014 at 6:58 AM, Stanley Shi <ss...@gopivotal.com> wrote:
>
> > Thanks Konstantin!
> > I am running on vsphere VM with CentOS64 installed, Local file system is
> > ext3;
> >
> > Regards,
> > *Stanley Shi,*
> >
> >
> >
> > On Wed, Jan 15, 2014 at 8:27 AM, Konstantin Shvachko
> > <sh...@gmail.com>wrote:
> >
> > > Yes format should check "in_use.lock".
> > > What is your environment?
> > > Does it support locks on you local file system?
> > >
> > > Thanks,
> > > --Konst
> > >
> > >
> > > On Tue, Jan 14, 2014 at 1:38 AM, Stanley Shi <ss...@gopivotal.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > I have encountered this case in my environment:
> > > >
> > > > 1. NameNode is actively running without any problem;
> > > >
> > > > 2. someone accidentally formatted the namenode (even if there's one
> > > > instance running);
> > > >
> > > > 3. The running namenode continues to write edit log starting from
> some
> > > big
> > > > txid;
> > > >
> > > >
> > > > This caused the namenode now in an in-consistent state:
> > > >
> > > > It has edit log and fsimage but if you restart the namenode, it will
> > say
> > > > the namenode cannot be started because it expect some txid 1 ;
> > > >
> > > > I think this is a bug. At least when formating the namenode, it
> should
> > > > check if there's an namenode instance already running.
> > > >
> > > > Regards,
> > > > *Stanley Shi,*
> > > >
> > >
> >
>
Re: NameNode be formated while namenode is running
Posted by Konstantin Shvachko <sh...@gmail.com>.
Good point, Uma.
Thanks,
--Konst
On Tue, Jan 14, 2014 at 7:47 PM, Uma Maheswara Rao G
<ha...@gmail.com>wrote:
> Seems like you are hitting https://issues.apache.org/jira/browse/HDFS-1690?
>
>
> On Wed, Jan 15, 2014 at 6:58 AM, Stanley Shi <ss...@gopivotal.com> wrote:
>
> > Thanks Konstantin!
> > I am running on vsphere VM with CentOS64 installed, Local file system is
> > ext3;
> >
> > Regards,
> > *Stanley Shi,*
> >
> >
> >
> > On Wed, Jan 15, 2014 at 8:27 AM, Konstantin Shvachko
> > <sh...@gmail.com>wrote:
> >
> > > Yes format should check "in_use.lock".
> > > What is your environment?
> > > Does it support locks on you local file system?
> > >
> > > Thanks,
> > > --Konst
> > >
> > >
> > > On Tue, Jan 14, 2014 at 1:38 AM, Stanley Shi <ss...@gopivotal.com>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > I have encountered this case in my environment:
> > > >
> > > > 1. NameNode is actively running without any problem;
> > > >
> > > > 2. someone accidentally formatted the namenode (even if there's one
> > > > instance running);
> > > >
> > > > 3. The running namenode continues to write edit log starting from
> some
> > > big
> > > > txid;
> > > >
> > > >
> > > > This caused the namenode now in an in-consistent state:
> > > >
> > > > It has edit log and fsimage but if you restart the namenode, it will
> > say
> > > > the namenode cannot be started because it expect some txid 1 ;
> > > >
> > > > I think this is a bug. At least when formating the namenode, it
> should
> > > > check if there's an namenode instance already running.
> > > >
> > > > Regards,
> > > > *Stanley Shi,*
> > > >
> > >
> >
>
Re: NameNode be formated while namenode is running
Posted by Uma Maheswara Rao G <ha...@gmail.com>.
Seems like you are hitting https://issues.apache.org/jira/browse/HDFS-1690 ?
On Wed, Jan 15, 2014 at 6:58 AM, Stanley Shi <ss...@gopivotal.com> wrote:
> Thanks Konstantin!
> I am running on vsphere VM with CentOS64 installed, Local file system is
> ext3;
>
> Regards,
> *Stanley Shi,*
>
>
>
> On Wed, Jan 15, 2014 at 8:27 AM, Konstantin Shvachko
> <sh...@gmail.com>wrote:
>
> > Yes format should check "in_use.lock".
> > What is your environment?
> > Does it support locks on you local file system?
> >
> > Thanks,
> > --Konst
> >
> >
> > On Tue, Jan 14, 2014 at 1:38 AM, Stanley Shi <ss...@gopivotal.com> wrote:
> >
> > > Hi,
> > >
> > > I have encountered this case in my environment:
> > >
> > > 1. NameNode is actively running without any problem;
> > >
> > > 2. someone accidentally formatted the namenode (even if there's one
> > > instance running);
> > >
> > > 3. The running namenode continues to write edit log starting from some
> > big
> > > txid;
> > >
> > >
> > > This caused the namenode now in an in-consistent state:
> > >
> > > It has edit log and fsimage but if you restart the namenode, it will
> say
> > > the namenode cannot be started because it expect some txid 1 ;
> > >
> > > I think this is a bug. At least when formating the namenode, it should
> > > check if there's an namenode instance already running.
> > >
> > > Regards,
> > > *Stanley Shi,*
> > >
> >
>
Re: NameNode be formated while namenode is running
Posted by Konstantin Shvachko <sh...@gmail.com>.
Stanley,
Sounds like you should file a jira then. ext3 is standard.
Thanks,
--Konst
On Tue, Jan 14, 2014 at 5:28 PM, Stanley Shi <ss...@gopivotal.com> wrote:
> Thanks Konstantin!
> I am running on vsphere VM with CentOS64 installed, Local file system is
> ext3;
>
> Regards,
> *Stanley Shi,*
>
>
>
> On Wed, Jan 15, 2014 at 8:27 AM, Konstantin Shvachko
> <sh...@gmail.com>wrote:
>
> > Yes format should check "in_use.lock".
> > What is your environment?
> > Does it support locks on you local file system?
> >
> > Thanks,
> > --Konst
> >
> >
> > On Tue, Jan 14, 2014 at 1:38 AM, Stanley Shi <ss...@gopivotal.com> wrote:
> >
> > > Hi,
> > >
> > > I have encountered this case in my environment:
> > >
> > > 1. NameNode is actively running without any problem;
> > >
> > > 2. someone accidentally formatted the namenode (even if there's one
> > > instance running);
> > >
> > > 3. The running namenode continues to write edit log starting from some
> > big
> > > txid;
> > >
> > >
> > > This caused the namenode now in an in-consistent state:
> > >
> > > It has edit log and fsimage but if you restart the namenode, it will
> say
> > > the namenode cannot be started because it expect some txid 1 ;
> > >
> > > I think this is a bug. At least when formating the namenode, it should
> > > check if there's an namenode instance already running.
> > >
> > > Regards,
> > > *Stanley Shi,*
> > >
> >
>
Re: NameNode be formated while namenode is running
Posted by Stanley Shi <ss...@gopivotal.com>.
Thanks Konstantin!
I am running on vsphere VM with CentOS64 installed, Local file system is
ext3;
Regards,
*Stanley Shi,*
On Wed, Jan 15, 2014 at 8:27 AM, Konstantin Shvachko
<sh...@gmail.com>wrote:
> Yes format should check "in_use.lock".
> What is your environment?
> Does it support locks on you local file system?
>
> Thanks,
> --Konst
>
>
> On Tue, Jan 14, 2014 at 1:38 AM, Stanley Shi <ss...@gopivotal.com> wrote:
>
> > Hi,
> >
> > I have encountered this case in my environment:
> >
> > 1. NameNode is actively running without any problem;
> >
> > 2. someone accidentally formatted the namenode (even if there's one
> > instance running);
> >
> > 3. The running namenode continues to write edit log starting from some
> big
> > txid;
> >
> >
> > This caused the namenode now in an in-consistent state:
> >
> > It has edit log and fsimage but if you restart the namenode, it will say
> > the namenode cannot be started because it expect some txid 1 ;
> >
> > I think this is a bug. At least when formating the namenode, it should
> > check if there's an namenode instance already running.
> >
> > Regards,
> > *Stanley Shi,*
> >
>
Re: NameNode be formated while namenode is running
Posted by Konstantin Shvachko <sh...@gmail.com>.
Yes format should check "in_use.lock".
What is your environment?
Does it support locks on you local file system?
Thanks,
--Konst
On Tue, Jan 14, 2014 at 1:38 AM, Stanley Shi <ss...@gopivotal.com> wrote:
> Hi,
>
> I have encountered this case in my environment:
>
> 1. NameNode is actively running without any problem;
>
> 2. someone accidentally formatted the namenode (even if there's one
> instance running);
>
> 3. The running namenode continues to write edit log starting from some big
> txid;
>
>
> This caused the namenode now in an in-consistent state:
>
> It has edit log and fsimage but if you restart the namenode, it will say
> the namenode cannot be started because it expect some txid 1 ;
>
> I think this is a bug. At least when formating the namenode, it should
> check if there's an namenode instance already running.
>
> Regards,
> *Stanley Shi,*
>