You are viewing a plain text version of this content. The canonical link for it is here.
Posted to mapreduce-user@hadoop.apache.org by Manoj Samel <ma...@gmail.com> on 2014/10/17 02:15:28 UTC
hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
from 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
write the new 2.4 protobuf format.
After using 2.4, if there is a need to downgrade back to 2.3, how would
that work ?
Thanks,
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Azuryy Yu <az...@gmail.com>.
just stop your cluster, then start your HDFS with '-rollback'. but it's
only if you don't finalize HDFS upgrade using command line.
On Fri, Oct 17, 2014 at 8:15 AM, Manoj Samel <ma...@gmail.com>
wrote:
> Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
> from 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
> write the new 2.4 protobuf format.
>
> After using 2.4, if there is a need to downgrade back to 2.3, how would
> that work ?
>
> Thanks,
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Azuryy Yu <az...@gmail.com>.
after upgrade, there are current an previous subdirs in the namenode
metadata directory. so during rollback, just delete current directory and
rename previous to current again.
On Tue, Oct 21, 2014 at 9:27 AM, Manoj Samel <ma...@gmail.com>
wrote:
> Is the pre-upgrade metadata also kept updated with any changes one in
> 2.4.0 ? Or is it just the 2.3.0 snapshot preserved?
>
> Thanks,
>
> On Sat, Oct 18, 2014 at 1:46 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Until you issue a finalize command, the pre-upgrade metadata is kept
>> aside for rolling back. When you issue the rollback command, it
>> replaces the past metadata files back.
>>
>> On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com>
>> wrote:
>> > Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
>> from
>> > 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
>> write
>> > the new 2.4 protobuf format.
>> >
>> > After using 2.4, if there is a need to downgrade back to 2.3, how would
>> that
>> > work ?
>> >
>> > Thanks,
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Susheel Kumar Gadalay <sk...@gmail.com>.
New files added in 2.4.0 will not be there in the metadata of 2.3.0.
You need to add once again.
On 10/21/14, Manoj Samel <ma...@gmail.com> wrote:
> Is the pre-upgrade metadata also kept updated with any changes one in 2.4.0
> ? Or is it just the 2.3.0 snapshot preserved?
>
> Thanks,
>
> On Sat, Oct 18, 2014 at 1:46 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Until you issue a finalize command, the pre-upgrade metadata is kept
>> aside for rolling back. When you issue the rollback command, it
>> replaces the past metadata files back.
>>
>> On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com>
>> wrote:
>> > Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
>> from
>> > 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
>> write
>> > the new 2.4 protobuf format.
>> >
>> > After using 2.4, if there is a need to downgrade back to 2.3, how would
>> that
>> > work ?
>> >
>> > Thanks,
>>
>>
>>
>> --
>> Harsh J
>>
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Susheel Kumar Gadalay <sk...@gmail.com>.
New files added in 2.4.0 will not be there in the metadata of 2.3.0.
You need to add once again.
On 10/21/14, Manoj Samel <ma...@gmail.com> wrote:
> Is the pre-upgrade metadata also kept updated with any changes one in 2.4.0
> ? Or is it just the 2.3.0 snapshot preserved?
>
> Thanks,
>
> On Sat, Oct 18, 2014 at 1:46 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Until you issue a finalize command, the pre-upgrade metadata is kept
>> aside for rolling back. When you issue the rollback command, it
>> replaces the past metadata files back.
>>
>> On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com>
>> wrote:
>> > Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
>> from
>> > 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
>> write
>> > the new 2.4 protobuf format.
>> >
>> > After using 2.4, if there is a need to downgrade back to 2.3, how would
>> that
>> > work ?
>> >
>> > Thanks,
>>
>>
>>
>> --
>> Harsh J
>>
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Azuryy Yu <az...@gmail.com>.
after upgrade, there are current an previous subdirs in the namenode
metadata directory. so during rollback, just delete current directory and
rename previous to current again.
On Tue, Oct 21, 2014 at 9:27 AM, Manoj Samel <ma...@gmail.com>
wrote:
> Is the pre-upgrade metadata also kept updated with any changes one in
> 2.4.0 ? Or is it just the 2.3.0 snapshot preserved?
>
> Thanks,
>
> On Sat, Oct 18, 2014 at 1:46 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Until you issue a finalize command, the pre-upgrade metadata is kept
>> aside for rolling back. When you issue the rollback command, it
>> replaces the past metadata files back.
>>
>> On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com>
>> wrote:
>> > Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
>> from
>> > 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
>> write
>> > the new 2.4 protobuf format.
>> >
>> > After using 2.4, if there is a need to downgrade back to 2.3, how would
>> that
>> > work ?
>> >
>> > Thanks,
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Susheel Kumar Gadalay <sk...@gmail.com>.
New files added in 2.4.0 will not be there in the metadata of 2.3.0.
You need to add once again.
On 10/21/14, Manoj Samel <ma...@gmail.com> wrote:
> Is the pre-upgrade metadata also kept updated with any changes one in 2.4.0
> ? Or is it just the 2.3.0 snapshot preserved?
>
> Thanks,
>
> On Sat, Oct 18, 2014 at 1:46 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Until you issue a finalize command, the pre-upgrade metadata is kept
>> aside for rolling back. When you issue the rollback command, it
>> replaces the past metadata files back.
>>
>> On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com>
>> wrote:
>> > Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
>> from
>> > 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
>> write
>> > the new 2.4 protobuf format.
>> >
>> > After using 2.4, if there is a need to downgrade back to 2.3, how would
>> that
>> > work ?
>> >
>> > Thanks,
>>
>>
>>
>> --
>> Harsh J
>>
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Azuryy Yu <az...@gmail.com>.
after upgrade, there are current an previous subdirs in the namenode
metadata directory. so during rollback, just delete current directory and
rename previous to current again.
On Tue, Oct 21, 2014 at 9:27 AM, Manoj Samel <ma...@gmail.com>
wrote:
> Is the pre-upgrade metadata also kept updated with any changes one in
> 2.4.0 ? Or is it just the 2.3.0 snapshot preserved?
>
> Thanks,
>
> On Sat, Oct 18, 2014 at 1:46 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Until you issue a finalize command, the pre-upgrade metadata is kept
>> aside for rolling back. When you issue the rollback command, it
>> replaces the past metadata files back.
>>
>> On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com>
>> wrote:
>> > Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
>> from
>> > 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
>> write
>> > the new 2.4 protobuf format.
>> >
>> > After using 2.4, if there is a need to downgrade back to 2.3, how would
>> that
>> > work ?
>> >
>> > Thanks,
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Azuryy Yu <az...@gmail.com>.
after upgrade, there are current an previous subdirs in the namenode
metadata directory. so during rollback, just delete current directory and
rename previous to current again.
On Tue, Oct 21, 2014 at 9:27 AM, Manoj Samel <ma...@gmail.com>
wrote:
> Is the pre-upgrade metadata also kept updated with any changes one in
> 2.4.0 ? Or is it just the 2.3.0 snapshot preserved?
>
> Thanks,
>
> On Sat, Oct 18, 2014 at 1:46 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Until you issue a finalize command, the pre-upgrade metadata is kept
>> aside for rolling back. When you issue the rollback command, it
>> replaces the past metadata files back.
>>
>> On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com>
>> wrote:
>> > Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
>> from
>> > 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
>> write
>> > the new 2.4 protobuf format.
>> >
>> > After using 2.4, if there is a need to downgrade back to 2.3, how would
>> that
>> > work ?
>> >
>> > Thanks,
>>
>>
>>
>> --
>> Harsh J
>>
>
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Susheel Kumar Gadalay <sk...@gmail.com>.
New files added in 2.4.0 will not be there in the metadata of 2.3.0.
You need to add once again.
On 10/21/14, Manoj Samel <ma...@gmail.com> wrote:
> Is the pre-upgrade metadata also kept updated with any changes one in 2.4.0
> ? Or is it just the 2.3.0 snapshot preserved?
>
> Thanks,
>
> On Sat, Oct 18, 2014 at 1:46 PM, Harsh J <ha...@cloudera.com> wrote:
>
>> Until you issue a finalize command, the pre-upgrade metadata is kept
>> aside for rolling back. When you issue the rollback command, it
>> replaces the past metadata files back.
>>
>> On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com>
>> wrote:
>> > Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
>> from
>> > 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
>> write
>> > the new 2.4 protobuf format.
>> >
>> > After using 2.4, if there is a need to downgrade back to 2.3, how would
>> that
>> > work ?
>> >
>> > Thanks,
>>
>>
>>
>> --
>> Harsh J
>>
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Manoj Samel <ma...@gmail.com>.
Is the pre-upgrade metadata also kept updated with any changes one in 2.4.0
? Or is it just the 2.3.0 snapshot preserved?
Thanks,
On Sat, Oct 18, 2014 at 1:46 PM, Harsh J <ha...@cloudera.com> wrote:
> Until you issue a finalize command, the pre-upgrade metadata is kept
> aside for rolling back. When you issue the rollback command, it
> replaces the past metadata files back.
>
> On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com>
> wrote:
> > Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
> from
> > 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
> write
> > the new 2.4 protobuf format.
> >
> > After using 2.4, if there is a need to downgrade back to 2.3, how would
> that
> > work ?
> >
> > Thanks,
>
>
>
> --
> Harsh J
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Manoj Samel <ma...@gmail.com>.
Is the pre-upgrade metadata also kept updated with any changes one in 2.4.0
? Or is it just the 2.3.0 snapshot preserved?
Thanks,
On Sat, Oct 18, 2014 at 1:46 PM, Harsh J <ha...@cloudera.com> wrote:
> Until you issue a finalize command, the pre-upgrade metadata is kept
> aside for rolling back. When you issue the rollback command, it
> replaces the past metadata files back.
>
> On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com>
> wrote:
> > Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
> from
> > 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
> write
> > the new 2.4 protobuf format.
> >
> > After using 2.4, if there is a need to downgrade back to 2.3, how would
> that
> > work ?
> >
> > Thanks,
>
>
>
> --
> Harsh J
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Manoj Samel <ma...@gmail.com>.
Is the pre-upgrade metadata also kept updated with any changes one in 2.4.0
? Or is it just the 2.3.0 snapshot preserved?
Thanks,
On Sat, Oct 18, 2014 at 1:46 PM, Harsh J <ha...@cloudera.com> wrote:
> Until you issue a finalize command, the pre-upgrade metadata is kept
> aside for rolling back. When you issue the rollback command, it
> replaces the past metadata files back.
>
> On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com>
> wrote:
> > Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
> from
> > 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
> write
> > the new 2.4 protobuf format.
> >
> > After using 2.4, if there is a need to downgrade back to 2.3, how would
> that
> > work ?
> >
> > Thanks,
>
>
>
> --
> Harsh J
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Manoj Samel <ma...@gmail.com>.
Is the pre-upgrade metadata also kept updated with any changes one in 2.4.0
? Or is it just the 2.3.0 snapshot preserved?
Thanks,
On Sat, Oct 18, 2014 at 1:46 PM, Harsh J <ha...@cloudera.com> wrote:
> Until you issue a finalize command, the pre-upgrade metadata is kept
> aside for rolling back. When you issue the rollback command, it
> replaces the past metadata files back.
>
> On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com>
> wrote:
> > Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
> from
> > 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
> write
> > the new 2.4 protobuf format.
> >
> > After using 2.4, if there is a need to downgrade back to 2.3, how would
> that
> > work ?
> >
> > Thanks,
>
>
>
> --
> Harsh J
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Harsh J <ha...@cloudera.com>.
Until you issue a finalize command, the pre-upgrade metadata is kept
aside for rolling back. When you issue the rollback command, it
replaces the past metadata files back.
On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com> wrote:
> Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade from
> 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and write
> the new 2.4 protobuf format.
>
> After using 2.4, if there is a need to downgrade back to 2.3, how would that
> work ?
>
> Thanks,
--
Harsh J
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Harsh J <ha...@cloudera.com>.
Until you issue a finalize command, the pre-upgrade metadata is kept
aside for rolling back. When you issue the rollback command, it
replaces the past metadata files back.
On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com> wrote:
> Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade from
> 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and write
> the new 2.4 protobuf format.
>
> After using 2.4, if there is a need to downgrade back to 2.3, how would that
> work ?
>
> Thanks,
--
Harsh J
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Azuryy Yu <az...@gmail.com>.
just stop your cluster, then start your HDFS with '-rollback'. but it's
only if you don't finalize HDFS upgrade using command line.
On Fri, Oct 17, 2014 at 8:15 AM, Manoj Samel <ma...@gmail.com>
wrote:
> Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
> from 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
> write the new 2.4 protobuf format.
>
> After using 2.4, if there is a need to downgrade back to 2.3, how would
> that work ?
>
> Thanks,
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Harsh J <ha...@cloudera.com>.
Until you issue a finalize command, the pre-upgrade metadata is kept
aside for rolling back. When you issue the rollback command, it
replaces the past metadata files back.
On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com> wrote:
> Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade from
> 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and write
> the new 2.4 protobuf format.
>
> After using 2.4, if there is a need to downgrade back to 2.3, how would that
> work ?
>
> Thanks,
--
Harsh J
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Azuryy Yu <az...@gmail.com>.
just stop your cluster, then start your HDFS with '-rollback'. but it's
only if you don't finalize HDFS upgrade using command line.
On Fri, Oct 17, 2014 at 8:15 AM, Manoj Samel <ma...@gmail.com>
wrote:
> Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
> from 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
> write the new 2.4 protobuf format.
>
> After using 2.4, if there is a need to downgrade back to 2.3, how would
> that work ?
>
> Thanks,
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Azuryy Yu <az...@gmail.com>.
just stop your cluster, then start your HDFS with '-rollback'. but it's
only if you don't finalize HDFS upgrade using command line.
On Fri, Oct 17, 2014 at 8:15 AM, Manoj Samel <ma...@gmail.com>
wrote:
> Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade
> from 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and
> write the new 2.4 protobuf format.
>
> After using 2.4, if there is a need to downgrade back to 2.3, how would
> that work ?
>
> Thanks,
>
Re: hadoop 2.4 using Protobuf - How does downgrade back to 2.3 works ?
Posted by Harsh J <ha...@cloudera.com>.
Until you issue a finalize command, the pre-upgrade metadata is kept
aside for rolling back. When you issue the rollback command, it
replaces the past metadata files back.
On Fri, Oct 17, 2014 at 5:45 AM, Manoj Samel <ma...@gmail.com> wrote:
> Hadoop 2.4.0 mentions that FSImage is stored using protobuf. So upgrade from
> 2.3.0 to 2.4 would work since 2.4 can read old (2.3) binary format and write
> the new 2.4 protobuf format.
>
> After using 2.4, if there is a need to downgrade back to 2.3, how would that
> work ?
>
> Thanks,
--
Harsh J