You are viewing a plain text version of this content. The canonical link for it is here.
Posted to common-user@hadoop.apache.org by David Ritch <da...@gmail.com> on 2009/03/18 18:08:17 UTC
Downgrading Hdfs
There is an established procedure for upgrading from one release of Hadoop
to a newer release. Is there something similar to move back to an
lower-numered release?
Specifically, we have data in a cloud running Hadoop-19.0. Because of
stability issues, we are wondering whether we should move back to 18, but we
don't want to lose our data. Is there a downward migration path?
Thanks,
David
Re: Downgrading Hdfs
Posted by Aaron Kimball <aa...@cloudera.com>.
It should also be pointed out that the "rollback" process restores your HDFS
cluster to the state it was at when it was upgraded by restoring a
checkpoint -- it does not "downgrade" your current set of blocks. So if
you've pulled new data into HDFS since the upgrade, that will be lost
- Aaron
On Wed, Mar 18, 2009 at 11:47 AM, David Ritch <da...@gmail.com> wrote:
> Bill,
>
> Thank you. Unfortunately, that will not work for us, as our hadoop-19 has
> long been finalized.
>
> dbr
>
> On Wed, Mar 18, 2009 at 2:35 PM, Bill Au <bi...@gmail.com> wrote:
>
> > As long as the version upgrade has not been finalized, here's the
> procedure
> > I use to downgrade:
> >
> >
> > 1. make sure that the previous version upgrade has not been finalized.
> A
> > version upgrade cannot be rollback if it has been finalized.
> >
> > bin/hadoop dfsadmin -upgradeProgress status
> >
> > 2. stop map-reduce cluster.
> >
> > bin/stop-mapred.sh
> >
> > 3. stop all applications and make sure that there are no running tasks.
> > 4. stop HDFS cluster.
> >
> > bin/stop-dfs.sh
> >
> > 5. rollback version of Hadoop. Install previous version of Hadoop if
> > previous version of Hadoop has been removed from the system.
> > 6. start HDFS with the rollback option.
> >
> > bin/start-dfs.sh -rollback
> >
> > 7. monitor the rollback until it is complete.
> >
> > bin/hadoop dfsadmin -upgradeProgress status
> >
> > 8. start map-reduce cluster.
> >
> > bin/start-mapred.sh
> >
> >
> >
> > Bill
> >
> > On Wed, Mar 18, 2009 at 1:08 PM, David Ritch <da...@gmail.com>
> > wrote:
> >
> > > There is an established procedure for upgrading from one release of
> > Hadoop
> > > to a newer release. Is there something similar to move back to an
> > > lower-numered release?
> > >
> > > Specifically, we have data in a cloud running Hadoop-19.0. Because of
> > > stability issues, we are wondering whether we should move back to 18,
> but
> > > we
> > > don't want to lose our data. Is there a downward migration path?
> > >
> > > Thanks,
> > >
> > > David
> > >
> >
>
Re: Downgrading Hdfs
Posted by David Ritch <da...@gmail.com>.
Bill,
Thank you. Unfortunately, that will not work for us, as our hadoop-19 has
long been finalized.
dbr
On Wed, Mar 18, 2009 at 2:35 PM, Bill Au <bi...@gmail.com> wrote:
> As long as the version upgrade has not been finalized, here's the procedure
> I use to downgrade:
>
>
> 1. make sure that the previous version upgrade has not been finalized. A
> version upgrade cannot be rollback if it has been finalized.
>
> bin/hadoop dfsadmin -upgradeProgress status
>
> 2. stop map-reduce cluster.
>
> bin/stop-mapred.sh
>
> 3. stop all applications and make sure that there are no running tasks.
> 4. stop HDFS cluster.
>
> bin/stop-dfs.sh
>
> 5. rollback version of Hadoop. Install previous version of Hadoop if
> previous version of Hadoop has been removed from the system.
> 6. start HDFS with the rollback option.
>
> bin/start-dfs.sh -rollback
>
> 7. monitor the rollback until it is complete.
>
> bin/hadoop dfsadmin -upgradeProgress status
>
> 8. start map-reduce cluster.
>
> bin/start-mapred.sh
>
>
>
> Bill
>
> On Wed, Mar 18, 2009 at 1:08 PM, David Ritch <da...@gmail.com>
> wrote:
>
> > There is an established procedure for upgrading from one release of
> Hadoop
> > to a newer release. Is there something similar to move back to an
> > lower-numered release?
> >
> > Specifically, we have data in a cloud running Hadoop-19.0. Because of
> > stability issues, we are wondering whether we should move back to 18, but
> > we
> > don't want to lose our data. Is there a downward migration path?
> >
> > Thanks,
> >
> > David
> >
>
Re: Downgrading Hdfs
Posted by Bill Au <bi...@gmail.com>.
As long as the version upgrade has not been finalized, here's the procedure
I use to downgrade:
1. make sure that the previous version upgrade has not been finalized. A
version upgrade cannot be rollback if it has been finalized.
bin/hadoop dfsadmin -upgradeProgress status
2. stop map-reduce cluster.
bin/stop-mapred.sh
3. stop all applications and make sure that there are no running tasks.
4. stop HDFS cluster.
bin/stop-dfs.sh
5. rollback version of Hadoop. Install previous version of Hadoop if
previous version of Hadoop has been removed from the system.
6. start HDFS with the rollback option.
bin/start-dfs.sh -rollback
7. monitor the rollback until it is complete.
bin/hadoop dfsadmin -upgradeProgress status
8. start map-reduce cluster.
bin/start-mapred.sh
Bill
On Wed, Mar 18, 2009 at 1:08 PM, David Ritch <da...@gmail.com> wrote:
> There is an established procedure for upgrading from one release of Hadoop
> to a newer release. Is there something similar to move back to an
> lower-numered release?
>
> Specifically, we have data in a cloud running Hadoop-19.0. Because of
> stability issues, we are wondering whether we should move back to 18, but
> we
> don't want to lose our data. Is there a downward migration path?
>
> Thanks,
>
> David
>
Re: Downgrading Hdfs
Posted by Bill Au <bi...@gmail.com>.
I though that both the capacity and fair share scheduler are available in
0.19. Are there new features add in 0.20? Is that documented anywhere?
How do I learn more?
Bill
On Fri, Mar 20, 2009 at 7:28 AM, David B. Ritch <Da...@gmail.com>wrote:
> Looks like there's been quite a bit of work on both the Capacity
> Scheduler and the Fair Scheduler. Are there others we should look at in
> 20.0?
>
> Again, thank you for your informative responses.
>
> David
>
> Raghu Angadi wrote:
> >
> > I think 0.20.0 is preferred because of new job scheduler(s) there.
> > Stability wise, 0.20 is expected to be as good as 0.19.2.
> >
> > I don't know the schedule for 0.19.2.
> >
> > Raghu.
> >
> > David Ritch wrote:
> >> Raghu,
> >>
> >> Thank you for your prompt and informative response. Moving to
> >> anything that
> >> ends in .0 is a bit scary - what are the reasons to go with 0.20.0
> >> instead
> >> of 0.19.2? Yahoo is jumping from 0.18.x directly to 0.20.0? Why is
> >> Yahoo
> >> skipping the 0.19.x release?
> >>
> >> Is the expectation that 0.19.2 will be released at the same time as
> >> 0.20.0?
> >>
> >> Thanks,
> >>
> >> David
> >>
> >> On Wed, Mar 18, 2009 at 1:31 PM, Raghu Angadi <ra...@yahoo-inc.com>
> >> wrote:
> >>
> >>> Short is answer I am afraid is no.
> >>>
> >>> As an alternative, I recommend upgrading to latest 0.19.x or 0.20.0
> >>> (to be
> >>> released in couple of days). 0.19.2 is certainly a lot better than
> >>> 0.19.0.
> >>> Yahoo is rolling out 0.20.x if that helps your confidence.
> >>>
> >>> Raghu.
> >>>
> >>>
> >>> David Ritch wrote:
> >>>
> >>>> There is an established procedure for upgrading from one release of
> >>>> Hadoop
> >>>> to a newer release. Is there something similar to move back to an
> >>>> lower-numered release?
> >>>>
> >>>> Specifically, we have data in a cloud running Hadoop-19.0. Because of
> >>>> stability issues, we are wondering whether we should move back to
> >>>> 18, but
> >>>> we
> >>>> don't want to lose our data. Is there a downward migration path?
> >>>>
> >>>> Thanks,
> >>>>
> >>>> David
> >>>>
> >>>>
> >>
> >
> >
>
>
Re: Downgrading Hdfs
Posted by "David B. Ritch" <Da...@gmail.com>.
Looks like there's been quite a bit of work on both the Capacity
Scheduler and the Fair Scheduler. Are there others we should look at in
20.0?
Again, thank you for your informative responses.
David
Raghu Angadi wrote:
>
> I think 0.20.0 is preferred because of new job scheduler(s) there.
> Stability wise, 0.20 is expected to be as good as 0.19.2.
>
> I don't know the schedule for 0.19.2.
>
> Raghu.
>
> David Ritch wrote:
>> Raghu,
>>
>> Thank you for your prompt and informative response. Moving to
>> anything that
>> ends in .0 is a bit scary - what are the reasons to go with 0.20.0
>> instead
>> of 0.19.2? Yahoo is jumping from 0.18.x directly to 0.20.0? Why is
>> Yahoo
>> skipping the 0.19.x release?
>>
>> Is the expectation that 0.19.2 will be released at the same time as
>> 0.20.0?
>>
>> Thanks,
>>
>> David
>>
>> On Wed, Mar 18, 2009 at 1:31 PM, Raghu Angadi <ra...@yahoo-inc.com>
>> wrote:
>>
>>> Short is answer I am afraid is no.
>>>
>>> As an alternative, I recommend upgrading to latest 0.19.x or 0.20.0
>>> (to be
>>> released in couple of days). 0.19.2 is certainly a lot better than
>>> 0.19.0.
>>> Yahoo is rolling out 0.20.x if that helps your confidence.
>>>
>>> Raghu.
>>>
>>>
>>> David Ritch wrote:
>>>
>>>> There is an established procedure for upgrading from one release of
>>>> Hadoop
>>>> to a newer release. Is there something similar to move back to an
>>>> lower-numered release?
>>>>
>>>> Specifically, we have data in a cloud running Hadoop-19.0. Because of
>>>> stability issues, we are wondering whether we should move back to
>>>> 18, but
>>>> we
>>>> don't want to lose our data. Is there a downward migration path?
>>>>
>>>> Thanks,
>>>>
>>>> David
>>>>
>>>>
>>
>
>
Re: Downgrading Hdfs
Posted by Raghu Angadi <ra...@yahoo-inc.com>.
I think 0.20.0 is preferred because of new job scheduler(s) there.
Stability wise, 0.20 is expected to be as good as 0.19.2.
I don't know the schedule for 0.19.2.
Raghu.
David Ritch wrote:
> Raghu,
>
> Thank you for your prompt and informative response. Moving to anything that
> ends in .0 is a bit scary - what are the reasons to go with 0.20.0 instead
> of 0.19.2? Yahoo is jumping from 0.18.x directly to 0.20.0? Why is Yahoo
> skipping the 0.19.x release?
>
> Is the expectation that 0.19.2 will be released at the same time as 0.20.0?
>
> Thanks,
>
> David
>
> On Wed, Mar 18, 2009 at 1:31 PM, Raghu Angadi <ra...@yahoo-inc.com> wrote:
>
>> Short is answer I am afraid is no.
>>
>> As an alternative, I recommend upgrading to latest 0.19.x or 0.20.0 (to be
>> released in couple of days). 0.19.2 is certainly a lot better than 0.19.0.
>> Yahoo is rolling out 0.20.x if that helps your confidence.
>>
>> Raghu.
>>
>>
>> David Ritch wrote:
>>
>>> There is an established procedure for upgrading from one release of Hadoop
>>> to a newer release. Is there something similar to move back to an
>>> lower-numered release?
>>>
>>> Specifically, we have data in a cloud running Hadoop-19.0. Because of
>>> stability issues, we are wondering whether we should move back to 18, but
>>> we
>>> don't want to lose our data. Is there a downward migration path?
>>>
>>> Thanks,
>>>
>>> David
>>>
>>>
>
Re: Downgrading Hdfs
Posted by David Ritch <da...@gmail.com>.
Raghu,
Thank you for your prompt and informative response. Moving to anything that
ends in .0 is a bit scary - what are the reasons to go with 0.20.0 instead
of 0.19.2? Yahoo is jumping from 0.18.x directly to 0.20.0? Why is Yahoo
skipping the 0.19.x release?
Is the expectation that 0.19.2 will be released at the same time as 0.20.0?
Thanks,
David
On Wed, Mar 18, 2009 at 1:31 PM, Raghu Angadi <ra...@yahoo-inc.com> wrote:
>
> Short is answer I am afraid is no.
>
> As an alternative, I recommend upgrading to latest 0.19.x or 0.20.0 (to be
> released in couple of days). 0.19.2 is certainly a lot better than 0.19.0.
> Yahoo is rolling out 0.20.x if that helps your confidence.
>
> Raghu.
>
>
> David Ritch wrote:
>
>> There is an established procedure for upgrading from one release of Hadoop
>> to a newer release. Is there something similar to move back to an
>> lower-numered release?
>>
>> Specifically, we have data in a cloud running Hadoop-19.0. Because of
>> stability issues, we are wondering whether we should move back to 18, but
>> we
>> don't want to lose our data. Is there a downward migration path?
>>
>> Thanks,
>>
>> David
>>
>>
>
Re: Downgrading Hdfs
Posted by Raghu Angadi <ra...@yahoo-inc.com>.
Short is answer I am afraid is no.
As an alternative, I recommend upgrading to latest 0.19.x or 0.20.0 (to
be released in couple of days). 0.19.2 is certainly a lot better than
0.19.0. Yahoo is rolling out 0.20.x if that helps your confidence.
Raghu.
David Ritch wrote:
> There is an established procedure for upgrading from one release of Hadoop
> to a newer release. Is there something similar to move back to an
> lower-numered release?
>
> Specifically, we have data in a cloud running Hadoop-19.0. Because of
> stability issues, we are wondering whether we should move back to 18, but we
> don't want to lose our data. Is there a downward migration path?
>
> Thanks,
>
> David
>