You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@aurora.apache.org by Mauricio Garavaglia <ma...@gmail.com> on 2015/09/21 04:40:19 UTC
Upgrading from 0.7
Hi guys,
I'm about to upgrade a cluster from 0.7 to 0.9 and was wondering what are
the things to consider to make it as seamless as possible?
For example, update the executors first and then the schedulers; just
update the schedulers one at a time, etc.
Thanks!
Mauricio
Re: Upgrading from 0.7
Posted by Mauricio Garavaglia <ma...@gmail.com>.
Yes, existing tasks kept running with the old executors.
On Sep 24, 2015 2:10 AM, "Raghuveer Pallikonda" <pa...@yahoo.com.invalid>
wrote:
> Hi Mauricio, Thank you for the update regarding the rollout, When you say
> the tasks continued running uninterrupted through the executor procedure,
> does it mean the existing tasks are still running with the old version of
> the executors? As that process did not exit during the upgrade.
> Would appreciate your clarification on the above.
> ThanksRaghu
>
>
> On Wednesday, September 23, 2015 9:19 PM, Mauricio Garavaglia <
> mauriciogaravaglia@gmail.com> wrote:
>
>
> Hi,
> I updated the executors and then rolled the schedulers (have 5 of them).
> Everything went smooth, and the tasks kept running :)
>
> On Tue, Sep 22, 2015 at 1:18 PM, Mauricio Garavaglia <
> mauriciogaravaglia@gmail.com> wrote:
>
> > Thanks for answers guys, really appreciated, I'll let you know how it
> goes.
> >
> > On Mon, Sep 21, 2015 at 4:09 PM, Joshua Cohen <jc...@twopensource.com>
> > wrote:
> >
> >> Filed https://issues.apache.org/jira/browse/AURORA-1497 to add
> >> documentation on how best to upgrade the various Aurora components.
> >>
> >> On Mon, Sep 21, 2015 at 12:58 PM, Joseph Smith <ya...@gmail.com>
> >> wrote:
> >>
> >> > We also shut down all of the schedulers, upgrade to the new version,
> >> then
> >> > bring them all back up. This isn’t a hard requirement, but it has made
> >> > things a bit simpler for us in practice to always know we’re dealing
> >> with
> >> > the same version of the schedulers on all hosts at once.
> >> >
> >> > > On Sep 21, 2015, at 10:55 AM, Zameer Manji <zm...@apache.org>
> wrote:
> >> > >
> >> > > I don't think the 0.7 -> 0.9 upgrade path has been tested. I think
> it
> >> is
> >> > > advisable to upgrade from 0.7 to 0.8 then later upgrade from 0.8 to
> >> 0.9.
> >> > I
> >> > > think Jeff's upgrade steps are reasonable and shouldn't give you any
> >> > > trouble. Note that upgrading the executors won't affect existing
> >> tasks.
> >> > If
> >> > > you want the current tasks on the system to adopt the latest
> executor
> >> you
> >> > > will need to upgrade the executor and then roll your tasks so they
> >> can be
> >> > > relaunched with the updated binary. This isn't necessary to do but
> >> keep
> >> > > this in mind if you are expecting tasks to leverage the latest
> >> executor
> >> > > functionality.
> >> > >
> >> > > On Sun, Sep 20, 2015 at 8:30 PM, Jeff Schroeder <
> >> > jeffschroeder@computer.org>
> >> > > wrote:
> >> > >
> >> > >> When I did this, I updated the executors, then the observers, and
> >> then
> >> > the
> >> > >> schedulers, one at a time, in a rolling fashion. Seemed to work
> fine,
> >> > but
> >> > >> the clusters were lightly utilized.
> >> > >>
> >> > >> On Sun, Sep 20, 2015 at 9:40 PM, Mauricio Garavaglia <
> >> > >> mauriciogaravaglia@gmail.com> wrote:
> >> > >>
> >> > >>> Hi guys,
> >> > >>>
> >> > >>> I'm about to upgrade a cluster from 0.7 to 0.9 and was wondering
> >> what
> >> > are
> >> > >>> the things to consider to make it as seamless as possible?
> >> > >>>
> >> > >>> For example, update the executors first and then the schedulers;
> >> just
> >> > >>> update the schedulers one at a time, etc.
> >> > >>> Thanks!
> >> > >>>
> >> > >>> Mauricio
> >> > >>>
> >> > >>
> >> > >>
> >> > >>
> >> > >> --
> >> > >> Jeff Schroeder
> >> > >>
> >> > >> Don't drink and derive, alcohol and analysis don't mix.
> >> > >> http://www.digitalprognosis.com
> >> > >>
> >> > >> --
> >> > >> Zameer Manji
> >> > >>
> >> > >> <http://www.digitalprognosis.com>
> >> >
> >> >
> >>
> >
> >
>
>
Re: Upgrading from 0.7
Posted by Raghuveer Pallikonda <pa...@yahoo.com.INVALID>.
Hi Mauricio, Thank you for the update regarding the rollout, When you say the tasks continued running uninterrupted through the executor procedure, does it mean the existing tasks are still running with the old version of the executors? As that process did not exit during the upgrade.
Would appreciate your clarification on the above.
ThanksRaghu
On Wednesday, September 23, 2015 9:19 PM, Mauricio Garavaglia <ma...@gmail.com> wrote:
Hi,
I updated the executors and then rolled the schedulers (have 5 of them).
Everything went smooth, and the tasks kept running :)
On Tue, Sep 22, 2015 at 1:18 PM, Mauricio Garavaglia <
mauriciogaravaglia@gmail.com> wrote:
> Thanks for answers guys, really appreciated, I'll let you know how it goes.
>
> On Mon, Sep 21, 2015 at 4:09 PM, Joshua Cohen <jc...@twopensource.com>
> wrote:
>
>> Filed https://issues.apache.org/jira/browse/AURORA-1497 to add
>> documentation on how best to upgrade the various Aurora components.
>>
>> On Mon, Sep 21, 2015 at 12:58 PM, Joseph Smith <ya...@gmail.com>
>> wrote:
>>
>> > We also shut down all of the schedulers, upgrade to the new version,
>> then
>> > bring them all back up. This isn’t a hard requirement, but it has made
>> > things a bit simpler for us in practice to always know we’re dealing
>> with
>> > the same version of the schedulers on all hosts at once.
>> >
>> > > On Sep 21, 2015, at 10:55 AM, Zameer Manji <zm...@apache.org> wrote:
>> > >
>> > > I don't think the 0.7 -> 0.9 upgrade path has been tested. I think it
>> is
>> > > advisable to upgrade from 0.7 to 0.8 then later upgrade from 0.8 to
>> 0.9.
>> > I
>> > > think Jeff's upgrade steps are reasonable and shouldn't give you any
>> > > trouble. Note that upgrading the executors won't affect existing
>> tasks.
>> > If
>> > > you want the current tasks on the system to adopt the latest executor
>> you
>> > > will need to upgrade the executor and then roll your tasks so they
>> can be
>> > > relaunched with the updated binary. This isn't necessary to do but
>> keep
>> > > this in mind if you are expecting tasks to leverage the latest
>> executor
>> > > functionality.
>> > >
>> > > On Sun, Sep 20, 2015 at 8:30 PM, Jeff Schroeder <
>> > jeffschroeder@computer.org>
>> > > wrote:
>> > >
>> > >> When I did this, I updated the executors, then the observers, and
>> then
>> > the
>> > >> schedulers, one at a time, in a rolling fashion. Seemed to work fine,
>> > but
>> > >> the clusters were lightly utilized.
>> > >>
>> > >> On Sun, Sep 20, 2015 at 9:40 PM, Mauricio Garavaglia <
>> > >> mauriciogaravaglia@gmail.com> wrote:
>> > >>
>> > >>> Hi guys,
>> > >>>
>> > >>> I'm about to upgrade a cluster from 0.7 to 0.9 and was wondering
>> what
>> > are
>> > >>> the things to consider to make it as seamless as possible?
>> > >>>
>> > >>> For example, update the executors first and then the schedulers;
>> just
>> > >>> update the schedulers one at a time, etc.
>> > >>> Thanks!
>> > >>>
>> > >>> Mauricio
>> > >>>
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> Jeff Schroeder
>> > >>
>> > >> Don't drink and derive, alcohol and analysis don't mix.
>> > >> http://www.digitalprognosis.com
>> > >>
>> > >> --
>> > >> Zameer Manji
>> > >>
>> > >> <http://www.digitalprognosis.com>
>> >
>> >
>>
>
>
Re: Upgrading from 0.7
Posted by Mauricio Garavaglia <ma...@gmail.com>.
Hi,
I updated the executors and then rolled the schedulers (have 5 of them).
Everything went smooth, and the tasks kept running :)
On Tue, Sep 22, 2015 at 1:18 PM, Mauricio Garavaglia <
mauriciogaravaglia@gmail.com> wrote:
> Thanks for answers guys, really appreciated, I'll let you know how it goes.
>
> On Mon, Sep 21, 2015 at 4:09 PM, Joshua Cohen <jc...@twopensource.com>
> wrote:
>
>> Filed https://issues.apache.org/jira/browse/AURORA-1497 to add
>> documentation on how best to upgrade the various Aurora components.
>>
>> On Mon, Sep 21, 2015 at 12:58 PM, Joseph Smith <ya...@gmail.com>
>> wrote:
>>
>> > We also shut down all of the schedulers, upgrade to the new version,
>> then
>> > bring them all back up. This isn’t a hard requirement, but it has made
>> > things a bit simpler for us in practice to always know we’re dealing
>> with
>> > the same version of the schedulers on all hosts at once.
>> >
>> > > On Sep 21, 2015, at 10:55 AM, Zameer Manji <zm...@apache.org> wrote:
>> > >
>> > > I don't think the 0.7 -> 0.9 upgrade path has been tested. I think it
>> is
>> > > advisable to upgrade from 0.7 to 0.8 then later upgrade from 0.8 to
>> 0.9.
>> > I
>> > > think Jeff's upgrade steps are reasonable and shouldn't give you any
>> > > trouble. Note that upgrading the executors won't affect existing
>> tasks.
>> > If
>> > > you want the current tasks on the system to adopt the latest executor
>> you
>> > > will need to upgrade the executor and then roll your tasks so they
>> can be
>> > > relaunched with the updated binary. This isn't necessary to do but
>> keep
>> > > this in mind if you are expecting tasks to leverage the latest
>> executor
>> > > functionality.
>> > >
>> > > On Sun, Sep 20, 2015 at 8:30 PM, Jeff Schroeder <
>> > jeffschroeder@computer.org>
>> > > wrote:
>> > >
>> > >> When I did this, I updated the executors, then the observers, and
>> then
>> > the
>> > >> schedulers, one at a time, in a rolling fashion. Seemed to work fine,
>> > but
>> > >> the clusters were lightly utilized.
>> > >>
>> > >> On Sun, Sep 20, 2015 at 9:40 PM, Mauricio Garavaglia <
>> > >> mauriciogaravaglia@gmail.com> wrote:
>> > >>
>> > >>> Hi guys,
>> > >>>
>> > >>> I'm about to upgrade a cluster from 0.7 to 0.9 and was wondering
>> what
>> > are
>> > >>> the things to consider to make it as seamless as possible?
>> > >>>
>> > >>> For example, update the executors first and then the schedulers;
>> just
>> > >>> update the schedulers one at a time, etc.
>> > >>> Thanks!
>> > >>>
>> > >>> Mauricio
>> > >>>
>> > >>
>> > >>
>> > >>
>> > >> --
>> > >> Jeff Schroeder
>> > >>
>> > >> Don't drink and derive, alcohol and analysis don't mix.
>> > >> http://www.digitalprognosis.com
>> > >>
>> > >> --
>> > >> Zameer Manji
>> > >>
>> > >> <http://www.digitalprognosis.com>
>> >
>> >
>>
>
>
Re: Upgrading from 0.7
Posted by Mauricio Garavaglia <ma...@gmail.com>.
Thanks for answers guys, really appreciated, I'll let you know how it goes.
On Mon, Sep 21, 2015 at 4:09 PM, Joshua Cohen <jc...@twopensource.com>
wrote:
> Filed https://issues.apache.org/jira/browse/AURORA-1497 to add
> documentation on how best to upgrade the various Aurora components.
>
> On Mon, Sep 21, 2015 at 12:58 PM, Joseph Smith <ya...@gmail.com>
> wrote:
>
> > We also shut down all of the schedulers, upgrade to the new version, then
> > bring them all back up. This isn’t a hard requirement, but it has made
> > things a bit simpler for us in practice to always know we’re dealing with
> > the same version of the schedulers on all hosts at once.
> >
> > > On Sep 21, 2015, at 10:55 AM, Zameer Manji <zm...@apache.org> wrote:
> > >
> > > I don't think the 0.7 -> 0.9 upgrade path has been tested. I think it
> is
> > > advisable to upgrade from 0.7 to 0.8 then later upgrade from 0.8 to
> 0.9.
> > I
> > > think Jeff's upgrade steps are reasonable and shouldn't give you any
> > > trouble. Note that upgrading the executors won't affect existing tasks.
> > If
> > > you want the current tasks on the system to adopt the latest executor
> you
> > > will need to upgrade the executor and then roll your tasks so they can
> be
> > > relaunched with the updated binary. This isn't necessary to do but keep
> > > this in mind if you are expecting tasks to leverage the latest executor
> > > functionality.
> > >
> > > On Sun, Sep 20, 2015 at 8:30 PM, Jeff Schroeder <
> > jeffschroeder@computer.org>
> > > wrote:
> > >
> > >> When I did this, I updated the executors, then the observers, and then
> > the
> > >> schedulers, one at a time, in a rolling fashion. Seemed to work fine,
> > but
> > >> the clusters were lightly utilized.
> > >>
> > >> On Sun, Sep 20, 2015 at 9:40 PM, Mauricio Garavaglia <
> > >> mauriciogaravaglia@gmail.com> wrote:
> > >>
> > >>> Hi guys,
> > >>>
> > >>> I'm about to upgrade a cluster from 0.7 to 0.9 and was wondering what
> > are
> > >>> the things to consider to make it as seamless as possible?
> > >>>
> > >>> For example, update the executors first and then the schedulers; just
> > >>> update the schedulers one at a time, etc.
> > >>> Thanks!
> > >>>
> > >>> Mauricio
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> Jeff Schroeder
> > >>
> > >> Don't drink and derive, alcohol and analysis don't mix.
> > >> http://www.digitalprognosis.com
> > >>
> > >> --
> > >> Zameer Manji
> > >>
> > >> <http://www.digitalprognosis.com>
> >
> >
>
Re: Upgrading from 0.7
Posted by Joshua Cohen <jc...@twopensource.com>.
Filed https://issues.apache.org/jira/browse/AURORA-1497 to add
documentation on how best to upgrade the various Aurora components.
On Mon, Sep 21, 2015 at 12:58 PM, Joseph Smith <ya...@gmail.com> wrote:
> We also shut down all of the schedulers, upgrade to the new version, then
> bring them all back up. This isn’t a hard requirement, but it has made
> things a bit simpler for us in practice to always know we’re dealing with
> the same version of the schedulers on all hosts at once.
>
> > On Sep 21, 2015, at 10:55 AM, Zameer Manji <zm...@apache.org> wrote:
> >
> > I don't think the 0.7 -> 0.9 upgrade path has been tested. I think it is
> > advisable to upgrade from 0.7 to 0.8 then later upgrade from 0.8 to 0.9.
> I
> > think Jeff's upgrade steps are reasonable and shouldn't give you any
> > trouble. Note that upgrading the executors won't affect existing tasks.
> If
> > you want the current tasks on the system to adopt the latest executor you
> > will need to upgrade the executor and then roll your tasks so they can be
> > relaunched with the updated binary. This isn't necessary to do but keep
> > this in mind if you are expecting tasks to leverage the latest executor
> > functionality.
> >
> > On Sun, Sep 20, 2015 at 8:30 PM, Jeff Schroeder <
> jeffschroeder@computer.org>
> > wrote:
> >
> >> When I did this, I updated the executors, then the observers, and then
> the
> >> schedulers, one at a time, in a rolling fashion. Seemed to work fine,
> but
> >> the clusters were lightly utilized.
> >>
> >> On Sun, Sep 20, 2015 at 9:40 PM, Mauricio Garavaglia <
> >> mauriciogaravaglia@gmail.com> wrote:
> >>
> >>> Hi guys,
> >>>
> >>> I'm about to upgrade a cluster from 0.7 to 0.9 and was wondering what
> are
> >>> the things to consider to make it as seamless as possible?
> >>>
> >>> For example, update the executors first and then the schedulers; just
> >>> update the schedulers one at a time, etc.
> >>> Thanks!
> >>>
> >>> Mauricio
> >>>
> >>
> >>
> >>
> >> --
> >> Jeff Schroeder
> >>
> >> Don't drink and derive, alcohol and analysis don't mix.
> >> http://www.digitalprognosis.com
> >>
> >> --
> >> Zameer Manji
> >>
> >> <http://www.digitalprognosis.com>
>
>
Re: Upgrading from 0.7
Posted by Joseph Smith <ya...@gmail.com>.
We also shut down all of the schedulers, upgrade to the new version, then bring them all back up. This isn’t a hard requirement, but it has made things a bit simpler for us in practice to always know we’re dealing with the same version of the schedulers on all hosts at once.
> On Sep 21, 2015, at 10:55 AM, Zameer Manji <zm...@apache.org> wrote:
>
> I don't think the 0.7 -> 0.9 upgrade path has been tested. I think it is
> advisable to upgrade from 0.7 to 0.8 then later upgrade from 0.8 to 0.9. I
> think Jeff's upgrade steps are reasonable and shouldn't give you any
> trouble. Note that upgrading the executors won't affect existing tasks. If
> you want the current tasks on the system to adopt the latest executor you
> will need to upgrade the executor and then roll your tasks so they can be
> relaunched with the updated binary. This isn't necessary to do but keep
> this in mind if you are expecting tasks to leverage the latest executor
> functionality.
>
> On Sun, Sep 20, 2015 at 8:30 PM, Jeff Schroeder <je...@computer.org>
> wrote:
>
>> When I did this, I updated the executors, then the observers, and then the
>> schedulers, one at a time, in a rolling fashion. Seemed to work fine, but
>> the clusters were lightly utilized.
>>
>> On Sun, Sep 20, 2015 at 9:40 PM, Mauricio Garavaglia <
>> mauriciogaravaglia@gmail.com> wrote:
>>
>>> Hi guys,
>>>
>>> I'm about to upgrade a cluster from 0.7 to 0.9 and was wondering what are
>>> the things to consider to make it as seamless as possible?
>>>
>>> For example, update the executors first and then the schedulers; just
>>> update the schedulers one at a time, etc.
>>> Thanks!
>>>
>>> Mauricio
>>>
>>
>>
>>
>> --
>> Jeff Schroeder
>>
>> Don't drink and derive, alcohol and analysis don't mix.
>> http://www.digitalprognosis.com
>>
>> --
>> Zameer Manji
>>
>> <http://www.digitalprognosis.com>
Re: Upgrading from 0.7
Posted by Zameer Manji <zm...@apache.org>.
I don't think the 0.7 -> 0.9 upgrade path has been tested. I think it is
advisable to upgrade from 0.7 to 0.8 then later upgrade from 0.8 to 0.9. I
think Jeff's upgrade steps are reasonable and shouldn't give you any
trouble. Note that upgrading the executors won't affect existing tasks. If
you want the current tasks on the system to adopt the latest executor you
will need to upgrade the executor and then roll your tasks so they can be
relaunched with the updated binary. This isn't necessary to do but keep
this in mind if you are expecting tasks to leverage the latest executor
functionality.
On Sun, Sep 20, 2015 at 8:30 PM, Jeff Schroeder <je...@computer.org>
wrote:
> When I did this, I updated the executors, then the observers, and then the
> schedulers, one at a time, in a rolling fashion. Seemed to work fine, but
> the clusters were lightly utilized.
>
> On Sun, Sep 20, 2015 at 9:40 PM, Mauricio Garavaglia <
> mauriciogaravaglia@gmail.com> wrote:
>
> > Hi guys,
> >
> > I'm about to upgrade a cluster from 0.7 to 0.9 and was wondering what are
> > the things to consider to make it as seamless as possible?
> >
> > For example, update the executors first and then the schedulers; just
> > update the schedulers one at a time, etc.
> > Thanks!
> >
> > Mauricio
> >
>
>
>
> --
> Jeff Schroeder
>
> Don't drink and derive, alcohol and analysis don't mix.
> http://www.digitalprognosis.com
>
> --
> Zameer Manji
>
> <http://www.digitalprognosis.com>
Re: Upgrading from 0.7
Posted by Jeff Schroeder <je...@computer.org>.
When I did this, I updated the executors, then the observers, and then the
schedulers, one at a time, in a rolling fashion. Seemed to work fine, but
the clusters were lightly utilized.
On Sun, Sep 20, 2015 at 9:40 PM, Mauricio Garavaglia <
mauriciogaravaglia@gmail.com> wrote:
> Hi guys,
>
> I'm about to upgrade a cluster from 0.7 to 0.9 and was wondering what are
> the things to consider to make it as seamless as possible?
>
> For example, update the executors first and then the schedulers; just
> update the schedulers one at a time, etc.
> Thanks!
>
> Mauricio
>
--
Jeff Schroeder
Don't drink and derive, alcohol and analysis don't mix.
http://www.digitalprognosis.com