You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by craig w <co...@gmail.com> on 2015/10/13 11:39:47 UTC
upgrade from 0.24.1 to 0.25
When upgrading from 0.23.0 to 0.24.1, I installed the new binaries and
restarted the masters (all at once), then restarted all of the slaves.
I then observed all of the tasks that were running (via Marathon 0.10.x)
were restarted. I had expected "no downtime" or restarts, did I
misunderstand the upgrade instructions or did I perhaps do something
incorrectly?
When restartng the masters, should they be restarted gradually (restart a
master, wait 30 seconds, restart the next)?
I'm looking to upgrade from 0.24.1 to 0.25.0 but want to avoid having all
of the tasks restart again.
Thanks,
Craig
Re: upgrade from 0.24.1 to 0.25
Posted by Jeremy Olexa <jo...@spscommerce.com>.
Hi Craig, it was posted on the marathon email list that 0.11.0 is not safe for production.
https://groups.google.com/d/msg/marathon-framework/u4-FKVkh5RQ/wH-s1sdECgAJ
________________________________
From: craig w <co...@gmail.com>
Sent: Thursday, October 15, 2015 4:13 AM
To: user@mesos.apache.org
Subject: Re: upgrade from 0.24.1 to 0.25
Itamar,
Where did you see Marathon 0.11.0 was not ready for production? I ran into this issue (https://github.com/mesosphere/marathon/issues/2369), however, it turned out I just needed to update a configuration parameter. So far, Mesos 0.24.1 and Marathon 0.11.0 are working great. I'm hoping to upgrade to 0.25.0 soon.
On Thu, Oct 15, 2015 at 4:16 AM, Itamar Ostricher <it...@yowza3d.com>> wrote:
Also note that you're on a problematic Marathon version.
I was thinking to upgrade from 0.23&0.10.1 to latest versions, but decided to wait after I saw an announcement on the marathon users list that 0.11 is not recommended for production from last week (Oct 9).
I'm waiting until 0.11.1 is out... (hopefully soon?)
On Wed, Oct 14, 2015, 10:16 Adam Bordelon <ad...@mesosphere.io>> wrote:
> When restartng the masters, should they be restarted gradually (restart a master, wait 30 seconds, restart the next)?
Craig, ideally you should roll the masters one at a time (maintain a --quorum of masters up at all times), providing enough time for a new master to recover the replicated log and be ready for a failover. This reduces downtime and guarantees a continuous connection to frameworks/agents. If you restart all the masters at once, there's a period when they're all inaccessible, and you don't maintain quorum.
On Tue, Oct 13, 2015 at 9:34 AM, craig w <co...@gmail.com>> wrote:
I have not tried to upgrade to 0.25.0 yet. I'm hoping to try the upgrade this week. We're currently on mesos 0.24.1 and marathon 0.11.0 (just upgraded this today).
I'll give the upgrade to 0.25.0 a shot in a test environment, in the meantime if you perform an experiment I'd be interested in your findings.
Thanks,
Craig
On Tue, Oct 13, 2015 at 12:25 PM, Niklas Nielsen <ni...@mesosphere.io>> wrote:
Hi Craig,
That should definitely not happen; did you try to upgrade to 0.25.0 already? If not, we can try to run an upgrade experiment with that marathon version.
Niklas
On 13 October 2015 at 02:39, craig w <co...@gmail.com>> wrote:
When upgrading from 0.23.0 to 0.24.1, I installed the new binaries and restarted the masters (all at once), then restarted all of the slaves.
I then observed all of the tasks that were running (via Marathon 0.10.x) were restarted. I had expected "no downtime" or restarts, did I misunderstand the upgrade instructions or did I perhaps do something incorrectly?
When restartng the masters, should they be restarted gradually (restart a master, wait 30 seconds, restart the next)?
I'm looking to upgrade from 0.24.1 to 0.25.0 but want to avoid having all of the tasks restart again.
Thanks,
Craig
--
https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links
--
https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links
Re: upgrade from 0.24.1 to 0.25
Posted by craig w <co...@gmail.com>.
Itamar,
Where did you see Marathon 0.11.0 was not ready for production? I ran into
this issue (https://github.com/mesosphere/marathon/issues/2369), however,
it turned out I just needed to update a configuration parameter. So far,
Mesos 0.24.1 and Marathon 0.11.0 are working great. I'm hoping to upgrade
to 0.25.0 soon.
On Thu, Oct 15, 2015 at 4:16 AM, Itamar Ostricher <it...@yowza3d.com>
wrote:
> Also note that you're on a problematic Marathon version.
> I was thinking to upgrade from 0.23&0.10.1 to latest versions, but decided
> to wait after I saw an announcement on the marathon users list that 0.11 is
> not recommended for production from last week (Oct 9).
> I'm waiting until 0.11.1 is out... (hopefully soon?)
>
> On Wed, Oct 14, 2015, 10:16 Adam Bordelon <ad...@mesosphere.io> wrote:
>
>> > When restartng the masters, should they be restarted gradually (restart
>> a master, wait 30 seconds, restart the next)?
>> Craig, ideally you should roll the masters one at a time (maintain a
>> --quorum of masters up at all times), providing enough time for a new
>> master to recover the replicated log and be ready for a failover. This
>> reduces downtime and guarantees a continuous connection to
>> frameworks/agents. If you restart all the masters at once, there's a period
>> when they're all inaccessible, and you don't maintain quorum.
>>
>> On Tue, Oct 13, 2015 at 9:34 AM, craig w <co...@gmail.com> wrote:
>>
>>> I have not tried to upgrade to 0.25.0 yet. I'm hoping to try the upgrade
>>> this week. We're currently on mesos 0.24.1 and marathon 0.11.0 (just
>>> upgraded this today).
>>>
>>> I'll give the upgrade to 0.25.0 a shot in a test environment, in the
>>> meantime if you perform an experiment I'd be interested in your findings.
>>>
>>> Thanks,
>>> Craig
>>>
>>> On Tue, Oct 13, 2015 at 12:25 PM, Niklas Nielsen <ni...@mesosphere.io>
>>> wrote:
>>>
>>>> Hi Craig,
>>>>
>>>> That should definitely not happen; did you try to upgrade to 0.25.0
>>>> already? If not, we can try to run an upgrade experiment with that marathon
>>>> version.
>>>>
>>>> Niklas
>>>>
>>>> On 13 October 2015 at 02:39, craig w <co...@gmail.com> wrote:
>>>>
>>>>> When upgrading from 0.23.0 to 0.24.1, I installed the new binaries and
>>>>> restarted the masters (all at once), then restarted all of the slaves.
>>>>>
>>>>> I then observed all of the tasks that were running (via Marathon
>>>>> 0.10.x) were restarted. I had expected "no downtime" or restarts, did I
>>>>> misunderstand the upgrade instructions or did I perhaps do something
>>>>> incorrectly?
>>>>>
>>>>> When restartng the masters, should they be restarted gradually
>>>>> (restart a master, wait 30 seconds, restart the next)?
>>>>>
>>>>> I'm looking to upgrade from 0.24.1 to 0.25.0 but want to avoid having
>>>>> all of the tasks restart again.
>>>>>
>>>>> Thanks,
>>>>> Craig
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>>
>>> https://github.com/mindscratch
>>> https://www.google.com/+CraigWickesser
>>> https://twitter.com/mind_scratch
>>> https://twitter.com/craig_links
>>>
>>>
>>
--
https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links
Re: upgrade from 0.24.1 to 0.25
Posted by Itamar Ostricher <it...@yowza3d.com>.
Also note that you're on a problematic Marathon version.
I was thinking to upgrade from 0.23&0.10.1 to latest versions, but decided
to wait after I saw an announcement on the marathon users list that 0.11 is
not recommended for production from last week (Oct 9).
I'm waiting until 0.11.1 is out... (hopefully soon?)
On Wed, Oct 14, 2015, 10:16 Adam Bordelon <ad...@mesosphere.io> wrote:
> > When restartng the masters, should they be restarted gradually (restart
> a master, wait 30 seconds, restart the next)?
> Craig, ideally you should roll the masters one at a time (maintain a
> --quorum of masters up at all times), providing enough time for a new
> master to recover the replicated log and be ready for a failover. This
> reduces downtime and guarantees a continuous connection to
> frameworks/agents. If you restart all the masters at once, there's a period
> when they're all inaccessible, and you don't maintain quorum.
>
> On Tue, Oct 13, 2015 at 9:34 AM, craig w <co...@gmail.com> wrote:
>
>> I have not tried to upgrade to 0.25.0 yet. I'm hoping to try the upgrade
>> this week. We're currently on mesos 0.24.1 and marathon 0.11.0 (just
>> upgraded this today).
>>
>> I'll give the upgrade to 0.25.0 a shot in a test environment, in the
>> meantime if you perform an experiment I'd be interested in your findings.
>>
>> Thanks,
>> Craig
>>
>> On Tue, Oct 13, 2015 at 12:25 PM, Niklas Nielsen <ni...@mesosphere.io>
>> wrote:
>>
>>> Hi Craig,
>>>
>>> That should definitely not happen; did you try to upgrade to 0.25.0
>>> already? If not, we can try to run an upgrade experiment with that marathon
>>> version.
>>>
>>> Niklas
>>>
>>> On 13 October 2015 at 02:39, craig w <co...@gmail.com> wrote:
>>>
>>>> When upgrading from 0.23.0 to 0.24.1, I installed the new binaries and
>>>> restarted the masters (all at once), then restarted all of the slaves.
>>>>
>>>> I then observed all of the tasks that were running (via Marathon
>>>> 0.10.x) were restarted. I had expected "no downtime" or restarts, did I
>>>> misunderstand the upgrade instructions or did I perhaps do something
>>>> incorrectly?
>>>>
>>>> When restartng the masters, should they be restarted gradually (restart
>>>> a master, wait 30 seconds, restart the next)?
>>>>
>>>> I'm looking to upgrade from 0.24.1 to 0.25.0 but want to avoid having
>>>> all of the tasks restart again.
>>>>
>>>> Thanks,
>>>> Craig
>>>>
>>>
>>>
>>
>>
>> --
>>
>> https://github.com/mindscratch
>> https://www.google.com/+CraigWickesser
>> https://twitter.com/mind_scratch
>> https://twitter.com/craig_links
>>
>>
>
Re: upgrade from 0.24.1 to 0.25
Posted by Adam Bordelon <ad...@mesosphere.io>.
> When restartng the masters, should they be restarted gradually (restart a
master, wait 30 seconds, restart the next)?
Craig, ideally you should roll the masters one at a time (maintain a
--quorum of masters up at all times), providing enough time for a new
master to recover the replicated log and be ready for a failover. This
reduces downtime and guarantees a continuous connection to
frameworks/agents. If you restart all the masters at once, there's a period
when they're all inaccessible, and you don't maintain quorum.
On Tue, Oct 13, 2015 at 9:34 AM, craig w <co...@gmail.com> wrote:
> I have not tried to upgrade to 0.25.0 yet. I'm hoping to try the upgrade
> this week. We're currently on mesos 0.24.1 and marathon 0.11.0 (just
> upgraded this today).
>
> I'll give the upgrade to 0.25.0 a shot in a test environment, in the
> meantime if you perform an experiment I'd be interested in your findings.
>
> Thanks,
> Craig
>
> On Tue, Oct 13, 2015 at 12:25 PM, Niklas Nielsen <ni...@mesosphere.io>
> wrote:
>
>> Hi Craig,
>>
>> That should definitely not happen; did you try to upgrade to 0.25.0
>> already? If not, we can try to run an upgrade experiment with that marathon
>> version.
>>
>> Niklas
>>
>> On 13 October 2015 at 02:39, craig w <co...@gmail.com> wrote:
>>
>>> When upgrading from 0.23.0 to 0.24.1, I installed the new binaries and
>>> restarted the masters (all at once), then restarted all of the slaves.
>>>
>>> I then observed all of the tasks that were running (via Marathon 0.10.x)
>>> were restarted. I had expected "no downtime" or restarts, did I
>>> misunderstand the upgrade instructions or did I perhaps do something
>>> incorrectly?
>>>
>>> When restartng the masters, should they be restarted gradually (restart
>>> a master, wait 30 seconds, restart the next)?
>>>
>>> I'm looking to upgrade from 0.24.1 to 0.25.0 but want to avoid having
>>> all of the tasks restart again.
>>>
>>> Thanks,
>>> Craig
>>>
>>
>>
>
>
> --
>
> https://github.com/mindscratch
> https://www.google.com/+CraigWickesser
> https://twitter.com/mind_scratch
> https://twitter.com/craig_links
>
>
Re: upgrade from 0.24.1 to 0.25
Posted by craig w <co...@gmail.com>.
I have not tried to upgrade to 0.25.0 yet. I'm hoping to try the upgrade
this week. We're currently on mesos 0.24.1 and marathon 0.11.0 (just
upgraded this today).
I'll give the upgrade to 0.25.0 a shot in a test environment, in the
meantime if you perform an experiment I'd be interested in your findings.
Thanks,
Craig
On Tue, Oct 13, 2015 at 12:25 PM, Niklas Nielsen <ni...@mesosphere.io>
wrote:
> Hi Craig,
>
> That should definitely not happen; did you try to upgrade to 0.25.0
> already? If not, we can try to run an upgrade experiment with that marathon
> version.
>
> Niklas
>
> On 13 October 2015 at 02:39, craig w <co...@gmail.com> wrote:
>
>> When upgrading from 0.23.0 to 0.24.1, I installed the new binaries and
>> restarted the masters (all at once), then restarted all of the slaves.
>>
>> I then observed all of the tasks that were running (via Marathon 0.10.x)
>> were restarted. I had expected "no downtime" or restarts, did I
>> misunderstand the upgrade instructions or did I perhaps do something
>> incorrectly?
>>
>> When restartng the masters, should they be restarted gradually (restart a
>> master, wait 30 seconds, restart the next)?
>>
>> I'm looking to upgrade from 0.24.1 to 0.25.0 but want to avoid having all
>> of the tasks restart again.
>>
>> Thanks,
>> Craig
>>
>
>
--
https://github.com/mindscratch
https://www.google.com/+CraigWickesser
https://twitter.com/mind_scratch
https://twitter.com/craig_links
Re: upgrade from 0.24.1 to 0.25
Posted by Niklas Nielsen <ni...@mesosphere.io>.
Hi Craig,
That should definitely not happen; did you try to upgrade to 0.25.0
already? If not, we can try to run an upgrade experiment with that marathon
version.
Niklas
On 13 October 2015 at 02:39, craig w <co...@gmail.com> wrote:
> When upgrading from 0.23.0 to 0.24.1, I installed the new binaries and
> restarted the masters (all at once), then restarted all of the slaves.
>
> I then observed all of the tasks that were running (via Marathon 0.10.x)
> were restarted. I had expected "no downtime" or restarts, did I
> misunderstand the upgrade instructions or did I perhaps do something
> incorrectly?
>
> When restartng the masters, should they be restarted gradually (restart a
> master, wait 30 seconds, restart the next)?
>
> I'm looking to upgrade from 0.24.1 to 0.25.0 but want to avoid having all
> of the tasks restart again.
>
> Thanks,
> Craig
>