You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by Grzegorz Graczyk <gr...@gmail.com> on 2015/10/22 12:18:23 UTC
Recover docker containers when mesos-slave is contained in docker as well
Hi everyone,
I was wondering if it's possible to recover running docker containers after
restart of mesos-slave?
If it is possible - what are the requirements to do so?
Regards,
Grzegorz Graczyk
Re: Recover docker containers when mesos-slave is contained in docker
as well
Posted by haosdent <ha...@gmail.com>.
And if your slave id change after restart, the slave would be treated as a
new slave and could not recover successfully. It would recover the docker
containers which start with "mesos-${slave_id}".
On Thu, Oct 22, 2015 at 7:08 PM, haosdent <ha...@gmail.com> wrote:
> As far as I know, when mesos-slave run in docker, the recover process is
> same as the slave run outside. But you need make sure you add "-v
> /var/run/docker.sock:/var/run/docker.sock" when you launch mesos-slave.
>
> On Thu, Oct 22, 2015 at 6:59 PM, Grzegorz Graczyk <gr...@gmail.com>
> wrote:
>
>> Docker is running when slave exits - and so are docker containers started
>> by mesos slave. The problem starts when slave is online again and cannot
>> see already started containers and recover them...
>> Isn't this supposed to fix that problem?
>> https://issues.apache.org/jira/browse/MESOS-2115
>>
>> On 22 October 2015 at 12:54, Klaus Ma <kl...@gmail.com> wrote:
>>
>>> It seems we can NOT keep docker running but slave exit.
>>>
>>> On Thu, Oct 22, 2015 at 6:18 AM, Grzegorz Graczyk <gr...@gmail.com>
>>> wrote:
>>>
>>>> Hi everyone,
>>>> I was wondering if it's possible to recover running docker containers
>>>> after restart of mesos-slave?
>>>> If it is possible - what are the requirements to do so?
>>>>
>>>> Regards,
>>>> Grzegorz Graczyk
>>>>
>>>
>>>
>>>
>>> --
>>> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
>>> Platform Symphony/DCOS Development & Support, STG, IBM GCG
>>> +86-10-8245 4084 | madaxa@cn.ibm.com | http://www.cguru.net
>>>
>>
>>
>
>
> --
> Best Regards,
> Haosdent Huang
>
--
Best Regards,
Haosdent Huang
Re: Recover docker containers when mesos-slave is contained in docker
as well
Posted by haosdent <ha...@gmail.com>.
As far as I know, when mesos-slave run in docker, the recover process is
same as the slave run outside. But you need make sure you add "-v
/var/run/docker.sock:/var/run/docker.sock" when you launch mesos-slave.
On Thu, Oct 22, 2015 at 6:59 PM, Grzegorz Graczyk <gr...@gmail.com>
wrote:
> Docker is running when slave exits - and so are docker containers started
> by mesos slave. The problem starts when slave is online again and cannot
> see already started containers and recover them...
> Isn't this supposed to fix that problem?
> https://issues.apache.org/jira/browse/MESOS-2115
>
> On 22 October 2015 at 12:54, Klaus Ma <kl...@gmail.com> wrote:
>
>> It seems we can NOT keep docker running but slave exit.
>>
>> On Thu, Oct 22, 2015 at 6:18 AM, Grzegorz Graczyk <gr...@gmail.com>
>> wrote:
>>
>>> Hi everyone,
>>> I was wondering if it's possible to recover running docker containers
>>> after restart of mesos-slave?
>>> If it is possible - what are the requirements to do so?
>>>
>>> Regards,
>>> Grzegorz Graczyk
>>>
>>
>>
>>
>> --
>> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
>> Platform Symphony/DCOS Development & Support, STG, IBM GCG
>> +86-10-8245 4084 | madaxa@cn.ibm.com | http://www.cguru.net
>>
>
>
--
Best Regards,
Haosdent Huang
Re: Recover docker containers when mesos-slave is contained in docker
as well
Posted by Grzegorz Graczyk <gr...@gmail.com>.
Thank you for all the answers!
This is how I run mesos-slave on CoreOS 766.4.0(latest stable), Docker
1.7.1:
https://gist.github.com/gregory90/5588bd8980e3186a4442
I'm trying to run container using Marathon 0.11.0:
https://gist.github.com/gregory90/3cc03cce48389d198171
Container isn't started, and I get these logs(from mesos-slave and docker):
https://gist.github.com/gregory90/dfb70f09f8fde75a0f2d
Everything work fine if I
remove --docker_mesos_image=mesosphere/mesos-slave:0.25.0-0.2.70.ubuntu1404
from mesos-slave start command, but then recovering docker containers after
restart doesn't work.
Is this misconfiguration on my side?
On 22 October 2015 at 14:07, Jan Stabenow <j....@livespottingmedia.com>
wrote:
> Hey Grzegorz,
>
> here is my example:
>
> https://github.com/jstabenow/mesos-on-docker-on-coreos/blob/master/mesos-slave.service
>
> Hope this will help you.
>
> Regards,
> Jan
>
>
> Am 22.10.2015 um 13:26 schrieb Timothy Chen <ti...@mesosphere.io>:
>
> Hi Grzegorz,
>
> Yes it's possible, but do require some configuration for the slave recover
> the running containers. This is needed to run Mesos on CoreOS as well, so
> it's made possible I believe around 0.24.1 or later.
>
> Basically to have the slave to recover the task containers, the executors
> that watches the tasks need to be launched in containers as well. This is
> made possible with the docker_mesos_image slave flag, where the docker
> containerizer will use this image to launch executors. This should be the
> same image used to launch the slave itself.
>
> Also when launching the slave in a docker container it must have the
> following docker flags:
>
> --pid=host (so all processes can be visible to the slave)
> -v /var/run/docker.sock:/var/run/docker.sock (slave can then launch
> containers as peers)
> -v /tmp/mesos:/tmp/mesos (we need slave work directory information to
> persist when slave recovers. You can also create a separate dir on the host
> if you want to run multiple slaves)
>
> Tim
>
> On Oct 22, 2015, at 6:59 PM, Grzegorz Graczyk <gr...@gmail.com> wrote:
>
> Docker is running when slave exits - and so are docker containers started
> by mesos slave. The problem starts when slave is online again and cannot
> see already started containers and recover them...
> Isn't this supposed to fix that problem?
> https://issues.apache.org/jira/browse/MESOS-2115
>
> On 22 October 2015 at 12:54, Klaus Ma <kl...@gmail.com> wrote:
>
>> It seems we can NOT keep docker running but slave exit.
>>
>> On Thu, Oct 22, 2015 at 6:18 AM, Grzegorz Graczyk <gr...@gmail.com>
>> wrote:
>>
>>> Hi everyone,
>>> I was wondering if it's possible to recover running docker containers
>>> after restart of mesos-slave?
>>> If it is possible - what are the requirements to do so?
>>>
>>> Regards,
>>> Grzegorz Graczyk
>>>
>>
>>
>>
>> --
>> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
>> Platform Symphony/DCOS Development & Support, STG, IBM GCG
>> +86-10-8245 4084 | madaxa@cn.ibm.com | http://www.cguru.net
>>
>
>
>
Re: Recover docker containers when mesos-slave is contained in docker as well
Posted by Jan Stabenow <j....@livespottingmedia.com>.
Hey Grzegorz,
here is my example:
https://github.com/jstabenow/mesos-on-docker-on-coreos/blob/master/mesos-slave.service <https://github.com/jstabenow/mesos-on-docker-on-coreos/blob/master/mesos-slave.service>
Hope this will help you.
Regards,
Jan
> Am 22.10.2015 um 13:26 schrieb Timothy Chen <ti...@mesosphere.io>:
>
> Hi Grzegorz,
>
> Yes it's possible, but do require some configuration for the slave recover the running containers. This is needed to run Mesos on CoreOS as well, so it's made possible I believe around 0.24.1 or later.
>
> Basically to have the slave to recover the task containers, the executors that watches the tasks need to be launched in containers as well. This is made possible with the docker_mesos_image slave flag, where the docker containerizer will use this image to launch executors. This should be the same image used to launch the slave itself.
>
> Also when launching the slave in a docker container it must have the following docker flags:
>
> --pid=host (so all processes can be visible to the slave)
> -v /var/run/docker.sock:/var/run/docker.sock (slave can then launch containers as peers)
> -v /tmp/mesos:/tmp/mesos (we need slave work directory information to persist when slave recovers. You can also create a separate dir on the host if you want to run multiple slaves)
>
> Tim
>
> On Oct 22, 2015, at 6:59 PM, Grzegorz Graczyk <gregory90@gmail.com <ma...@gmail.com>> wrote:
>
>> Docker is running when slave exits - and so are docker containers started by mesos slave. The problem starts when slave is online again and cannot see already started containers and recover them...
>> Isn't this supposed to fix that problem? https://issues.apache.org/jira/browse/MESOS-2115 <https://issues.apache.org/jira/browse/MESOS-2115>
>>
>> On 22 October 2015 at 12:54, Klaus Ma <klaus1982.cn@gmail.com <ma...@gmail.com>> wrote:
>> It seems we can NOT keep docker running but slave exit.
>>
>> On Thu, Oct 22, 2015 at 6:18 AM, Grzegorz Graczyk <gregory90@gmail.com <ma...@gmail.com>> wrote:
>> Hi everyone,
>> I was wondering if it's possible to recover running docker containers after restart of mesos-slave?
>> If it is possible - what are the requirements to do so?
>>
>> Regards,
>> Grzegorz Graczyk
>>
>>
>>
>> --
>> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
>> Platform Symphony/DCOS Development & Support, STG, IBM GCG
>> +86-10-8245 4084 <tel:%2B86-10-8245%204084> | madaxa@cn.ibm.com <ma...@cn.ibm.com> | http://www.cguru.net <http://www.cguru.net/>
>>
Re: Recover docker containers when mesos-slave is contained in docker as well
Posted by Timothy Chen <ti...@mesosphere.io>.
Hi Grzegorz,
Yes it's possible, but do require some configuration for the slave recover the running containers. This is needed to run Mesos on CoreOS as well, so it's made possible I believe around 0.24.1 or later.
Basically to have the slave to recover the task containers, the executors that watches the tasks need to be launched in containers as well. This is made possible with the docker_mesos_image slave flag, where the docker containerizer will use this image to launch executors. This should be the same image used to launch the slave itself.
Also when launching the slave in a docker container it must have the following docker flags:
--pid=host (so all processes can be visible to the slave)
-v /var/run/docker.sock:/var/run/docker.sock (slave can then launch containers as peers)
-v /tmp/mesos:/tmp/mesos (we need slave work directory information to persist when slave recovers. You can also create a separate dir on the host if you want to run multiple slaves)
Tim
> On Oct 22, 2015, at 6:59 PM, Grzegorz Graczyk <gr...@gmail.com> wrote:
>
> Docker is running when slave exits - and so are docker containers started by mesos slave. The problem starts when slave is online again and cannot see already started containers and recover them...
> Isn't this supposed to fix that problem? https://issues.apache.org/jira/browse/MESOS-2115
>
>> On 22 October 2015 at 12:54, Klaus Ma <kl...@gmail.com> wrote:
>> It seems we can NOT keep docker running but slave exit.
>>
>>> On Thu, Oct 22, 2015 at 6:18 AM, Grzegorz Graczyk <gr...@gmail.com> wrote:
>>> Hi everyone,
>>> I was wondering if it's possible to recover running docker containers after restart of mesos-slave?
>>> If it is possible - what are the requirements to do so?
>>>
>>> Regards,
>>> Grzegorz Graczyk
>>
>>
>>
>> --
>> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
>> Platform Symphony/DCOS Development & Support, STG, IBM GCG
>> +86-10-8245 4084 | madaxa@cn.ibm.com | http://www.cguru.net
>
Re: Recover docker containers when mesos-slave is contained in docker
as well
Posted by Grzegorz Graczyk <gr...@gmail.com>.
Docker is running when slave exits - and so are docker containers started
by mesos slave. The problem starts when slave is online again and cannot
see already started containers and recover them...
Isn't this supposed to fix that problem?
https://issues.apache.org/jira/browse/MESOS-2115
On 22 October 2015 at 12:54, Klaus Ma <kl...@gmail.com> wrote:
> It seems we can NOT keep docker running but slave exit.
>
> On Thu, Oct 22, 2015 at 6:18 AM, Grzegorz Graczyk <gr...@gmail.com>
> wrote:
>
>> Hi everyone,
>> I was wondering if it's possible to recover running docker containers
>> after restart of mesos-slave?
>> If it is possible - what are the requirements to do so?
>>
>> Regards,
>> Grzegorz Graczyk
>>
>
>
>
> --
> Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
> Platform Symphony/DCOS Development & Support, STG, IBM GCG
> +86-10-8245 4084 | madaxa@cn.ibm.com | http://www.cguru.net
>
Re: Recover docker containers when mesos-slave is contained in docker
as well
Posted by Klaus Ma <kl...@gmail.com>.
It seems we can NOT keep docker running but slave exit.
On Thu, Oct 22, 2015 at 6:18 AM, Grzegorz Graczyk <gr...@gmail.com>
wrote:
> Hi everyone,
> I was wondering if it's possible to recover running docker containers
> after restart of mesos-slave?
> If it is possible - what are the requirements to do so?
>
> Regards,
> Grzegorz Graczyk
>
--
Da (Klaus), Ma (马达) | PMP® | Advisory Software Engineer
Platform Symphony/DCOS Development & Support, STG, IBM GCG
+86-10-8245 4084 | madaxa@cn.ibm.com | http://www.cguru.net