You are viewing a plain text version of this content. The canonical link for it is here.
Posted to user@mesos.apache.org by haosdent <ha...@gmail.com> on 2016/08/01 01:57:12 UTC
Re: Running a mesos executor within a container....
>First, the binaries I get mesos to download into my sandbox don't appear
to be within the docker image
The sandbox folder should mount into the docker container with the same
path, does this not work for you?
>running the executor causes it to complain about libmesos being missing.
Yes, you need package the native library(libmesos) which required by
executor into your jar or package it into your docker container.
Or you could consider implementing your executor base on the new HTTP API (
https://github.com/mesosphere/mesos-rxjava)
On Sun, Jul 31, 2016 at 9:47 PM, Mark Hammons <mark.hammons@inaf.cnrs-gif.fr
> wrote:
> Hi all,
>
>
>
> So far I've been developing a mesos framework using a custom executor and
> scheduler. Custom executors seem to provide a lot of flexibility in
> responding to and communicating with the parent mesos-slave, so I'd like to
> keep that around. However, my users would like to have customizable
> environments for their algorithms to run in (like maybe one user would
> design their application in an ubuntu environment and doesn't want to take
> the time to adapt to the CentOS 7 environment the mesos slave is running).
> So, I'd like to set the sandbox of the executor to be within the context of
> a docker image. I thought doing something like this would work:
> http://pastie.org/10924936
>
>
>
> but it doesn't for three big reasons. First, the binaries I get mesos to
> download into my sandbox don't appear to be within the docker image, and
> second the command defined by execCommand is executed at the root of the
> docker image instead of within the mesos sandbox. Even when I get around
> these issues by adding directives in execCommand to download the missing
> binaries into the docker image, running the executor causes it to complain
> about libmesos being missing. Is there any way to do what I'm trying to do?
> Could someone give me an example project that works this way?
>
>
>
> Thanks,
>
> Mark
> --
>
> Mark Hammons - +33 06 03 69 56 56
>
> Research Engineer @ BioEmergences <http://bioemergences.iscpif.fr>
>
> Lab Phone: 01 69 82 34 19
>
--
Best Regards,
Haosdent Huang
Re: Running a mesos executor within a container....
Posted by Mark Hammons <ma...@inaf.cnrs-gif.fr>.
I tried this yesterday, and the sandbox is definitely mounted there, but i cannot cd into
it or interact with its contents at all, even as root in this container.
On Monday, August 01, 2016 11:32:07 PM haosdent wrote:
> I think you need du -sh /mnt/mesos/sandbox in the container because it is
> the mounted path of the sandbox in the docker container.
>
> On Mon, Aug 1, 2016 at 8:21 PM, Mark Hammons <mark.hammons@inaf.cnrs-
gif.fr>
> wrote:
> > I'll double check, but last time I du -sh the var folder in the container
> > it was too small to contain the binaries I downloaded.
> >
> > Mark Edgar Hammons II - Research Engineer at BioEmergences
> > 0603695656
> >
> > On 01 Aug 2016, at 03:57, haosdent <ha...@gmail.com> wrote:
> > >First, the binaries I get mesos to download into my sandbox don't appear
> >
> > to be within the docker image
> > The sandbox folder should mount into the docker container with the same
> > path, does this not work for you?
> >
> > >running the executor causes it to complain about libmesos being missing.
> >
> > Yes, you need package the native library(libmesos) which required by
> > executor into your jar or package it into your docker container.
> > Or you could consider implementing your executor base on the new HTTP API
> > (
> > https://github.com/mesosphere/mesos-rxjava)
> >
> > On Sun, Jul 31, 2016 at 9:47 PM, Mark Hammons <
> >
> > mark.hammons@inaf.cnrs-gif.fr> wrote:
> >> Hi all,
> >>
> >>
> >>
> >> So far I've been developing a mesos framework using a custom executor and
> >> scheduler. Custom executors seem to provide a lot of flexibility in
> >> responding to and communicating with the parent mesos-slave, so I'd like
> >> to
> >> keep that around. However, my users would like to have customizable
> >> environments for their algorithms to run in (like maybe one user would
> >> design their application in an ubuntu environment and doesn't want to
> >> take
> >> the time to adapt to the CentOS 7 environment the mesos slave is
> >> running).
> >> So, I'd like to set the sandbox of the executor to be within the context
> >> of
> >> a docker image. I thought doing something like this would work:
> >> http://pastie.org/10924936
> >>
> >>
> >>
> >> but it doesn't for three big reasons. First, the binaries I get mesos to
> >> download into my sandbox don't appear to be within the docker image, and
> >> second the command defined by execCommand is executed at the root of the
> >> docker image instead of within the mesos sandbox. Even when I get around
> >> these issues by adding directives in execCommand to download the missing
> >> binaries into the docker image, running the executor causes it to
> >> complain
> >> about libmesos being missing. Is there any way to do what I'm trying to
> >> do?
> >> Could someone give me an example project that works this way?
> >>
> >>
> >>
> >> Thanks,
> >>
> >> Mark
> >> --
> >>
> >> Mark Hammons - +33 06 03 69 56 56
> >>
> >> Research Engineer @ BioEmergences <http://bioemergences.iscpif.fr>
> >>
> >> Lab Phone: 01 69 82 34 19
> >
> > --
> > Best Regards,
> > Haosdent Huang--
Mark Hammons - +33 06 03 69 56 56
Research Engineer @ BioEmergences[1]
Lab Phone: 01 69 82 34 19
--------
[1] bioemergences.iscpif.fr
Re: Running a mesos executor within a container....
Posted by haosdent <ha...@gmail.com>.
I think you need du -sh /mnt/mesos/sandbox in the container because it is
the mounted path of the sandbox in the docker container.
On Mon, Aug 1, 2016 at 8:21 PM, Mark Hammons <ma...@inaf.cnrs-gif.fr>
wrote:
> I'll double check, but last time I du -sh the var folder in the container
> it was too small to contain the binaries I downloaded.
>
> Mark Edgar Hammons II - Research Engineer at BioEmergences
> 0603695656
>
> On 01 Aug 2016, at 03:57, haosdent <ha...@gmail.com> wrote:
>
> >First, the binaries I get mesos to download into my sandbox don't appear
> to be within the docker image
> The sandbox folder should mount into the docker container with the same
> path, does this not work for you?
>
> >running the executor causes it to complain about libmesos being missing.
> Yes, you need package the native library(libmesos) which required by
> executor into your jar or package it into your docker container.
> Or you could consider implementing your executor base on the new HTTP API (
> https://github.com/mesosphere/mesos-rxjava)
>
> On Sun, Jul 31, 2016 at 9:47 PM, Mark Hammons <
> mark.hammons@inaf.cnrs-gif.fr> wrote:
>
>> Hi all,
>>
>>
>>
>> So far I've been developing a mesos framework using a custom executor and
>> scheduler. Custom executors seem to provide a lot of flexibility in
>> responding to and communicating with the parent mesos-slave, so I'd like to
>> keep that around. However, my users would like to have customizable
>> environments for their algorithms to run in (like maybe one user would
>> design their application in an ubuntu environment and doesn't want to take
>> the time to adapt to the CentOS 7 environment the mesos slave is running).
>> So, I'd like to set the sandbox of the executor to be within the context of
>> a docker image. I thought doing something like this would work:
>> http://pastie.org/10924936
>>
>>
>>
>> but it doesn't for three big reasons. First, the binaries I get mesos to
>> download into my sandbox don't appear to be within the docker image, and
>> second the command defined by execCommand is executed at the root of the
>> docker image instead of within the mesos sandbox. Even when I get around
>> these issues by adding directives in execCommand to download the missing
>> binaries into the docker image, running the executor causes it to complain
>> about libmesos being missing. Is there any way to do what I'm trying to do?
>> Could someone give me an example project that works this way?
>>
>>
>>
>> Thanks,
>>
>> Mark
>> --
>>
>> Mark Hammons - +33 06 03 69 56 56
>>
>> Research Engineer @ BioEmergences <http://bioemergences.iscpif.fr>
>>
>> Lab Phone: 01 69 82 34 19
>>
>
>
>
> --
> Best Regards,
> Haosdent Huang
>
>
--
Best Regards,
Haosdent Huang
Re: Running a mesos executor within a container....
Posted by Mark Hammons <ma...@inaf.cnrs-gif.fr>.
I'll double check, but last time I du -sh the var folder in the container it was too small to contain the binaries I downloaded.
Mark Edgar Hammons II - Research Engineer at BioEmergences
0603695656
> On 01 Aug 2016, at 03:57, haosdent <ha...@gmail.com> wrote:
>
> >First, the binaries I get mesos to download into my sandbox don't appear to be within the docker image
> The sandbox folder should mount into the docker container with the same path, does this not work for you?
>
> >running the executor causes it to complain about libmesos being missing.
> Yes, you need package the native library(libmesos) which required by executor into your jar or package it into your docker container.
> Or you could consider implementing your executor base on the new HTTP API (https://github.com/mesosphere/mesos-rxjava)
>
>> On Sun, Jul 31, 2016 at 9:47 PM, Mark Hammons <ma...@inaf.cnrs-gif.fr> wrote:
>> Hi all,
>>
>> So far I've been developing a mesos framework using a custom executor and scheduler. Custom executors seem to provide a lot of flexibility in responding to and communicating with the parent mesos-slave, so I'd like to keep that around. However, my users would like to have customizable environments for their algorithms to run in (like maybe one user would design their application in an ubuntu environment and doesn't want to take the time to adapt to the CentOS 7 environment the mesos slave is running). So, I'd like to set the sandbox of the executor to be within the context of a docker image. I thought doing something like this would work: http://pastie.org/10924936
>>
>> but it doesn't for three big reasons. First, the binaries I get mesos to download into my sandbox don't appear to be within the docker image, and second the command defined by execCommand is executed at the root of the docker image instead of within the mesos sandbox. Even when I get around these issues by adding directives in execCommand to download the missing binaries into the docker image, running the executor causes it to complain about libmesos being missing. Is there any way to do what I'm trying to do? Could someone give me an example project that works this way?
>>
>> Thanks,
>> Mark
>> --
>> Mark Hammons - +33 06 03 69 56 56
>> Research Engineer @ BioEmergences
>> Lab Phone: 01 69 82 34 19
>
>
>
> --
> Best Regards,
> Haosdent Huang