You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Tom Arnfeld <to...@duedil.com> on 2014/02/18 13:10:47 UTC
Docker Pluggable Containerizer
Hey,
I’ve been working the past few days on building a Pluggable Containerizer for Docker (I am aware the guys at Mesosphere are already doing this, but i’ve not seen anything shared and wanted to get started on testing mesos as I intend to use it). I’d very much appreciate if anyone involved in the new external/pluggable containerizer feature could take a few minutes to have a look and/or test it out!
It’s on github with a getting-started readme: http://tfld.me/3m2m1v322w0y
I’ve implemented it pretty blindly and the example containerizers don’t help too much as they’re pretty basic. Some documentation on what all of the different commands mean would go a long way with others users that want to implement their own containerizers, i’m sure.
There are a few outstanding bits worth noting;
- The `usage` stats don’t reflect the usage of the container
- I’m not sure how this works with custom executors hosted on hdfs:// or s3n:// (needs testing and documenting)
- The `recover` method of the containerizer isn’t implemented
I’m not sure what this is for exactly… since the comments/docs are pretty sparse
- The resource limits are not enforced on the container
Tom.
—
Tom Arnfeld
Developer // DueDil
Re: Docker Pluggable Containerizer
Posted by Tom Arnfeld <to...@duedil.com>.
Thanks Jason – I was aware you were working on something but not that you’d released an early version yet. I started mine simply as I needed to get a sense of how i’d actually be using Mesos.
It looks like there’s a nice overlap of the two projects, i’ve “solved” (to the best of my understanding) the stdout/stderr integration, but a mount is interesting, i’ve taken a different approach. Nice idea by bundling the mesos.proto inside… though that means the containerizer is directly tied to a mesos version (though arguably that’s inferred by the implementation).
I’m very keen to see how you’re going to implement `usage`. I’ll no doubt be scrapping my code in favour of yours once it’s stable.
—
Tom Arnfeld
Developer // DueDil
On 18 Feb 2014, at 18:06, Jason Dusek <ja...@gmail.com> wrote:
> 2014-02-18 12:10 GMT+00:00 Tom Arnfeld <to...@duedil.com>:
>> I’ve been working the past few days on building a Pluggable
>> Containerizer for Docker (I am aware the guys at Mesosphere
>> are already doing this, but i’ve not seen anything shared and
>> wanted to get started on testing mesos as I intend to use it).
>> I’d very much appreciate if anyone involved in the new
>> external/pluggable containerizer feature could take a few
>> minutes to have a look and/or test it out!
>>
>> It’s on github with a getting-started readme:
>> http://tfld.me/3m2m1v322w0y
>>
>> I’ve implemented it pretty blindly and the example
>> containerizers don’t help too much as they’re pretty basic.
>> Some documentation on what all of the different commands mean
>> would go a long way with others users that want to implement
>> their own containerizers, i’m sure.
>>
>> There are a few outstanding bits worth noting;
>>
>> - The `usage` stats don’t reflect the usage of the container
>> - I’m not sure how this works with custom executors hosted on hdfs:// or s3n:// (needs testing and documenting)
>> - The `recover` method of the containerizer isn’t implemented
>> I’m not sure what this is for exactly… since the comments/docs are pretty sparse
>> - The resource limits are not enforced on the container
>
> We're opening up the code that we have today.
>
> https://github.com/mesosphere/medea
>
> There are some things we do already -- like resource limitations
> -- but we haven't finished standard error/out integration or
> filesystem browsing (probably to be handled via a read-only
> mount in to the sandbox).
>
> Medea is different in that it is packaged with the mesos.proto
> file and intended to be installable with setuptools/distutils.
> As the interface between the containerizer and Mesos is all
> protos over standard in/out, it is feasible to ship it as a true
> Python package.
>
> Please look forward to an implementation of `usage` later today.
>
> --
> Jason Dusek
> pgp // solidsnack // C1EBC57DC55144F35460C8DF1FD4C6C1FED18A2B
Re: Docker Pluggable Containerizer
Posted by Jason Dusek <ja...@gmail.com>.
2014-02-18 12:10 GMT+00:00 Tom Arnfeld <to...@duedil.com>:
> I’ve been working the past few days on building a Pluggable
> Containerizer for Docker (I am aware the guys at Mesosphere
> are already doing this, but i’ve not seen anything shared and
> wanted to get started on testing mesos as I intend to use it).
> I’d very much appreciate if anyone involved in the new
> external/pluggable containerizer feature could take a few
> minutes to have a look and/or test it out!
>
> It’s on github with a getting-started readme:
> http://tfld.me/3m2m1v322w0y
>
> I’ve implemented it pretty blindly and the example
> containerizers don’t help too much as they’re pretty basic.
> Some documentation on what all of the different commands mean
> would go a long way with others users that want to implement
> their own containerizers, i’m sure.
>
> There are a few outstanding bits worth noting;
>
> - The `usage` stats don’t reflect the usage of the container
> - I’m not sure how this works with custom executors hosted on hdfs:// or s3n:// (needs testing and documenting)
> - The `recover` method of the containerizer isn’t implemented
> I’m not sure what this is for exactly… since the comments/docs are pretty sparse
> - The resource limits are not enforced on the container
We're opening up the code that we have today.
https://github.com/mesosphere/medea
There are some things we do already -- like resource limitations
-- but we haven't finished standard error/out integration or
filesystem browsing (probably to be handled via a read-only
mount in to the sandbox).
Medea is different in that it is packaged with the mesos.proto
file and intended to be installable with setuptools/distutils.
As the interface between the containerizer and Mesos is all
protos over standard in/out, it is feasible to ship it as a true
Python package.
Please look forward to an implementation of `usage` later today.
--
Jason Dusek
pgp // solidsnack // C1EBC57DC55144F35460C8DF1FD4C6C1FED18A2B