You are viewing a plain text version of this content. The canonical link for it is here.
Posted to dev@mesos.apache.org by Tom Arnfeld <to...@duedil.com> on 2014/02/18 13:10:47 UTC

Docker Pluggable Containerizer

Hey,

I’ve been working the past few days on building a Pluggable Containerizer for Docker (I am aware the guys at Mesosphere are already doing this, but i’ve not seen anything shared and wanted to get started on testing mesos as I intend to use it). I’d very much appreciate if anyone involved in the new external/pluggable containerizer feature could take a few minutes to have a look and/or test it out!

It’s on github with a getting-started readme: http://tfld.me/3m2m1v322w0y

I’ve implemented it pretty blindly and the example containerizers don’t help too much as they’re pretty basic. Some documentation on what all of the different commands mean would go a long way with others users that want to implement their own containerizers, i’m sure.

There are a few outstanding bits worth noting;

- The `usage` stats don’t reflect the usage of the container
- I’m not sure how this works with custom executors hosted on hdfs:// or s3n:// (needs testing and documenting)
- The `recover` method of the containerizer isn’t implemented
	I’m not sure what this is for exactly… since the comments/docs are pretty sparse
- The resource limits are not enforced on the container

Tom.

—

Tom Arnfeld
Developer // DueDil


Re: Docker Pluggable Containerizer

Posted by Tom Arnfeld <to...@duedil.com>.
Thanks Jason – I was aware you were working on something but not that you’d released an early version yet. I started mine simply as I needed to get a sense of how i’d actually be using Mesos.

It looks like there’s a nice overlap of the two projects, i’ve “solved” (to the best of my understanding) the stdout/stderr integration, but a mount is interesting, i’ve taken a different approach. Nice idea by bundling the mesos.proto inside… though that means the containerizer is directly tied to a mesos version (though arguably that’s inferred by the implementation).

I’m very keen to see how you’re going to implement `usage`. I’ll no doubt be scrapping my code in favour of yours once it’s stable.

—

Tom Arnfeld
Developer // DueDil

On 18 Feb 2014, at 18:06, Jason Dusek <ja...@gmail.com> wrote:

> 2014-02-18 12:10 GMT+00:00 Tom Arnfeld <to...@duedil.com>:
>> I’ve been working the past few days on building a Pluggable
>> Containerizer for Docker (I am aware the guys at Mesosphere
>> are already doing this, but i’ve not seen anything shared and
>> wanted to get started on testing mesos as I intend to use it).
>> I’d very much appreciate if anyone involved in the new
>> external/pluggable containerizer feature could take a few
>> minutes to have a look and/or test it out!
>> 
>> It’s on github with a getting-started readme:
>> http://tfld.me/3m2m1v322w0y
>> 
>> I’ve implemented it pretty blindly and the example
>> containerizers don’t help too much as they’re pretty basic.
>> Some documentation on what all of the different commands mean
>> would go a long way with others users that want to implement
>> their own containerizers, i’m sure.
>> 
>> There are a few outstanding bits worth noting;
>> 
>> - The `usage` stats don’t reflect the usage of the container
>> - I’m not sure how this works with custom executors hosted on hdfs:// or s3n:// (needs testing and documenting)
>> - The `recover` method of the containerizer isn’t implemented
>>  I’m not sure what this is for exactly… since the comments/docs are pretty sparse
>> - The resource limits are not enforced on the container
> 
> We're opening up the code that we have today.
> 
>  https://github.com/mesosphere/medea
> 
> There are some things we do already -- like resource limitations
> -- but we haven't finished standard error/out integration or
> filesystem browsing (probably to be handled via a read-only
> mount in to the sandbox).
> 
> Medea is different in that it is packaged with the mesos.proto
> file and intended to be installable with setuptools/distutils.
> As the interface between the containerizer and Mesos is all
> protos over standard in/out, it is feasible to ship it as a true
> Python package.
> 
> Please look forward to an implementation of `usage` later today.
> 
> --
> Jason Dusek
> pgp // solidsnack // C1EBC57DC55144F35460C8DF1FD4C6C1FED18A2B


Re: Docker Pluggable Containerizer

Posted by Jason Dusek <ja...@gmail.com>.
2014-02-18 12:10 GMT+00:00 Tom Arnfeld <to...@duedil.com>:
> I’ve been working the past few days on building a Pluggable
> Containerizer for Docker (I am aware the guys at Mesosphere
> are already doing this, but i’ve not seen anything shared and
> wanted to get started on testing mesos as I intend to use it).
> I’d very much appreciate if anyone involved in the new
> external/pluggable containerizer feature could take a few
> minutes to have a look and/or test it out!
>
> It’s on github with a getting-started readme:
> http://tfld.me/3m2m1v322w0y
>
> I’ve implemented it pretty blindly and the example
> containerizers don’t help too much as they’re pretty basic.
> Some documentation on what all of the different commands mean
> would go a long way with others users that want to implement
> their own containerizers, i’m sure.
>
> There are a few outstanding bits worth noting;
>
> - The `usage` stats don’t reflect the usage of the container
> - I’m not sure how this works with custom executors hosted on hdfs:// or s3n:// (needs testing and documenting)
> - The `recover` method of the containerizer isn’t implemented
>   I’m not sure what this is for exactly… since the comments/docs are pretty sparse
> - The resource limits are not enforced on the container

We're opening up the code that we have today.

  https://github.com/mesosphere/medea

There are some things we do already -- like resource limitations
-- but we haven't finished standard error/out integration or
filesystem browsing (probably to be handled via a read-only
mount in to the sandbox).

Medea is different in that it is packaged with the mesos.proto
file and intended to be installable with setuptools/distutils.
As the interface between the containerizer and Mesos is all
protos over standard in/out, it is feasible to ship it as a true
Python package.

Please look forward to an implementation of `usage` later today.

--
Jason Dusek
pgp // solidsnack // C1EBC57DC55144F35460C8DF1FD4C6C1FED18A2B