Disk quota enforcement with docker

[David Kesler <DK...@yodle.com>]

I've been playing around with the new disk isolation feature in mesos 0.22.  It appears that it only applies to files written to the task's mesos sanbox, so it won't apply to docker containers directly.  (Correct me if I'm wrong on that).  However I was hoping I'd be able to at least use it to enforce quotas if the app within the docker container was well behaved and used the mounted mesos sandbox as it's scratch space.  Unfortunately it doesn't seem to be working as I'd expect.  My setup is as follows:

Mesos slaves running with -containerizers=docker,mesos, --enforce_container_disk_quota, and -isolation=posix/disk.
Verified quota is working by running `mesos-execute --resources="cpus:.1;mem:100;disk:10" --command="dd bs=1024 count=3276 if=/dev/zero of=foo; sleep 70" --master=<master ip> --name=test` which  succeeded and `mesos-execute --resources="cpus:.1;mem:100;disk:1" --command="dd bs=1024 count=3276 if=/dev/zero of=foo; sleep 70" --master=10.100.24.56:5050 --name=test` which failed
Deployed an app in marathon, setting the disk quota to 1
On a docker slave running the app, `docker exec`d into the container, navigated to /mnt/mesos/sandbox and wrote a 3MB file

The task corresponding to the container continues to run.  Viewing the executor in the mesos master ui shows the executor using and allocated 0 disk.  The written file shows up in the task's sandbox though.

My only guess to the problem is that since the app is running via docker containerization, mesos is deferring entirely to docker for isolation and so mesos's disk isolation doesn't even try to run (either to report usage or to enforce quotas)?  Does this make sense?  And is there any other way to do this, or will we need to wait for explicit support (i.e. https://issues.apache.org/jira/browse/MESOS-2502)?

Re: Disk quota enforcement with docker

[Vinod Kone <vi...@apache.org>]

Disk quota is only enabled for Mesos containerizer. You have to wait for
MESOS-2502 for quota support for Docker containerizer.

On Thu, May 7, 2015 at 2:27 PM, David Kesler <DK...@yodle.com> wrote:

>  I’ve been playing around with the new disk isolation feature in mesos
> 0.22.  It appears that it only applies to files written to the task’s mesos
> sanbox, so it won’t apply to docker containers directly.  (Correct me if
> I’m wrong on that).  However I was hoping I’d be able to at least use it to
> enforce quotas if the app within the docker container was well behaved and
> used the mounted mesos sandbox as it’s scratch space.  Unfortunately it
> doesn’t seem to be working as I’d expect.  My setup is as follows:
>
>
>
> Mesos slaves running with –containerizers=docker,mesos,
> --enforce_container_disk_quota, and –isolation=posix/disk.
>
> Verified quota is working by running `mesos-execute
> --resources="cpus:.1;mem:100;disk:10" --command="dd bs=1024 count=3276
> if=/dev/zero of=foo; sleep 70" --master=<master ip> --name=test` which
>  succeeded and `mesos-execute --resources="cpus:.1;mem:100;disk:1"
> --command="dd bs=1024 count=3276 if=/dev/zero of=foo; sleep 70" --master=
> 10.100.24.56:5050 --name=test` which failed
>
> Deployed an app in marathon, setting the disk quota to 1
>
> On a docker slave running the app, `docker exec`d into the container,
> navigated to /mnt/mesos/sandbox and wrote a 3MB file
>
>
>
> The task corresponding to the container continues to run.  Viewing the
> executor in the mesos master ui shows the executor using and allocated 0
> disk.  The written file shows up in the task’s sandbox though.
>
>
>
> My only guess to the problem is that since the app is running via docker
> containerization, mesos is deferring entirely to docker for isolation and
> so mesos’s disk isolation doesn’t even try to run (either to report usage
> or to enforce quotas)?  Does this make sense?  And is there any other way
> to do this, or will we need to wait for explicit support (i.e.
> https://issues.apache.org/jira/browse/MESOS-2502)?
>