[jira] [Commented] (MESOS-6360) The handling of whiteout files in provisioner is not correct

["Qian Zhang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MESOS-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15568646#comment-15568646 ] 

Qian Zhang commented on MESOS-6360:
-----------------------------------

Here is a specific issue that I found for the current implementation of whiteout handling. Suppose I build a Docker image from the following Dockerfile:
{code}
FROM cirros
RUN touch /opt/data
RUN rm -rf /opt/data
RUN echo yes > /opt/data
{code}

And I launch a container from that image via Mesos using overlay provisioner backend, then I found there is NO {{data}} file under {{/opt}} in the container, but if I launch a Docker container from that image via Docker engine, then I found there is the {{data}} file under {{/opt}} in the container and its content is {{yes}}. Obviously the result of Docker container is correct.

The root cause of this issue is the current way that we handle the whiteout file generated by the line 3 in the above Dockerfile is not correct: we always handle the whiteout files after the whole rootfs for the container has been provisioned, but that will lead to wrongly deleting a file which should not be deleted.

> The handling of whiteout files in provisioner is not correct
> ------------------------------------------------------------
>
>                 Key: MESOS-6360
>                 URL: https://issues.apache.org/jira/browse/MESOS-6360
>             Project: Mesos
>          Issue Type: Bug
>            Reporter: Qian Zhang
>            Assignee: Qian Zhang
>            Priority: Blocker
>
> Currently when user launches a container from a Docker image via universal containerizer, we always handle the whiteout files in {{ProvisionerProcess::__provision()}} regardless of which backend is used.
> However this is actually not correct, because the way to handle whiteout files is backend dependent, that means for different backends, we need to handle whiteout files in different ways, e.g.:
> * AUFS backend: It seems the AUFS whiteout ({{.wh.<filename/dirname>}} and {{.wh..wh..opq}}) is the whiteout standard in Docker (see [this comment | https://github.com/docker/docker/blob/v1.12.1/pkg/archive/archive.go#L259:L262] for details), so that means after the Docker image is pulled, its whiteout files in the store are already in aufs format, then we do not need to do anything about whiteout file handling because the aufs mount done in {{AufsBackendProcess::provision()}} will handle it automatically.
> * Overlay backend: Overlayfs has its own whiteout files (see [this doc | https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt] for details), so we need to convert the aufs whiteout files to overlayfs whiteout files before we do the overlay mount in {{OverlayBackendProcess::provision}} which will automatically handle the overlayfs whiteout files.
> * Copy backend: We need to manually handle the aufs whiteout files when we copy each layer in {{CopyBackendProcess::_provision()}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)