[jira] [Issue Comment Deleted] (MESOS-6360) The handling of whiteout files in provisioner is not correct

["Jie Yu (JIRA)" <ji...@apache.org>]

     [ https://issues.apache.org/jira/browse/MESOS-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jie Yu updated MESOS-6360:
--------------------------
    Comment: was deleted

(was: commit 3f6503861e92896a619e3b26a65f9b679a2dd3a9
Author: Qian Zhang <zh...@cn.ibm.com>
Date:   Fri Oct 14 11:41:02 2016 -0700

    Added backend suffix to image layer rootfs path.
    
    Previously image layer rootfs path is in the format below regardless
    of which backend is used.
      <image_store_dir>/layers/<layer_id>/rootfs
    This introduced an issue: when agent is restarted with a different
    backend, we will wrongly handle the whiteout files since different
    backends(e.g., aufs and overlay) have different whiteout standard.
    
    In this commit, we added backend suffix to image layer rootfs path
    for overlay backend like below.
      <image_store_dir>/layers/<layer_id>/rootfs.overlay
    For non-overlay backends, it is still in the previous format, this
    is because they share the same whiteout standard (aufs standard),
    and also used to handle backward compatibility.
    
    So the expected result of this commit is:
    1. If user switches backend from overlay to a non-overlay (or vice
    versa) when restarting agent, all the image layers of the previous
    backend in the store will just be ignored.
    2. In the upgrade case, if user starts a new version of Mesos agent
    with overlay backend, then all the image layers in the store pulled
    by the old agent will just be ignored, but if user starts the new
    agent with a non-overlay backend, then all the image layers in the
    store can still be used.
    
    Review: https://reviews.apache.org/r/52827/)

> The handling of whiteout files in provisioner is not correct
> ------------------------------------------------------------
>
>                 Key: MESOS-6360
>                 URL: https://issues.apache.org/jira/browse/MESOS-6360
>             Project: Mesos
>          Issue Type: Bug
>            Reporter: Qian Zhang
>            Assignee: Qian Zhang
>            Priority: Blocker
>
> Currently when user launches a container from a Docker image via universal containerizer, we always handle the whiteout files in {{ProvisionerProcess::__provision()}} regardless of which backend is used.
> However this is actually not correct, because the way to handle whiteout files is backend dependent, that means for different backends, we need to handle whiteout files in different ways, e.g.:
> * AUFS backend: It seems the AUFS whiteout ({{.wh.<filename/dirname>}} and {{.wh..wh..opq}}) is the whiteout standard in Docker (see [this comment | https://github.com/docker/docker/blob/v1.12.1/pkg/archive/archive.go#L259:L262] for details), so that means after the Docker image is pulled, its whiteout files in the store are already in aufs format, then we do not need to do anything about whiteout file handling because the aufs mount done in {{AufsBackendProcess::provision()}} will handle it automatically.
> * Overlay backend: Overlayfs has its own whiteout files (see [this doc | https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt] for details), so we need to convert the aufs whiteout files to overlayfs whiteout files before we do the overlay mount in {{OverlayBackendProcess::provision}} which will automatically handle the overlayfs whiteout files.
> * Copy backend: We need to manually handle the aufs whiteout files when we copy each layer in {{CopyBackendProcess::_provision()}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)