[jira] [Commented] (MESOS-8279) Persistent volumes are not visible in Mesos UI using default executor on Linux.

["Qian Zhang (JIRA)" <ji...@apache.org>]

    [ https://issues.apache.org/jira/browse/MESOS-8279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16293797#comment-16293797 ] 

Qian Zhang commented on MESOS-8279:
-----------------------------------

The root cause of this issue is, the PV directory we show in Mesos UI for the nested container is a mount point created in {{volume/sandbox_path}} isolator, but we only mount the PV in the nested container’s mount namespace but not in the host mount namespace, so that mount point is just an empty directory in host mount namespace, that’s why we have this issue.

The symlink solution mentioned in the description may not work, because the target of the symlink (like {{/opt/mesos/slaves/f61b06bb-3573-4bbd-91c7-512542f644eb-S0/frameworks/c1a0c615-dbcd-4de8-8e9f-cb38fb28a10c-0000/executors/default-executor/runs/82b3a060-0708-4d8a-9f0b-d314446470bb/<pv>}}) which is created by the {{filesystem/linux}} isolator for the executor container will not be visible in the nested container’s mount namespace after we do the {{chroot}} for the nested container. As long as we do the {{chroot}}, that symlink will be a dead link which means the nested container can not access the PV directory. So this solution will only work for the nested container which has no image (i.e., no need {{chroot}}).

I was also thinking about a solution with bind mount instead of symlink: In {{volume/sandbox_path}} isolator, right after we call {{mkdir}} to create the mount point for the nested container, we can bind mount the executor container's PV directory to that mount point so that it will be visible in host mount namespace, and when the nested container is launched, that mount will be inherited, so the PV will be visible in the nested container’s mount namespace as well. But the problem with this solution is the host mount namespace will be polluted.

We were also thinking about a workaround in Mesos UI code to redirect the nested container’s PV directory (but not any other directories/files in its sandbox) to the executor container’s PV directory, but that can not work too, because we need to let the UI know which one in the nested container’s sandbox is the PV dir. But such info is put into {{LaunchNestedContainer.container.volumes}} by the default executor as a workaround, so UI can not know it from agent’s {{/state}} endpoint.

> Persistent volumes are not visible in Mesos UI using default executor on Linux.
> -------------------------------------------------------------------------------
>
>                 Key: MESOS-8279
>                 URL: https://issues.apache.org/jira/browse/MESOS-8279
>             Project: Mesos
>          Issue Type: Bug
>    Affects Versions: 1.2.2, 1.3.1, 1.4.1
>            Reporter: Jie Yu
>
> The reason is because on Linux, if multiple containers in a default executor want to share a persistent volume, it'll use SANDBOX_PATH volume source with type PARENT. This will be translated into a bind mount in the nested container's mount namespace, thus not visible in the host mount namespace. Mesos UI operates in the host mount namespace.
> One potential solution for that is to create a symlink (instead of just a mkdir) in the sandbox. The symlink will be shadowed by the bind mount in the nested container, but in the host mount namespace, it'll points to the corresponding persistent volume.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)