You are viewing a plain text version of this content. The canonical link for it is here.
Posted to reviews@mesos.apache.org by James Peach <jp...@apache.org> on 2018/12/07 19:41:31 UTC

Review Request 69528: Fixed a regression in binding GPU container devices.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69528/
-----------------------------------------------------------

Review request for mesos, Jie Yu and Jiang Yan Xu.


Bugs: MESOS-9319
    https://issues.apache.org/jira/browse/MESOS-9319


Repository: mesos


Description
-------

When we changed container devices to be bind mounts, we added an extra
`/dev` path component to the container moount point. This resulted in
devices being mounted as `/dev/dev/nvidia0`.


Diffs
-----

  src/slave/containerizer/mesos/isolators/gpu/isolator.cpp 14871287db8be96b3bb4d655caf43f92f0dbce13 


Diff: https://reviews.apache.org/r/69528/diff/1/


Testing
-------

make check (Fedora 29)

Tested manually on a GPU host.

Before:
```
|-/dev                     tmpfs
| |-/dev/pts               devpts
| |-/dev/shm               tmpfs
| |-/dev/full              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/full]
| |-/dev/null              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/null]
| |-/dev/random            /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/random]
| |-/dev/tty               /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/tty]
| |-/dev/urandom           /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/urandom]
| |-/dev/zero              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/zero]
| |-/dev/fuse              devtmpfs[/fuse]
| |-/dev/dev/nvidia-uvm    /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/nvidia-uvm/dev/nvidia-uvm]
| |-/dev/dev/nvidiactl     /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/nvidiactl/dev/nvidiactl]
| |-/dev/dev/nvidia1       /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/nvidia1/dev/nvidia1]
| `-/dev/dev/nvidia0       /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/nvidia0/dev/nvidia0]
```

After:
```
|-/dev                     tmpfs
| |-/dev/pts               devpts
| |-/dev/shm               tmpfs
| |-/dev/full              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/full]
| |-/dev/null              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/null]
| |-/dev/random            /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/random]
| |-/dev/tty               /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/tty]
| |-/dev/urandom           /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/urandom]
| |-/dev/zero              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/zero]
| |-/dev/fuse              devtmpfs[/fuse]
| |-/dev/nvidia-uvm        /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/nvidia-uvm/dev/nvidia-uvm]
| |-/dev/nvidiactl         /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/nvidiactl/dev/nvidiactl]
| |-/dev/nvidia1           /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/nvidia1/dev/nvidia1]
| `-/dev/nvidia0           /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/nvidia0/dev/nvidia0]
```


Thanks,

James Peach


Re: Review Request 69528: Fixed a regression in binding GPU container devices.

Posted by Mesos Reviewbot Windows <re...@mesos.apache.org>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69528/#review211121
-----------------------------------------------------------



PASS: Mesos patch 69528 was successfully built and tested.

Reviews applied: `['69528']`

All the build artifacts available at: http://dcos-win.westus2.cloudapp.azure.com/artifacts/mesos-reviewbot-testing/2682/mesos-review-69528

- Mesos Reviewbot Windows


On Dec. 7, 2018, 11:41 a.m., James Peach wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69528/
> -----------------------------------------------------------
> 
> (Updated Dec. 7, 2018, 11:41 a.m.)
> 
> 
> Review request for mesos, Jie Yu and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-9319
>     https://issues.apache.org/jira/browse/MESOS-9319
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> When we changed container devices to be bind mounts, we added an extra
> `/dev` path component to the container moount point. This resulted in
> devices being mounted as `/dev/dev/nvidia0`.
> 
> 
> Diffs
> -----
> 
>   src/slave/containerizer/mesos/isolators/gpu/isolator.cpp 14871287db8be96b3bb4d655caf43f92f0dbce13 
> 
> 
> Diff: https://reviews.apache.org/r/69528/diff/1/
> 
> 
> Testing
> -------
> 
> make check (Fedora 29)
> 
> Tested manually on a GPU host.
> 
> Before:
> ```
> |-/dev                     tmpfs
> | |-/dev/pts               devpts
> | |-/dev/shm               tmpfs
> | |-/dev/full              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/full]
> | |-/dev/null              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/null]
> | |-/dev/random            /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/random]
> | |-/dev/tty               /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/tty]
> | |-/dev/urandom           /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/urandom]
> | |-/dev/zero              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/zero]
> | |-/dev/fuse              devtmpfs[/fuse]
> | |-/dev/dev/nvidia-uvm    /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/nvidia-uvm/dev/nvidia-uvm]
> | |-/dev/dev/nvidiactl     /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/nvidiactl/dev/nvidiactl]
> | |-/dev/dev/nvidia1       /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/nvidia1/dev/nvidia1]
> | `-/dev/dev/nvidia0       /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/nvidia0/dev/nvidia0]
> ```
> 
> After:
> ```
> |-/dev                     tmpfs
> | |-/dev/pts               devpts
> | |-/dev/shm               tmpfs
> | |-/dev/full              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/full]
> | |-/dev/null              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/null]
> | |-/dev/random            /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/random]
> | |-/dev/tty               /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/tty]
> | |-/dev/urandom           /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/urandom]
> | |-/dev/zero              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/zero]
> | |-/dev/fuse              devtmpfs[/fuse]
> | |-/dev/nvidia-uvm        /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/nvidia-uvm/dev/nvidia-uvm]
> | |-/dev/nvidiactl         /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/nvidiactl/dev/nvidiactl]
> | |-/dev/nvidia1           /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/nvidia1/dev/nvidia1]
> | `-/dev/nvidia0           /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/nvidia0/dev/nvidia0]
> ```
> 
> 
> Thanks,
> 
> James Peach
> 
>


Re: Review Request 69528: Fixed a regression in binding GPU container devices.

Posted by Jie Yu <yu...@gmail.com>.
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/69528/#review211120
-----------------------------------------------------------


Ship it!




Ship It!

- Jie Yu


On Dec. 7, 2018, 7:41 p.m., James Peach wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/69528/
> -----------------------------------------------------------
> 
> (Updated Dec. 7, 2018, 7:41 p.m.)
> 
> 
> Review request for mesos, Jie Yu and Jiang Yan Xu.
> 
> 
> Bugs: MESOS-9319
>     https://issues.apache.org/jira/browse/MESOS-9319
> 
> 
> Repository: mesos
> 
> 
> Description
> -------
> 
> When we changed container devices to be bind mounts, we added an extra
> `/dev` path component to the container moount point. This resulted in
> devices being mounted as `/dev/dev/nvidia0`.
> 
> 
> Diffs
> -----
> 
>   src/slave/containerizer/mesos/isolators/gpu/isolator.cpp 14871287db8be96b3bb4d655caf43f92f0dbce13 
> 
> 
> Diff: https://reviews.apache.org/r/69528/diff/1/
> 
> 
> Testing
> -------
> 
> make check (Fedora 29)
> 
> Tested manually on a GPU host.
> 
> Before:
> ```
> |-/dev                     tmpfs
> | |-/dev/pts               devpts
> | |-/dev/shm               tmpfs
> | |-/dev/full              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/full]
> | |-/dev/null              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/null]
> | |-/dev/random            /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/random]
> | |-/dev/tty               /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/tty]
> | |-/dev/urandom           /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/urandom]
> | |-/dev/zero              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/zero]
> | |-/dev/fuse              devtmpfs[/fuse]
> | |-/dev/dev/nvidia-uvm    /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/nvidia-uvm/dev/nvidia-uvm]
> | |-/dev/dev/nvidiactl     /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/nvidiactl/dev/nvidiactl]
> | |-/dev/dev/nvidia1       /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/nvidia1/dev/nvidia1]
> | `-/dev/dev/nvidia0       /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/28f9b34e-9cd4-4b1a-8bb1-237743e4a483/devices/nvidia0/dev/nvidia0]
> ```
> 
> After:
> ```
> |-/dev                     tmpfs
> | |-/dev/pts               devpts
> | |-/dev/shm               tmpfs
> | |-/dev/full              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/full]
> | |-/dev/null              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/null]
> | |-/dev/random            /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/random]
> | |-/dev/tty               /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/tty]
> | |-/dev/urandom           /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/urandom]
> | |-/dev/zero              /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/zero]
> | |-/dev/fuse              devtmpfs[/fuse]
> | |-/dev/nvidia-uvm        /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/nvidia-uvm/dev/nvidia-uvm]
> | |-/dev/nvidiactl         /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/nvidiactl/dev/nvidiactl]
> | |-/dev/nvidia1           /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/nvidia1/dev/nvidia1]
> | `-/dev/nvidia0           /dev/mapper/rootvg-rootlv[/var/run/mesos/containers/450a0823-78bf-4556-9221-eb1d569ee097/devices/nvidia0/dev/nvidia0]
> ```
> 
> 
> Thanks,
> 
> James Peach
> 
>