[GitHub] [beam] tvalentyn commented on pull request #16658: [BEAM-12792] Install pipline dependencies to temporary venv

[GitBox <gi...@apache.org>]

tvalentyn commented on PR #16658:
URL: https://github.com/apache/beam/pull/16658#issuecomment-1093395167

   I spent some time debugging this today and got this far:
   
   Dataflow container pod for SDK container has the following mount (from container manifest):
   
   ```
   "volumeMounts": [ {
           "mountPath": "/var/opt/google",
           "name": "persist"
         }, 
   
   ...
   "volumes": [ {
         "hostPath": {
           "path": "/var/opt/google/dataflow"
         },
         "name": "persist"
       }
   
   ```
   
   The above maps a persistent directory `/var/opt/google/dataflow` on the host VM into a directory `/var/opt/google` in the running containers. This directory is passed to sdk harness container in 	`--semi_persist_dir=/var/opt/google` param.
   
   We can see this directory referenced in the errors:
   
   ```
   "2022/04/08 20:40:15 Failed to install required packages: failed to install SDK: fork/exec /var/opt/google/beam-venv/beam-venv-worker-sdk-0-0/bin/pip: permission denied
   ```
   
   I can reproduce this behavior if I manually SSH into a VM created by Dataflow, and manually start a container via:
   
   docker run -it --entrypoint=/bin/bash -v /var/opt/google/dataflow:/var/opt/google apache/beam_python3.7_sdk
   
   ```
   root@beamapp-valentyn-04082037-04081337-w2xj-harness-zq0w:/# python -m venv /var/opt/google/env
   root@beamapp-valentyn-04082037-04081337-w2xj-harness-zq0w:/# /var/opt/google/env/bin/pip
   bash: /var/opt/google/env/bin/pip: Permission denied
   ```
   
   On the other hand:
   ```
   root@beamapp-valentyn-04082037-04081337-w2xj-harness-zq0w:/# python -m venv /var/opt/env
   root@beamapp-valentyn-04082037-04081337-w2xj-harness-zq0w:/# /var/opt/env/bin/pip
   
   Usage:   
     pip <command> [options]
   
   ```
   
   Permissions on the files are identical:
   
   ```
   root@beamapp-valentyn-04082037-04081337-w2xj-harness-zq0w:/# ls -al /var/opt/env/bin/pip
   -rwxr-xr-x 1 root root 228 Apr  8 21:29 /var/opt/env/bin/pip
   root@beamapp-valentyn-04082037-04081337-w2xj-harness-zq0w:/# ls -al /var/opt/google/env/bin/pip
   -rwxr-xr-x 1 root root 235 Apr  8 21:29 /var/opt/google/env/bin/pip
   ```
   
   but for some reasons, when we create a venv in the mounted directory, the `pip` script cannot launch. I am not sure what is going on. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: github-unsubscribe@beam.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org