[GitHub] [airflow] jason-brian-anderson commented on issue #30554: Unable to access host bind mounts as root within DockerOperator-spawned container from Windowd Docker Desktop 4.18.0

["jason-brian-anderson (via GitHub)" <gi...@apache.org>]

jason-brian-anderson commented on issue #30554:
URL: https://github.com/apache/airflow/issues/30554#issuecomment-1502031635

   Thanks for the fast reply @potiuk ,
   
   I agree this does not seem to be an Airflow issue. Your feedback did give me hints that made the solution obvious;  but it doesn't seem that docker-in-docker host bind mounts are impossible on Linux after all, as described below:
   
   ## Root cause:
   Wsl2 does not explicitly allow host bind mounts for docker-in-docker usage. There is a docker desktop file sharing configuration that must be set.  The problem is that this necessary configuration interface is not present in the Docker Desktop UI (at least on v4.18.0)  Settings->Resources tab **unless** you turn off wsl2 engine support under Settings->General.  But to do this, you must install Hyper-V under “Turn Windows Features on and off”.   Once you have restarted in Hyper-V Docker Desktop mode, you will then have the ‘File Sharing’ tab within Settings->Resources where you can set your development directory and subdirectories for mounting.  Once you do this you can stay in Hyper-V mode as long as you don’t need to use the GPU, which as of this writing Hyper-V does not support, but you **can** go back to WSL2 mode and the file sharing settings only available in hyper-V mode  will stick.  If you ever need to edit filesharing, you’ll have to switch back temporarily to
  Hyper-V mode. 
   
   _Note that GPU training support is not currently available in Hyper-V, so plan to live permanently in WSL2_
   
   
   
   ## Solution:
   Installed and switch docker temporarily over to hyper-v.  
   Specifically, 
   i implemented hyper-v in windows features
   I unchecked Use the WSL 2 based engine
   This allowed access to a ‘file sharing’ interface in Resources that was not previously available.
   In it, i put my dev directory and restarted the airflow app
   Root was able to access the host mounted file system
   I then re-checked ‘Use the WSL 2 based engine’ and root was still able to access the host mounted fs.  
   solved.
   
   
   
   ## Details
   Some details not included above - Once hyperv was installed, rebooted, and wsl 2 engine was unchecked, and the host mounts became available,  GPU support was no longer available. I found from here that it is not supported:
   https://learn.microsoft.com/en-us/virtualization/windowscontainers/deploy-containers/gpu-acceleration 
   
   But, fortunately, i could just switch back to Use WSL 2 for engine and the root local host bind mounts *were still available*  even though the file sharing menu was no longer available!  So clearly Docker’s claim from https://docs.docker.com/desktop/settings/windows/#file-sharing
    
   
   
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org