You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/04/02 16:37:23 UTC
[GitHub] [airflow] gudata opened a new issue #15161: Container run do not obey the AIRFLOW_UID changed user
gudata opened a new issue #15161:
URL: https://github.com/apache/airflow/issues/15161
**Apache Airflow version**:
2.0.1
**Environment**:
- **Cloud provider or hardware configuration**: ec2
- **OS** (e.g. from /etc/os-release): amazon linux 2
- **Kernel** (e.g. `uname -a`): Linux ip-10-1-20-149.rifiniti.local 4.14.225-168.357.amzn2.x86_64 #1 SMP Mon Mar 15 18:00:02 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
- **Install tools**:
- **Others**:
**What happened**:
I created .env and add the ids of my system local airflow user
```
AIRFLOW_UID=1001
AIRFLOW_GID=1003
```
then I used the image from [docker-compose.yml](https://airflow.apache.org/docs/apache-airflow/stable/docker-compose.yaml) which I found [here](https://airflow.apache.org/docs/apache-airflow/stable/start/docker.html)
I ran
```
docker-compose run airflow-worker airflow info
```
and it failed with
```
Creating airflow_airflow-worker_run ... done
BACKEND=postgres+psycopg2
DB_HOST=postgres.*****.com
DB_PORT=5432
Traceback (most recent call last):
File "/home/airflow/.local/bin/airflow", line 5, in <module>
from airflow.__main__ import main
ModuleNotFoundError: No module named 'airflow'
ERROR: 1
```
**What you expected to happen**:
I expect the container to run fine.
Here I see a mix of permissions, it looks that the default userid was used
```
(base) [root@ip-10-1-xx-xxx airflow]# docker-compose run airflow-worker bash
Creating airflow_airflow-worker_run ... done
BACKEND=postgres+psycopg2
DB_HOST=postgres.****.com
DB_PORT=5432
I have no name!@84577b619af5:/opt/airflow$ id
uid=1001 gid=1003 groups=1003
I have no name!@84577b619af5:/opt/airflow$ grep airflow /etc/passwd
airflow:x:50000:50000:,,,:/home/airflow:/bin/bash
I have no name!@84577b619af5:/opt/airflow$ ls -la .
total 4
drwxrwxr-x 1 airflow root 60 Apr 2 16:30 .
drwxr-xr-x 1 root root 21 Feb 14 23:31 ..
drwxrwxr-x 3 1001 1003 108 Mar 31 09:52 config
drwxr-xr-x 2 1001 1003 6 Apr 2 16:08 dags
drwxr-xr-x 2 1001 1003 6 Apr 2 16:08 logs
drwxr-xr-x 2 1001 1003 6 Apr 2 16:08 plugins
I have no name!@84577b619af5:/opt/airflow$ ls -la /home/airflow/
total 12
drwxr-xr-x 1 airflow root 20 Feb 14 23:31 .
drwxr-xr-x 1 root root 21 Feb 14 23:31 ..
-rw-r--r-- 1 airflow root 220 Feb 14 23:31 .bash_logout
-rw-r--r-- 1 airflow root 3526 Feb 14 23:31 .bashrc
drwxr-xr-x 5 airflow root 43 Feb 14 23:31 .local
-rw-r--r-- 1 airflow root 807 Feb 14 23:31 .profile
```
I think that this 50000 is already backed in the docker images and can't be changed in runtime, am I right?
If I want to go with custom airflowid I should go with building images, right?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15161: Container run do not obey the AIRFLOW_UID changed user
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15161:
URL: https://github.com/apache/airflow/issues/15161#issuecomment-812690758
Yeah. But it was not clear enough and other people head similar problems - it is not obvious - so checking it and failing if it is not (with redirecting to the docs) :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #15161: Container run do not obey the AIRFLOW_UID changed user
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #15161:
URL: https://github.com/apache/airflow/issues/15161
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15161: Container run do not obey the AIRFLOW_UID changed user
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15161:
URL: https://github.com/apache/airflow/issues/15161#issuecomment-812646106
In your case you can already use custom user_id, but you MUST also specify GID=0 if you want to use it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] gudata commented on issue #15161: Container run do not obey the AIRFLOW_UID changed user
Posted by GitBox <gi...@apache.org>.
gudata commented on issue #15161:
URL: https://github.com/apache/airflow/issues/15161#issuecomment-812668572
Thank you for pointing my problem. Actually, on the second read, it is written that the GID must 0.
```
mkdir ./dags ./logs ./plugins
echo -e "AIRFLOW_UID=$(id -u)\nAIRFLOW_GID=0" > .env
```
probably because of the one-liner expression I missed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #15161: Container run do not obey the AIRFLOW_UID changed user
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #15161:
URL: https://github.com/apache/airflow/issues/15161#issuecomment-812606687
Thanks for opening your first issue here! Be sure to follow the issue template!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #15161: Container run do not obey the AIRFLOW_UID changed user
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #15161:
URL: https://github.com/apache/airflow/issues/15161#issuecomment-812645680
You are not supposed to run the image with arbitrary user id, unless you also use GID=0. This is the way how to handle access to a number of directories inside the docker image, as our image is OpenShift-compatible.
https://docs.openshift.com/container-platform/4.7/openshift_images/create-images.html#images-create-guide-openshift_create-images
I am just adding an error message when wrong combination of uid/gid is used when entering the image (#15612), because while we have it in the documentation, it is not always obvious that you should use it this way (BTW our 2.0.2 upcoming documentation will also be quite a bit more clear about it).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #15161: Container run do not obey the AIRFLOW_UID changed user
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #15161:
URL: https://github.com/apache/airflow/issues/15161#issuecomment-812690758
Yeah. But it was not clear enough and other people head similar problems - it is not obvious - so checking it and failing if it is not (with redirecting to the docs) :) seems the right thing to do.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15161: Container run do not obey the AIRFLOW_UID changed user
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15161:
URL: https://github.com/apache/airflow/issues/15161#issuecomment-812645680
You are not supposed to run the image with arbitrary user id, unless you also use GID=0. This is the way how to handle access to a number of directories inside the docker image, as our image is OpenShift-compatible.
https://docs.openshift.com/container-platform/4.7/openshift_images/create-images.html#images-create-guide-openshift_create-images
I am just adding an error message when wrong combination of uid/gid is used when entering the image (#15612), because while we have it in the documentation, it is not always obvious that you should use it this way (BTW our 2.0.2 upcoming documentation will also be quite a bit more clearer about it).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org