You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/01/26 05:49:15 UTC
[GitHub] [airflow] PKJonas opened a new issue #13905: DockerOperator fails to pull an image
PKJonas opened a new issue #13905:
URL: https://github.com/apache/airflow/issues/13905
### Bug
**Expected behavior**:
`DockerOperator` should attempt to pull an image when it is not present locally.
**Actual behavior**:
`DockerOperator` does not attempt to pull an image unless `force_pull` is set to `True`, instead displaying a misleading 404 error.
**Package versions**:
`apache-airflow-providers-docker 1.0.0`
`docker 3.7.3`
**Workaround**:
Set `force_pull` to `True`
### Minimal repro
Make sure you don't have an image tagged `ubuntu:latest` present locally.
```
DockerOperator(
task_id=f'try_to_pull_ubuntu',
image='ubuntu:latest',
command=f'''echo hello'''
)
```
prints: `{taskinstance.py:1396} ERROR - 404 Client Error: Not Found ("No such image: ubuntu:latest")`
This, on the other hand:
```
DockerOperator(
task_id=f'try_to_pull_ubuntu',
image='ubuntu:latest',
command=f'''echo hello''',
force_pull=True
)
```
pulls the image and prints `{docker.py:263} INFO - hello`
### Source of the bug
When trying to run an image that's not present locally, `self.cli.images(name=self.image)` in the line: https://github.com/apache/airflow/blob/8723b1feb82339d7a4ba5b40a6c4d4bbb995a4f9/airflow/providers/docker/operators/docker.py#L286 returns a non-empty array even when the image has been deleted from the local machine. If `force_pull` is not set, a pull is not attempted and a misleading `404` error is raised downstream.
In fact, `self.cli.images` appears to return non-empty arrays even when supplied with nonsense image names.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] fredthomsen edited a comment on issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
fredthomsen edited a comment on issue #13905:
URL: https://github.com/apache/airflow/issues/13905#issuecomment-834787835
Looks like the problem here is the filter argument being deprecated in docker API version 1.25: https://github.com/docker/cli/blob/v20.10.0-rc1/docs/deprecated.md#filter-param-for-imagesjson-endpoint. This fix (https://github.com/docker/docker-py/commit/1757c974fa3a05b0e9b783af85242b18df09d05d) is available in `docker>=4.4.1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ldealmei commented on issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
ldealmei commented on issue #13905:
URL: https://github.com/apache/airflow/issues/13905#issuecomment-781211381
I am facing the same issue. This seems to only happen when running Airflow inside a Docker container. I had a non-dockerized deployment for which it worked fine without the `force_pull=True` option. Since going to a docker-compose deployment, I've been having the same 404 error.
```
[2021-02-18 09:24:44,360] {docker.py:224} INFO - Starting docker container from image busybox:stable
[2021-02-18 09:24:44,394] {taskinstance.py:1396} ERROR - 404 Client Error: Not Found ("No such image: busybox:stable")
Traceback (most recent call last):
File "/home/airflow/.local/lib/python3.6/site-packages/docker/api/client.py", line 261, in _raise_for_status
response.raise_for_status()
File "/home/airflow/.local/lib/python3.6/site-packages/requests/models.py", line 941, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http+docker://localhost/v1.41/containers/create
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] fredthomsen commented on issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
fredthomsen commented on issue #13905:
URL: https://github.com/apache/airflow/issues/13905#issuecomment-834787835
Looks like the problem here is the filter argument being deprecated in docker API version 1.25: https://github.com/docker/cli/blob/v20.10.0-rc1/docs/deprecated.md#filter-param-for-imagesjson-endpoint. This fix (https://github.com/docker/docker-py/commit/1757c974fa3a05b0e9b783af85242b18df09d05d) is available in docker-py>=4.4.1
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #13905:
URL: https://github.com/apache/airflow/issues/13905#issuecomment-767312232
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] PKJonas commented on issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
PKJonas commented on issue #13905:
URL: https://github.com/apache/airflow/issues/13905#issuecomment-768066364
I have updated the original issue using the bug report template.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] fredthomsen edited a comment on issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
fredthomsen edited a comment on issue #13905:
URL: https://github.com/apache/airflow/issues/13905#issuecomment-834787835
Looks like the problem here is the filter argument being removed in docker API version 1.41: https://github.com/docker/cli/blob/v20.10.0-rc1/docs/deprecated.md#filter-param-for-imagesjson-endpoint. This fix (https://github.com/docker/docker-py/commit/1757c974fa3a05b0e9b783af85242b18df09d05d) is available in `docker>=4.4.1`
I confirmed this by installing 4.4.4 and running a sample docker dag that was failing earlier due to the image not being present.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] kaxil commented on issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
kaxil commented on issue #13905:
URL: https://github.com/apache/airflow/issues/13905#issuecomment-767544084
> > Thanks for opening your first issue here! Be sure to follow the issue template!
>
> Relatedly, I looked for an issue template in the [contributing guidelines](https://github.com/apache/airflow/blob/df1503ea0aad34f6e1ed005444dacae5f78f7fde/CONTRIBUTING.rst#report-bugs) and couldn't find one.
The issue template is just all the questions asked when you create this Github issue like the Airflow version, how to reproduce, etc (https://github.com/apache/airflow/issues/new?assignees=&labels=kind%3Abug&template=bug_report.md&title=)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #13905:
URL: https://github.com/apache/airflow/issues/13905#issuecomment-846508680
You will have to wait for the next provider's release (which is going to happen in ~ week)
This issue have been solved in https://github.com/apache/airflow/pull/15731
This change was mostly about removing the unnecessary limitation, and seems that upgrading just docker package should fixit. A very dirty trick if you want to try it (but this is by no means a recommendation as a fix, just something you can try) is to downgrade PIP to 20.2.4 and install the docker package manually with `==` if PIP 21 does not let you do the same.
Note that the providers we are going to release in the next wave will require you also to migrate to Airflow 2.1 as those providers will be `Airlfow 2.1+` only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] PKJonas commented on issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
PKJonas commented on issue #13905:
URL: https://github.com/apache/airflow/issues/13905#issuecomment-767560104
> > > Thanks for opening your first issue here! Be sure to follow the issue template!
> >
> >
> > Relatedly, I looked for an issue template in the [contributing guidelines](https://github.com/apache/airflow/blob/df1503ea0aad34f6e1ed005444dacae5f78f7fde/CONTRIBUTING.rst#report-bugs) and couldn't find one.
>
> The issue template is just all the questions asked when you create this Github issue like the Airflow version, how to reproduce, etc (https://github.com/apache/airflow/issues/new?assignees=&labels=kind%3Abug&template=bug_report.md&title=)
Thank you! I'll update my issue using that template.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #13905:
URL: https://github.com/apache/airflow/issues/13905
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] fredthomsen edited a comment on issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
fredthomsen edited a comment on issue #13905:
URL: https://github.com/apache/airflow/issues/13905#issuecomment-834787835
Looks like the problem here is the filter argument being removed in docker API version 1.41: https://github.com/docker/cli/blob/v20.10.0-rc1/docs/deprecated.md#filter-param-for-imagesjson-endpoint. This fix (https://github.com/docker/docker-py/commit/1757c974fa3a05b0e9b783af85242b18df09d05d) is available in `docker>=4.4.1`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] PKJonas commented on issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
PKJonas commented on issue #13905:
URL: https://github.com/apache/airflow/issues/13905#issuecomment-767312738
> Thanks for opening your first issue here! Be sure to follow the issue template!
Relatedly, I looked for an issue template in the [contributing guidelines](https://github.com/apache/airflow/blob/df1503ea0aad34f6e1ed005444dacae5f78f7fde/CONTRIBUTING.rst#report-bugs) and couldn't find one.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #13905:
URL: https://github.com/apache/airflow/issues/13905#issuecomment-846508680
You will have to wait for the next provider's release (which is going to happen in ~ week)
This issue has been solved in https://github.com/apache/airflow/pull/15731
This change was mostly about removing the unnecessary limitation, and seems that upgrading just docker package should fixit. A very dirty trick if you want to try it (but this is by no means a recommendation as a fix, just something you can try) is to downgrade PIP to 20.2.4 and install the docker package manually with `==` if PIP 21 does not let you do the same.
Note that the providers we are going to release in the next wave will require you also to migrate to Airflow 2.1 as those providers will be `Airlfow 2.1+` only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] sktrinh12 commented on issue #13905: DockerOperator fails to pull an image
Posted by GitBox <gi...@apache.org>.
sktrinh12 commented on issue #13905:
URL: https://github.com/apache/airflow/issues/13905#issuecomment-846506788
> Looks like the problem here is the filter argument being removed in docker API version 1.41: https://github.com/docker/cli/blob/v20.10.0-rc1/docs/deprecated.md#filter-param-for-imagesjson-endpoint. This fix ([docker/docker-py@1757c97](https://github.com/docker/docker-py/commit/1757c974fa3a05b0e9b783af85242b18df09d05d)) is available in `docker>=4.4.1`
>
> I confirmed this by installing 4.4.4 and running a sample docker dag that was failing earlier due to the image not being present.
I tried going into the docker container (webserver) and did `pip install docker -U` and it gave me:
```
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
apache-airflow-providers-docker 1.2.0 requires docker~=3.0, but you have docker 5.0.0 which is incompatible.
```
I'm using the production ready docker-compose.yml from the website running on ubuntu 20.04. Anything else I should try?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org