You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/11/23 20:24:55 UTC
[GitHub] [airflow] metov opened a new issue #12576: DockerOperator causes log spam when pulling image
metov opened a new issue #12576:
URL: https://github.com/apache/airflow/issues/12576
**Apache Airflow version**: 1.10.4
**Environment**:
- **Cloud provider or hardware configuration**: AWS EC2
- **OS** (e.g. from /etc/os-release): ubuntu
**What happened**:
I am using `DockerOperator` in Airflow set to pull the image before executing. The pull step generates a lot of log spam:
For example the above log has 1104 lines, of which 1045 are Docker pull output.
**What you expected to happen**:
When nothing goes wrong, either of these three options:
* No output from Docker pull
* A single line saying the pull succeeded
* Several lines, one for each parent image, saying that the pull succeeded
When the pull fails, I expect a single line explaining that the pull failed and how/why.
The Docker pull output is clearly meant for an interactive shell as it is trying to display a live progress report. I see that [in my version]( Relatedly, the Docker pull output is clearly meant for an interactive shell as it is trying to display a live progress report.) the output is simply copied to the log. The [more recent version](https://github.com/apache/airflow/blob/8c42cf1b00c90f0d7f11b8a3a455381de8e003c5/airflow/providers/docker/operators/docker.py#L287-L291) also has similar code so I'm assuming the latest Airflow has the same problem.
**How to reproduce it**:
I think simply creating a DAG with a `DockerOperator` that has `force_pull=True` should reproduce the issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG commented on issue #12576: DockerOperator causes log spam when pulling image
Posted by GitBox <gi...@apache.org>.
XD-DENG commented on issue #12576:
URL: https://github.com/apache/airflow/issues/12576#issuecomment-737451553
Thanks @metov for reporting the issue. I will take a look.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] XD-DENG closed issue #12576: DockerOperator causes log spam when pulling image
Posted by GitBox <gi...@apache.org>.
XD-DENG closed issue #12576:
URL: https://github.com/apache/airflow/issues/12576
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #12576: DockerOperator causes log spam when pulling image
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #12576:
URL: https://github.com/apache/airflow/issues/12576#issuecomment-732406060
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org