You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2020/06/07 19:42:45 UTC
[GitHub] [airflow] nullhack opened a new issue #9172: DockerOperator auto_remove deleting container before saving to xcom
nullhack opened a new issue #9172:
URL: https://github.com/apache/airflow/issues/9172
**Apache Airflow version**: 2.0.0.dev0, 1.10.10
**Environment**:
docker container: `apache/airflow:master-ci`
**What happened**:
When using DockerOperator and setting `auto_remove=True` data is not stored on `xcom`
**What you expected to happen**:
I expect that the data is saved after the process and the container is deleted.
**Code to test**
```python
from airflow import DAG
from airflow.utils.dates import days_ago
from datetime import timedelta
from airflow.operators.docker_operator import DockerOperator
params = {
'dag_id': 'test_autoremove',
'schedule_interval': '@daily',
'catchup': True,
'max_active_runs': 2,
'default_args': {
'owner': 'airflow',
'start_date': days_ago(5),
'retries': 2,
'retry_delay': timedelta(minutes=5)
}
}
with DAG(**params) as dag:
write_xcom_docker_warning = DockerOperator(
task_id='write_xcom_docker_warning',
image='python:3-slim',
auto_remove=True,
api_version='auto',
command="""python -c "import logging;[print(i) for i in range(10)];logging.warning('this is a warning')" """,
do_xcom_push=True,
xcom_all=True,
network_mode='bridge')
write_xcom_docker_warning
```
**Sample Output**
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] mik-laj commented on issue #9172: DockerOperator auto_remove deleting container before saving to xcom
Posted by GitBox <gi...@apache.org>.
mik-laj commented on issue #9172:
URL: https://github.com/apache/airflow/issues/9172#issuecomment-640424204
If you create PR, you don't have to create a issue, you know that? issues are information for other people about an bug or about long-term plans. We do not like bureaucracy and additional forms before starting work.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nullhack closed issue #9172: DockerOperator auto_remove deleting container before saving to xcom
Posted by GitBox <gi...@apache.org>.
nullhack closed issue #9172:
URL: https://github.com/apache/airflow/issues/9172
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] nullhack commented on issue #9172: DockerOperator auto_remove deleting container before saving to xcom
Posted by GitBox <gi...@apache.org>.
nullhack commented on issue #9172:
URL: https://github.com/apache/airflow/issues/9172#issuecomment-640471514
> If you create PR, you don't have to create a issue, you know that? issues are information for other people about an bug or about long-term plans. We do not like bureaucracy and additional forms before starting work.
Noted! I didn't know that. From the [Documentation on contributing](https://github.com/apache/airflow/blob/master/CONTRIBUTING.rst#contribution-workflow-example) It was not clear to me. I thought all PRs should be backed by an issue.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org