You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Ashwini (JIRA)" <ji...@apache.org> on 2019/01/21 23:11:00 UTC
[jira] [Created] (AIRFLOW-3746) DockerOperator tasks in Airflow
celery worker are stuck in "Running" state
Ashwini created AIRFLOW-3746:
--------------------------------
Summary: DockerOperator tasks in Airflow celery worker are stuck in "Running" state
Key: AIRFLOW-3746
URL: https://issues.apache.org/jira/browse/AIRFLOW-3746
Project: Apache Airflow
Issue Type: Bug
Components: celery
Reporter: Ashwini
With the following DAG and task and using the celery executor, the task runs but never completes.
*from* *airflow* *import* DAG
*from* *airflow.operators.bash_operator* *import* BashOperator
*from* *airflow.operators.docker_operator* *import* DockerOperator
*from* *datetime* *import* datetime, timedelta
default_args = {
"owner": "airflow",
"depends_on_past": False,
"start_date": datetime(2018, 12, 31),
"email": ["airflow@airflow.com"],
"email_on_failure": False,
"email_on_retry": False,
"retries": 1,
"retry_delay": timedelta(minutes=5),
}
dag = DAG("celery_test", default_args=default_args, schedule_interval=timedelta(1))
DockerOperator(task_id ="test_docker", image = "gitlab-registry.nordstrom.com/merchprice/airflow:hello_python", retries=0, xcom_all=True , dag = dag)
t2.set_upstream(t1)
------------------------
This is verison of airfow and celery and using
python 3.6.
apache-airflow 1.10.1
celery 4.1.1
docker 3.7.0
------------------------------
Here is the logs:
*** Log file does not exist: /home/x9eu/airflow/logs/celery_test/test_docker/2019-01-16T00:00:00+00:00/1.log
*** Fetching from: http://test.com:8793/log/celery_test/test_docker/2019-01-16T00:00:00+00:00/1.log
[2019-01-21 20:49:26,260] \{models.py:1361} INFO - Dependencies all met for <TaskInstance: celery_test.test_docker 2019-01-16T00:00:00+00:00 [queued]>
[2019-01-21 20:49:26,742] \{models.py:1361} INFO - Dependencies all met for <TaskInstance: celery_test.test_docker 2019-01-16T00:00:00+00:00 [queued]>
[2019-01-21 20:49:26,742] \{models.py:1573} INFO -
--------------------------------------------------------------------------------
Starting attempt 1 of 1
--------------------------------------------------------------------------------
[2019-01-21 20:49:26,925] \{models.py:1595} INFO - Executing <Task(DockerOperator): test_docker> on 2019-01-16T00:00:00+00:00
[2019-01-21 20:49:26,925] \{base_task_runner.py:118} INFO - Running: ['bash', '-c', 'airflow run celery_test test_docker 2019-01-16T00:00:00+00:00 --pickle 20 --job_id 59 --raw --cfg_path /tmp/tmps0u9a_e0']
[2019-01-21 20:49:27,524] \{base_task_runner.py:101} INFO - Job 59: Subtask test_docker [2019-01-21 20:49:27,523] \{settings.py:174} INFO - setting.configure_orm(): Using pool settings. pool_size=5, pool_recycle=1800
[2019-01-21 20:49:28,187] \{base_task_runner.py:101} INFO - Job 59: Subtask test_docker [2019-01-21 20:49:28,185] \{__init__.py:51} INFO - Using executor CeleryExecutor
[2019-01-21 20:49:29,544] \{base_task_runner.py:101} INFO - Job 59: Subtask test_docker [2019-01-21 20:49:29,542] \{cli.py:470} INFO - Loading pickle id 20
[2019-01-21 20:49:31,140] \{base_task_runner.py:101} INFO - Job 59: Subtask test_docker [2019-01-21 20:49:31,137] \{cli.py:484} INFO - Running <TaskInstance: celery_test.test_docker 2019-01-16T00:00:00+00:00 [running]> on host test.com
[2019-01-21 20:49:32,603] \{docker_operator.py:182} INFO - Starting docker container from image registry/airflow:hello_python
[2019-01-21 20:49:48,770] \{docker_operator.py:228} INFO - Hello, %d 0
Hello, %d 1
Hello, %d 2
Hello, %d 3
Hello, %d 4
Hello, %d 5
Hello, %d 6
Hello, %d 7
Hello, %d 8
Hello, %d 9
Hello, %d 10
Hello, %d 11
Hello, %d 12
Hello, %d 13
Hello, %d 14
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)