You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "t oo (Jira)" <ji...@apache.org> on 2020/07/24 10:24:00 UTC
[jira] [Commented] (AIRFLOW-4481) DummyOperator performance
improvement - currently needs +87 seconds to do nothing
[ https://issues.apache.org/jira/browse/AIRFLOW-4481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17164348#comment-17164348 ]
t oo commented on AIRFLOW-4481:
-------------------------------
still seeing 6.88 seconds for Success state and 25.08 seconds for Skipped state
> DummyOperator performance improvement - currently needs +87 seconds to do nothing
> ---------------------------------------------------------------------------------
>
> Key: AIRFLOW-4481
> URL: https://issues.apache.org/jira/browse/AIRFLOW-4481
> Project: Apache Airflow
> Issue Type: Improvement
> Components: DAG, operators
> Affects Versions: 1.10.3
> Reporter: t oo
> Priority: Trivial
>
> I don't know why a DummyOperator can't finish in sub-second time, currently taking >4 seconds.
> *** Reading local file: /home/ec2-user/airflow/logs/stream_run/JOIN_tsk_/2019-01-01T00:00:00+00:00/1.log
> [2019-05-08 15:33:53,982] {__init__.py:1139} INFO - Dependencies all met for <TaskInstance: stream_run.JOIN_tsk_ 2019-01-01T00:00:00+00:00 [queued]>
> [2019-05-08 15:33:54,009] {__init__.py:1139} INFO - Dependencies all met for <TaskInstance: stream_run.JOIN_tsk_ 2019-01-01T00:00:00+00:00 [queued]>
> [2019-05-08 15:33:54,009] {__init__.py:1353} INFO -
> --------------------------------------------------------------------------------
> [2019-05-08 15:33:54,009] {__init__.py:1354} INFO - Starting attempt 1 of 1
> [2019-05-08 15:33:54,009] {__init__.py:1355} INFO -
> --------------------------------------------------------------------------------
> [2019-05-08 15:33:54,026] {__init__.py:1374} INFO - Executing <Task(DummyOperator): JOIN_tsk_> on 2019-01-01T00:00:00+00:00
> [2019-05-08 15:33:54,026] {base_task_runner.py:119} INFO - Running: [u'airflow', u'run', 'stream_run', 'JOIN_tsk_', '2019-01-01T00:00:00+00:00', u'--job_id', '2550', u'--raw', u'-sd', u'DAGS_FOLDER/stream_dag_generation.py', u'--cfg_path', '/tmp/tmpJR_rQX']
> [2019-05-08 15:33:56,117] {base_task_runner.py:101} INFO - Job 2550: Subtask JOIN_tsk_ [2019-05-08 15:33:56,117] {settings.py:182} INFO - settings.configure_orm(): Using pool settings. pool_size=5, pool_recycle=1800, pid=16385
> [2019-05-08 15:33:56,317] {base_task_runner.py:101} INFO - Job 2550: Subtask JOIN_tsk_ [2019-05-08 15:33:56,317] {__init__.py:51} INFO - Using executor LocalExecutor
> [2019-05-08 15:33:56,582] {base_task_runner.py:101} INFO - Job 2550: Subtask JOIN_tsk_ [2019-05-08 15:33:56,581] {__init__.py:305} INFO - Filling up the DagBag from /home/ec2-user/airflow/dags/stream_dag_generation.py
> [2019-05-08 15:33:58,002] {base_task_runner.py:101} INFO - Job 2550: Subtask JOIN_tsk_ [2019-05-08 15:33:58,001] {cli.py:517} INFO - Running <TaskInstance: stream_run.JOIN_tsk_ 2019-01-01T00:00:00+00:00 [running]> on host domain
> [2019-05-08 15:33:58,279] {logging_mixin.py:95} INFO - [2019-05-08 15:33:58,279] {jobs.py:2562} INFO - Task exited with return code 0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)