[GitHub] [airflow] eladkal commented on a diff in pull request #29979: Avoid considering EmptyOperator in mini scheduler

["eladkal (via GitHub)" <gi...@apache.org>]

eladkal commented on code in PR #29979:
URL: https://github.com/apache/airflow/pull/29979#discussion_r1129911387


##########
airflow/models/taskinstance.py:
##########
@@ -2651,7 +2651,11 @@ def schedule_downstream_tasks(self, session=None):
                 task_id for task_id in partial_dag.task_ids if task_id not in task.downstream_task_ids
             }
 
-            schedulable_tis = [ti for ti in info.schedulable_tis if ti.task_id not in skippable_task_ids]
+            schedulable_tis = [
+                ti
+                for ti in info.schedulable_tis
+                if ti.task_id not in skippable_task_ids and not ti.task.inherits_from_empty_operator
+            ]

Review Comment:
   This is schedule_downstream_tasks for the mini-scheduler
   It's being called in 
   https://github.com/apache/airflow/blob/f9e9d23457cba5d3e18b5bdb7b65ecc63735b65b/airflow/jobs/local_task_job.py#L225
   as part of `handle_task_exit` which is when the task is finished. Basically when task is finished the mini scheduler kicks in. We already have the dag loaded and we assume direct downstream tasks are next to be scheduled so if we can set them here it saves the need for scheduler to scan them in it's next loop.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org