You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "dzhigimont (via GitHub)" <gi...@apache.org> on 2023/04/26 20:00:46 UTC
[GitHub] [airflow] dzhigimont opened a new pull request, #30892: Fix depends_on_past work for Dynamic tasks
dzhigimont opened a new pull request, #30892:
URL: https://github.com/apache/airflow/pull/30892
Closes: https://github.com/apache/airflow/issues/29531
I propose a solution in which I check all previously mapped tasks for one task_id as one whole dependency for the current mapped task since we can have different amounts of mapped tasks, and if the current dag run has more mapped tasks and there is no such index in the previous dag run, it is difficult to understand that the previous dag run really doesn't have such an index or tasks haven't been expanded yet. Also, I think if tasks have the same map_index, it aren't guaranteed that they do the same work and it's better to wait until all mapped tasks in the past finish.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] hussein-awala commented on a diff in pull request #30892: Fix depends_on_past work for Dynamic tasks
Posted by "hussein-awala (via GitHub)" <gi...@apache.org>.
hussein-awala commented on code in PR #30892:
URL: https://github.com/apache/airflow/pull/30892#discussion_r1178467682
##########
airflow/ti_deps/deps/prev_dagrun_dep.py:
##########
@@ -80,8 +80,14 @@ def _get_dep_statuses(self, ti: TI, session, dep_context):
yield self._passing_status(reason="This task instance was the first task instance for its task.")
return
- previous_ti = last_dagrun.get_task_instance(ti.task_id, map_index=ti.map_index, session=session)
- if not previous_ti:
+ # For Dynamic tasks previous ti can be more than one
+ previous_tis = session.query(TI).filter(
+ TI.dag_id == ti.dag_id,
+ TI.task_id == ti.task_id,
+ TI.run_id == last_dagrun.run_id,
+ )
+
+ if not previous_tis:
Review Comment:
`previous_tis` is always not None because it's a Query object
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #30892: Fix depends_on_past work for Dynamic tasks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] commented on PR #30892:
URL: https://github.com/apache/airflow/pull/30892#issuecomment-1585869353
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for your contributions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] closed pull request #30892: Fix depends_on_past work for Dynamic tasks
Posted by "github-actions[bot] (via GitHub)" <gi...@apache.org>.
github-actions[bot] closed pull request #30892: Fix depends_on_past work for Dynamic tasks
URL: https://github.com/apache/airflow/pull/30892
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org