You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2022/01/18 11:32:54 UTC
[GitHub] [airflow] chenglongyan opened a new pull request #20920: Fix entire DAG stops when one task has end_date
chenglongyan opened a new pull request #20920:
URL: https://github.com/apache/airflow/pull/20920
related : #19917 , #20471
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chenglongyan commented on pull request #20920: Fix entire DAG stops when one task has end_date
Posted by GitBox <gi...@apache.org>.
chenglongyan commented on pull request #20920:
URL: https://github.com/apache/airflow/pull/20920#issuecomment-1073210316
I'm sorry, I've been too busy lately, I'll follow up in time
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed pull request #20920: Fix entire DAG stops when one task has end_date
Posted by GitBox <gi...@apache.org>.
potiuk closed pull request #20920:
URL: https://github.com/apache/airflow/pull/20920
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chenglongyan closed pull request #20920: Fix entire DAG stops when one task has end_date
Posted by GitBox <gi...@apache.org>.
chenglongyan closed pull request #20920:
URL: https://github.com/apache/airflow/pull/20920
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #20920: Fix entire DAG stops when one task has end_date
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #20920:
URL: https://github.com/apache/airflow/pull/20920#issuecomment-1060110143
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk merged pull request #20920: Fix entire DAG stops when one task has end_date
Posted by GitBox <gi...@apache.org>.
potiuk merged pull request #20920:
URL: https://github.com/apache/airflow/pull/20920
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ephraimbuddy commented on pull request #20920: Fix entire DAG stops when one task has end_date
Posted by GitBox <gi...@apache.org>.
ephraimbuddy commented on pull request #20920:
URL: https://github.com/apache/airflow/pull/20920#issuecomment-1073183850
@chenglongyan please rebase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on pull request #20920: Fix entire DAG stops when one task has end_date
Posted by GitBox <gi...@apache.org>.
uranusjr commented on pull request #20920:
URL: https://github.com/apache/airflow/pull/20920#issuecomment-1018244171
Please revert all the whitespace changes. They make static checks fail, and it’s extremely difficult to review what you actually want to change due to all the noise.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] chenglongyan commented on pull request #20920: Fix entire DAG stops when one task has end_date
Posted by GitBox <gi...@apache.org>.
chenglongyan commented on pull request #20920:
URL: https://github.com/apache/airflow/pull/20920#issuecomment-1075360351
@ephraimbuddy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #20920: Fix entire DAG stops when one task has end_date
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #20920:
URL: https://github.com/apache/airflow/pull/20920#discussion_r835053435
##########
File path: airflow/models/dagrun.py
##########
@@ -798,7 +798,9 @@ def verify_integrity(self, session: Session = NEW_SESSION):
def task_filter(task: "Operator") -> bool:
return task.task_id not in task_ids and (
- self.is_backfill or task.start_date <= self.execution_date
+ self.is_backfill
+ or task.start_date <= self.execution_date
+ and (task.end_date is None or self.execution_date <= task.end_date)
)
Review comment:
Let’s refactor this to early returns to improve readability, something like
```python
def task_filter(task: "Operator") -> bool:
if task.task_id in task_ids:
return False
if self.is_backfill:
return ...
...
return ...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #20920: Fix entire DAG stops when one task has end_date
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #20920:
URL: https://github.com/apache/airflow/pull/20920#discussion_r835053435
##########
File path: airflow/models/dagrun.py
##########
@@ -798,7 +798,9 @@ def verify_integrity(self, session: Session = NEW_SESSION):
def task_filter(task: "Operator") -> bool:
return task.task_id not in task_ids and (
- self.is_backfill or task.start_date <= self.execution_date
+ self.is_backfill
+ or task.start_date <= self.execution_date
+ and (task.end_date is None or self.execution_date <= task.end_date)
)
Review comment:
Let’s refactor this a bit to improve readability, something like
```python
def task_filter(task: "Operator") -> bool:
if task.task_id in task_ids:
return False
if self.is_backfill:
...
...
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org