You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/10/14 11:26:42 UTC
[GitHub] [airflow] robinedwards opened a new pull request #18979: Fixes #18943 null execution date on insert to task_fail violating NOT NULL
robinedwards opened a new pull request #18979:
URL: https://github.com/apache/airflow/pull/18979
Insert into task_fail table fails when reaping zombies from the dag processor.
The dag_run property isn't populated by `ti.refresh_from_db()` or by the failure handler.
This resulted in an IntegrityError violating the NOT NULL constraint on execution_date in the task_fail table on Airflow 2.2.0
See #18943 for details
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #18979: Fixes #18943 null execution date on insert to task_fail violating NOT NULL
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #18979:
URL: https://github.com/apache/airflow/pull/18979#discussion_r728895491
##########
File path: airflow/models/taskinstance.py
##########
@@ -1702,7 +1702,8 @@ def handle_failure(
session.add(Log(State.FAILED, self))
# Log failure duration
- session.add(TaskFail(task, self.execution_date, self.start_date, self.end_date))
+ dag_run = self.get_dagrun() # self.dag_run not populated by refresh_from_db
Review comment:
`self.get_dagrun()` already considers `self.run_id` so I don't think this is needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on a change in pull request #18979: Fixes #18943 null execution date on insert to task_fail violating NOT NULL
Posted by GitBox <gi...@apache.org>.
uranusjr commented on a change in pull request #18979:
URL: https://github.com/apache/airflow/pull/18979#discussion_r728895767
##########
File path: airflow/models/taskinstance.py
##########
@@ -1702,7 +1702,8 @@ def handle_failure(
session.add(Log(State.FAILED, self))
# Log failure duration
- session.add(TaskFail(task, self.execution_date, self.start_date, self.end_date))
+ dag_run = self.get_dagrun() # self.dag_run not populated by refresh_from_db
Review comment:
But
```suggestion
dag_run = self.get_dagrun(session=session) # self.dag_run not populated by refresh_from_db
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #18979: Fixes #18943 null execution date on insert to task_fail violating NOT NULL
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #18979:
URL: https://github.com/apache/airflow/pull/18979#discussion_r728895239
##########
File path: airflow/models/taskinstance.py
##########
@@ -1702,7 +1702,8 @@ def handle_failure(
session.add(Log(State.FAILED, self))
# Log failure duration
- session.add(TaskFail(task, self.execution_date, self.start_date, self.end_date))
+ dag_run = self.get_dagrun() # self.dag_run not populated by refresh_from_db
Review comment:
Ah no. sorry. Wrong dagrun
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #18979: Fixes #18943 null execution date on insert to task_fail violating NOT NULL
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #18979:
URL: https://github.com/apache/airflow/pull/18979#discussion_r728897186
##########
File path: airflow/models/taskinstance.py
##########
@@ -1702,7 +1702,8 @@ def handle_failure(
session.add(Log(State.FAILED, self))
# Log failure duration
- session.add(TaskFail(task, self.execution_date, self.start_date, self.end_date))
+ dag_run = self.get_dagrun() # self.dag_run not populated by refresh_from_db
Review comment:
Write - session should be added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] github-actions[bot] commented on pull request #18979: Fixes #18943 null execution date on insert to task_fail violating NOT NULL
Posted by GitBox <gi...@apache.org>.
github-actions[bot] commented on pull request #18979:
URL: https://github.com/apache/airflow/pull/18979#issuecomment-943271931
The PR most likely needs to run full matrix of tests because it modifies parts of the core of Airflow. However, committers might decide to merge it quickly and take the risk. If they don't merge it quickly - please rebase it to the latest main at your convenience, or amend the last commit of the PR, and push it with --force-with-lease.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #18979: Fixes #18943 null execution date on insert to task_fail violating NOT NULL
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #18979:
URL: https://github.com/apache/airflow/pull/18979#discussion_r728895923
##########
File path: airflow/models/taskinstance.py
##########
@@ -1702,7 +1702,8 @@ def handle_failure(
session.add(Log(State.FAILED, self))
# Log failure duration
- session.add(TaskFail(task, self.execution_date, self.start_date, self.end_date))
+ dag_run = self.get_dagrun() # self.dag_run not populated by refresh_from_db
Review comment:
yeah - looked at wrong get_dagrun() (From `dag.py`)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #18979: Fixes #18943 null execution date on insert to task_fail violating NOT NULL
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #18979:
URL: https://github.com/apache/airflow/pull/18979#discussion_r728894323
##########
File path: airflow/models/taskinstance.py
##########
@@ -1702,7 +1702,8 @@ def handle_failure(
session.add(Log(State.FAILED, self))
# Log failure duration
- session.add(TaskFail(task, self.execution_date, self.start_date, self.end_date))
+ dag_run = self.get_dagrun() # self.dag_run not populated by refresh_from_db
Review comment:
I believe this is correct:
```suggestion
dag_run = self.get_dagrun(run_id=self.run_id) # self.dag_run not populated by refresh_from_db
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #18979: Fixes #18943 null execution date on insert to task_fail violating NOT NULL
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #18979:
URL: https://github.com/apache/airflow/pull/18979#discussion_r728895239
##########
File path: airflow/models/taskinstance.py
##########
@@ -1702,7 +1702,8 @@ def handle_failure(
session.add(Log(State.FAILED, self))
# Log failure duration
- session.add(TaskFail(task, self.execution_date, self.start_date, self.end_date))
+ dag_run = self.get_dagrun() # self.dag_run not populated by refresh_from_db
Review comment:
Ah no. sorry. Wrong `get_dagrun` . All ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #18979: Fixes #18943 null execution date on insert to task_fail violating NOT NULL
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #18979:
URL: https://github.com/apache/airflow/pull/18979#discussion_r728894927
##########
File path: airflow/models/taskinstance.py
##########
@@ -1702,7 +1702,8 @@ def handle_failure(
session.add(Log(State.FAILED, self))
# Log failure duration
- session.add(TaskFail(task, self.execution_date, self.start_date, self.end_date))
+ dag_run = self.get_dagrun() # self.dag_run not populated by refresh_from_db
Review comment:
Otherwise you will get first (random) from the list of all the dagruns.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #18979: Fixes #18943 null execution date on insert to task_fail violating NOT NULL
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #18979:
URL: https://github.com/apache/airflow/pull/18979#discussion_r728895923
##########
File path: airflow/models/taskinstance.py
##########
@@ -1702,7 +1702,8 @@ def handle_failure(
session.add(Log(State.FAILED, self))
# Log failure duration
- session.add(TaskFail(task, self.execution_date, self.start_date, self.end_date))
+ dag_run = self.get_dagrun() # self.dag_run not populated by refresh_from_db
Review comment:
yeah - looked d wrong get_dagrun()
##########
File path: airflow/models/taskinstance.py
##########
@@ -1702,7 +1702,8 @@ def handle_failure(
session.add(Log(State.FAILED, self))
# Log failure duration
- session.add(TaskFail(task, self.execution_date, self.start_date, self.end_date))
+ dag_run = self.get_dagrun() # self.dag_run not populated by refresh_from_db
Review comment:
yeah - looked at wrong get_dagrun()
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on a change in pull request #18979: Fixes #18943 null execution date on insert to task_fail violating NOT NULL
Posted by GitBox <gi...@apache.org>.
potiuk commented on a change in pull request #18979:
URL: https://github.com/apache/airflow/pull/18979#discussion_r728897186
##########
File path: airflow/models/taskinstance.py
##########
@@ -1702,7 +1702,8 @@ def handle_failure(
session.add(Log(State.FAILED, self))
# Log failure duration
- session.add(TaskFail(task, self.execution_date, self.start_date, self.end_date))
+ dag_run = self.get_dagrun() # self.dag_run not populated by refresh_from_db
Review comment:
Right - session should be added.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org