You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/02/02 19:31:23 UTC
[GitHub] [airflow] eawer opened a new issue #14037: ti.duration is None inside the on_success callback if running dag completely
eawer opened a new issue #14037:
URL: https://github.com/apache/airflow/issues/14037
**Apache Airflow version**: 1.10.9
**Environment**: puckel/docker-airflow:1.10.9
**What happened**:
If I run the dag using the "Trigger dag" button, or if it is run on schedule - the `duration` property of `context["ti"]` is None. If I trigger each task manually via UI - everything works as expected.
<!-- (please include exact error messages if you can) -->
```
{logging_mixin.py:112} INFO - None
{taskinstance.py:1030} ERROR - Failed when executing success callback
{taskinstance.py:1031} ERROR - unsupported type for timedelta seconds component: NoneType
Traceback (most recent call last):
File "/usr/local/lib/python3.7/site-packages/airflow/models/taskinstance.py", line 1028, in _run_raw_task
task.on_success_callback(context)
File "/usr/local/airflow/dags/pipeline.py", line 32, in send_notification
message += f", duration {str(timedelta(seconds=ti.duration)).split('.')[0]}"
TypeError: unsupported type for timedelta seconds component: NoneType
```
**What you expected to happen**:
I expect `context["ti"].duration` to have value for each successful task run
**How to reproduce it**:
My code is as simple as this:
```python
from datetime import timedelta
from airflow import DAG
from airflow.operators.dummy_operator import DummyOperator
def send_notification(context):
ti = context["ti"]
print(ti.duration)
task_state = ti.state
message = f"[{task_state}] - Task `{ti.task_id}`"
if task_state == "success":
message += f", duration {str(timedelta(seconds=ti.duration)).split('.')[0]}"
print(message)
DAG_DEFAULT_ARGS = {
...
'provide_context': True,
"on_success_callback": send_notification,
"on_failure_callback": send_notification,
"on_retry_callback": send_notification,
}
with DAG("test", schedule_interval="* * * * *", default_args=DAG_DEFAULT_ARGS) as dag:
start_task = DummyOperator(
task_id='start',
dag=dag,
)
end_task = DummyOperator(
task_id='end',
dag=dag,
)
start_task >> end_task
```
**Anything else we need to know**: It happens each time the dag is triggered completely and does not happen if I trigger each task manually via UI
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #14037: ti.duration is None inside the on_success callback if running dag completely
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #14037:
URL: https://github.com/apache/airflow/issues/14037#issuecomment-771916735
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal edited a comment on issue #14037: ti.duration is None inside the on_success callback if running dag completely
Posted by GitBox <gi...@apache.org>.
eladkal edited a comment on issue #14037:
URL: https://github.com/apache/airflow/issues/14037#issuecomment-981633168
Works fine for me.
Scheduled:
```
AIRFLOW_CTX_DAG_RUN_ID=scheduled__2021-11-28T00:43:00+00:00
[2021-11-29, 13:23:35 UTC] {taskinstance.py:1278} INFO - Marking task as SUCCESS. dag_id=14037, task_id=end, execution_date=20211128T004300, start_date=20211129T132334, end_date=20211129T132335
[2021-11-29, 13:23:35 UTC] {local_task_job.py:154} INFO - Task exited with return code 0
[2021-11-29, 13:23:35 UTC] {logging_mixin.py:109} INFO - 0.596116
[2021-11-29, 13:23:35 UTC] {logging_mixin.py:109} INFO - [success] - Task `end`, duration 0:00:00
[2021-11-29, 13:23:35 UTC] {local_task_job.py:264} INFO - 0 downstream tasks scheduled from follow-on schedule check
```
Manual:
```
AIRFLOW_CTX_DAG_RUN_ID=manual__2021-11-29T13:24:12.442239+00:00
[2021-11-29, 13:24:19 UTC] {taskinstance.py:1278} INFO - Marking task as SUCCESS. dag_id=14037, task_id=end, execution_date=20211129T132412, start_date=20211129T132418, end_date=20211129T132419
[2021-11-29, 13:24:19 UTC] {local_task_job.py:154} INFO - Task exited with return code 0
[2021-11-29, 13:24:19 UTC] {logging_mixin.py:109} INFO - 0.610548
[2021-11-29, 13:24:19 UTC] {logging_mixin.py:109} INFO - [success] - Task `end`, duration 0:00:00
[2021-11-29, 13:24:19 UTC] {local_task_job.py:264} INFO - 0 downstream tasks scheduled from follow-on schedule check
```
If you still experiencing the issue please update Airflow to latest version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal commented on issue #14037: ti.duration is None inside the on_success callback if running dag completely
Posted by GitBox <gi...@apache.org>.
eladkal commented on issue #14037:
URL: https://github.com/apache/airflow/issues/14037#issuecomment-981633168
Works fine for me .
Scheduled:
```
AIRFLOW_CTX_DAG_RUN_ID=scheduled__2021-11-28T00:43:00+00:00
[2021-11-29, 13:23:35 UTC] {taskinstance.py:1278} INFO - Marking task as SUCCESS. dag_id=14037, task_id=end, execution_date=20211128T004300, start_date=20211129T132334, end_date=20211129T132335
[2021-11-29, 13:23:35 UTC] {local_task_job.py:154} INFO - Task exited with return code 0
[2021-11-29, 13:23:35 UTC] {logging_mixin.py:109} INFO - 0.596116
[2021-11-29, 13:23:35 UTC] {logging_mixin.py:109} INFO - [success] - Task `end`, duration 0:00:00
[2021-11-29, 13:23:35 UTC] {local_task_job.py:264} INFO - 0 downstream tasks scheduled from follow-on schedule check
```
Manual:
```
AIRFLOW_CTX_DAG_RUN_ID=manual__2021-11-29T13:24:12.442239+00:00
[2021-11-29, 13:24:19 UTC] {taskinstance.py:1278} INFO - Marking task as SUCCESS. dag_id=14037, task_id=end, execution_date=20211129T132412, start_date=20211129T132418, end_date=20211129T132419
[2021-11-29, 13:24:19 UTC] {local_task_job.py:154} INFO - Task exited with return code 0
[2021-11-29, 13:24:19 UTC] {logging_mixin.py:109} INFO - 0.610548
[2021-11-29, 13:24:19 UTC] {logging_mixin.py:109} INFO - [success] - Task `end`, duration 0:00:00
[2021-11-29, 13:24:19 UTC] {local_task_job.py:264} INFO - 0 downstream tasks scheduled from follow-on schedule check
```
If you still experiencing the issue please update Airflow to latest version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] eladkal closed issue #14037: ti.duration is None inside the on_success callback if running dag completely
Posted by GitBox <gi...@apache.org>.
eladkal closed issue #14037:
URL: https://github.com/apache/airflow/issues/14037
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] boring-cyborg[bot] commented on issue #14037: ti.duration is None inside the on_success callback if running dag completely
Posted by GitBox <gi...@apache.org>.
boring-cyborg[bot] commented on issue #14037:
URL: https://github.com/apache/airflow/issues/14037#issuecomment-771916735
Thanks for opening your first issue here! Be sure to follow the issue template!
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org