You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "atul-astronomer (via GitHub)" <gi...@apache.org> on 2025/03/05 07:39:21 UTC
[I] RetryOperator is failing: 'RuntimeTaskInstance' object has no attribute 'next_method' [airflow]
atul-astronomer opened a new issue, #47373:
URL: https://github.com/apache/airflow/issues/47373
### Apache Airflow version
3.0.0
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
RetryOperator is failing.
```
[2025-03-05T07:12:25.188823Z] ERROR - Task failed with exception logger="task" error_detail=
[{"exc_type":"AttributeError","exc_value":"'RuntimeTaskInstance' object has no attribute
'next_method'","syntax_error":null,"is_cause":false,"frames":
[{"filename":"/opt/airflow/task_sdk/src/airflow/sdk/execution_time/task_runner.py","lineno":605,"name":"run"},
{"filename":"/opt/airflow/task_sdk/src/airflow/sdk/execution_time/task_runner.py","lineno":726,"name":"_execut
e_task"},{"filename":"/opt/airflow/airflow/models/baseoperator.py","lineno":168,"name":"wrapper"}
,{"filename":"/files/dags/retry.py","lineno":17,"name":"execute"},{"filename":"/usr/local/lib/python3.9/site
-packages/pydantic/main.py","lineno":891,"name":"__getattr__"}]}]
```
### What you think should happen instead?
RetryOperator should show same behaviour as AF2
### How to reproduce
Run the below DAG:
```python
from datetime import datetime, timedelta
from airflow import DAG
from airflow.exceptions import AirflowException
from airflow.models import BaseOperator
from airflow.triggers.testing import SuccessTrigger
class RetryOperator(BaseOperator):
def execute(self, context):
ti = context["ti"]
has_next_method = bool(ti.next_method)
try_number = ti.try_number
self.log.info(
f"In `execute`: has_next_method: {has_next_method}, try_number:{try_number}"
)
self.defer(
trigger=SuccessTrigger(),
method_name="next",
kwargs={"execute_try_number": try_number},
)
def next(self, context, execute_try_number, event=None):
self.log.info("In next!")
ti = context["ti"]
has_next_method = bool(ti.next_method)
try_number = ti.try_number
self.log.info(
f"In `next`: has_next_method: {has_next_method}, try_number:{try_number}, excute_try_number: {execute_try_number}"
)
if try_number == 1:
# Force a retry
raise AirflowException("Force a retry")
# Did we run `execute`?
if execute_try_number != try_number:
raise AirflowException("`execute` wasn't run during retry!")
return None # Success!
with DAG(
"triggerer_retry", schedule=None, start_date=datetime(2021, 9, 13), tags=['core']
) as dag:
RetryOperator(task_id="retry", retries=1, retry_delay=timedelta(seconds=15))
```
### Operating System
Linux
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred TI object has no attribute 'next_method' [airflow]
Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2716354717
Yeah I tend to agree, in practice the operator does not really care what `next_method` is set to; it’s either set to None, or the runner is operating that exact method anyway.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred TI object has no attribute 'next_method' [airflow]
Posted by "atul-astronomer (via GitHub)" <gi...@apache.org>.
atul-astronomer commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2716570058
@dstandish I believe this DAG was written to test this ticket https://github.com/apache/airflow/issues/18146 and this is working fine in AF 2.10.
I tried executing the DAG again by removing to next_method check from the DAG but now getting the below error
[2025-03-12, 05:36:04] - Task failed with exception logger="task" error_detail=[{"exc_type":"AirflowException","exc_value":"`execute` wasn't run during retry!","syntax_error":null,"is_cause":false,"frames":[{"filename":"/opt/airflow/task_sdk/src/airflow/sdk/execution_time/task_runner.py","lineno":609,"name":"run"},{"filename":"/opt/airflow/task_sdk/src/airflow/sdk/execution_time/task_runner.py","lineno":730,"name":"_execute_task"},{"filename":"/opt/airflow/task_sdk/src/airflow/sdk/definitions/baseoperator.py","lineno":1527,"name":"resume_execution"},{"filename":"/files/dags/retry.py","lineno":43,"name":"next"}]}]
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred TI object has no attribute 'next_method' [airflow]
Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2713720949
> [@JASNOTJAZZ](https://github.com/JASNOTJAZZ) thats not very clear? What are you trying to convey?
@amoghrajesh That's AI slop is why it's not clear
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred TI object has no attribute 'next_method' [airflow]
Posted by "JASNOTJAZZ (via GitHub)" <gi...@apache.org>.
JASNOTJAZZ commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2700888681
Investigate if RuntimeTaskInstance should have next_method in Airflow 3.0.
Check how deferrable operators handle retries in Airflow 3.0—there may have been a change in how attributes are managed.
Modify the operator to handle cases where next_method is missing, ensuring compatibility with Airflow 3.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred TI object has no attribute 'next_method' [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2705586153
@JASNOTJAZZ thats not very clear? What are you trying to convey?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred TI object has no attribute 'next_method' [airflow]
Posted by "atul-astronomer (via GitHub)" <gi...@apache.org>.
atul-astronomer commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2709782146
@amoghrajesh I am able to reproduce this on beta2 as well.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred TI object has no attribute 'next_method' [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil closed issue #47373: Deferred TI object has no attribute 'next_method'
URL: https://github.com/apache/airflow/issues/47373
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred TI object has no attribute 'next_method' [airflow]
Posted by "tirkarthi (via GitHub)" <gi...@apache.org>.
tirkarthi commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2700824498
I am not able to reproduce the error on main . Some changes were made in fbbe59a2a927281042f5dbad76143f444e2a82ec that seems to be related to this. cc: @ashb @amoghrajesh
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred TI object has no attribute 'next_method' [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2717042223
Yeah agreed with TP & Daniel -- we don't need `next_method` on RuntimeTI.
If you want it just for testing, you might be able to do the following:
```python
ti = context["ti"]
has_next_method = bool(ti._ti_context_from_server.next_method)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred TI object has no attribute 'next_method' [airflow]
Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2716335066
So, the issue is simple: RuntimeTaskInstance doesn't have attr next_method, and the operator is trying to access it.
question is, is there a reason to add it? I'm not sure what the use case is. perhaps you can comment @atul-astronomer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Deferred TI object has no attribute 'next_method' [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2705587009
@atul-astronomer can you retry it again? I think https://github.com/apache/airflow/pull/47061 shouldve fixed it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org