You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "atul-astronomer (via GitHub)" <gi...@apache.org> on 2025/03/05 07:39:21 UTC

[I] RetryOperator is failing: 'RuntimeTaskInstance' object has no attribute 'next_method' [airflow]

atul-astronomer opened a new issue, #47373:
URL: https://github.com/apache/airflow/issues/47373

   ### Apache Airflow version
   
   3.0.0
   
   ### If "Other Airflow 2 version" selected, which one?
   
   _No response_
   
   ### What happened?
   
   RetryOperator is failing.
   
   ```
   [2025-03-05T07:12:25.188823Z] ERROR - Task failed with exception logger="task" error_detail=
   [{"exc_type":"AttributeError","exc_value":"'RuntimeTaskInstance' object has no attribute
    'next_method'","syntax_error":null,"is_cause":false,"frames":
   [{"filename":"/opt/airflow/task_sdk/src/airflow/sdk/execution_time/task_runner.py","lineno":605,"name":"run"},
   {"filename":"/opt/airflow/task_sdk/src/airflow/sdk/execution_time/task_runner.py","lineno":726,"name":"_execut
   e_task"},{"filename":"/opt/airflow/airflow/models/baseoperator.py","lineno":168,"name":"wrapper"}
   ,{"filename":"/files/dags/retry.py","lineno":17,"name":"execute"},{"filename":"/usr/local/lib/python3.9/site
   -packages/pydantic/main.py","lineno":891,"name":"__getattr__"}]}]
   ``` 
   
   ### What you think should happen instead?
   
   RetryOperator should show same behaviour as AF2
   
   ### How to reproduce
   
   Run the below DAG:
   
   ```python
   from datetime import datetime, timedelta
   
   from airflow import DAG
   from airflow.exceptions import AirflowException
   from airflow.models import BaseOperator
   from airflow.triggers.testing import SuccessTrigger
   
   
   class RetryOperator(BaseOperator):
       def execute(self, context):
           ti = context["ti"]
           has_next_method = bool(ti.next_method)
           try_number = ti.try_number
           self.log.info(
               f"In `execute`: has_next_method: {has_next_method}, try_number:{try_number}"
           )
   
           self.defer(
               trigger=SuccessTrigger(),
               method_name="next",
               kwargs={"execute_try_number": try_number},
           )
   
       def next(self, context, execute_try_number, event=None):
           self.log.info("In next!")
           ti = context["ti"]
           has_next_method = bool(ti.next_method)
           try_number = ti.try_number
           self.log.info(
               f"In `next`: has_next_method: {has_next_method}, try_number:{try_number}, excute_try_number: {execute_try_number}"
           )
   
           if try_number == 1:
               # Force a retry
               raise AirflowException("Force a retry")
           # Did we run `execute`?
           if execute_try_number != try_number:
               raise AirflowException("`execute` wasn't run during retry!")
           return None  # Success!
   
   
   with DAG(
       "triggerer_retry", schedule=None, start_date=datetime(2021, 9, 13), tags=['core']
   ) as dag:
       RetryOperator(task_id="retry", retries=1, retry_delay=timedelta(seconds=15))
   
   ``` 
   
   ### Operating System
   
   Linux
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Other
   
   ### Deployment details
   
   _No response_
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Deferred TI object has no attribute 'next_method' [airflow]

Posted by "uranusjr (via GitHub)" <gi...@apache.org>.
uranusjr commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2716354717

   Yeah I tend to agree, in practice the operator does not really care what `next_method` is set to; it’s either set to None, or the runner is operating that exact method anyway.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Deferred TI object has no attribute 'next_method' [airflow]

Posted by "atul-astronomer (via GitHub)" <gi...@apache.org>.
atul-astronomer commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2716570058

   @dstandish I believe this DAG was written to test this ticket https://github.com/apache/airflow/issues/18146 and this is working fine in AF 2.10.
   I tried executing the DAG again by removing to next_method check from the DAG but now getting the below error
   
   [2025-03-12, 05:36:04]  - Task failed with exception logger="task" error_detail=[{"exc_type":"AirflowException","exc_value":"`execute` wasn't run during retry!","syntax_error":null,"is_cause":false,"frames":[{"filename":"/opt/airflow/task_sdk/src/airflow/sdk/execution_time/task_runner.py","lineno":609,"name":"run"},{"filename":"/opt/airflow/task_sdk/src/airflow/sdk/execution_time/task_runner.py","lineno":730,"name":"_execute_task"},{"filename":"/opt/airflow/task_sdk/src/airflow/sdk/definitions/baseoperator.py","lineno":1527,"name":"resume_execution"},{"filename":"/files/dags/retry.py","lineno":43,"name":"next"}]}]


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Deferred TI object has no attribute 'next_method' [airflow]

Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2713720949

   > [@JASNOTJAZZ](https://github.com/JASNOTJAZZ) thats not very clear? What are you trying to convey?
   
   @amoghrajesh That's AI slop is why it's not clear


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Deferred TI object has no attribute 'next_method' [airflow]

Posted by "JASNOTJAZZ (via GitHub)" <gi...@apache.org>.
JASNOTJAZZ commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2700888681

   Investigate if RuntimeTaskInstance should have next_method in Airflow 3.0.
   Check how deferrable operators handle retries in Airflow 3.0—there may have been a change in how attributes are managed.
   Modify the operator to handle cases where next_method is missing, ensuring compatibility with Airflow 3.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Deferred TI object has no attribute 'next_method' [airflow]

Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2705586153

   @JASNOTJAZZ thats not very clear? What are you trying to convey?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Deferred TI object has no attribute 'next_method' [airflow]

Posted by "atul-astronomer (via GitHub)" <gi...@apache.org>.
atul-astronomer commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2709782146

   @amoghrajesh I am able to reproduce this on beta2 as well.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Deferred TI object has no attribute 'next_method' [airflow]

Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil closed issue #47373: Deferred TI object has no attribute 'next_method'
URL: https://github.com/apache/airflow/issues/47373


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Deferred TI object has no attribute 'next_method' [airflow]

Posted by "tirkarthi (via GitHub)" <gi...@apache.org>.
tirkarthi commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2700824498

   I am not able to reproduce the error on main . Some changes were made in fbbe59a2a927281042f5dbad76143f444e2a82ec that seems to be related to this. cc: @ashb @amoghrajesh 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Deferred TI object has no attribute 'next_method' [airflow]

Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2717042223

   Yeah agreed with TP & Daniel -- we don't need `next_method` on RuntimeTI.
   
   If you want it just for testing, you might be able to do the following:
   
   ```python
           ti = context["ti"]
           has_next_method = bool(ti._ti_context_from_server.next_method)
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Deferred TI object has no attribute 'next_method' [airflow]

Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2716335066

   So, the issue is simple: RuntimeTaskInstance doesn't have attr next_method,  and the operator is trying to access it.
   
   question is, is there a reason to add it? I'm not sure what the use case is.  perhaps you can comment @atul-astronomer 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Deferred TI object has no attribute 'next_method' [airflow]

Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on issue #47373:
URL: https://github.com/apache/airflow/issues/47373#issuecomment-2705587009

   @atul-astronomer can you retry it again? I think https://github.com/apache/airflow/pull/47061 shouldve fixed it


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org