You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "tirkarthi (via GitHub)" <gi...@apache.org> on 2024/04/13 09:28:06 UTC

[PR] Increment try_number while clearing deferred tasks. [airflow]

tirkarthi opened a new pull request, #38984:
URL: https://github.com/apache/airflow/pull/38984

   closes: #38735
   related: #38735
   
   This is already done while marking tasks as success/failure. Should this be done for up_for_reschedule tasks too? See also https://github.com/apache/airflow/pull/30669
   
   https://github.com/apache/airflow/blob/d03ba594b3158c127c1c1b3f1d0c31fb93104367/airflow/api/common/mark_tasks.py#L161-L164
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Increment try_number while clearing deferred tasks. [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #38984:
URL: https://github.com/apache/airflow/pull/38984#issuecomment-2146653182

   Yeah. 2.10 is not that far away - and until then @tirkarthi can simply run their forked Airlfow until they upgrade to 2.10.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Increment try_number while clearing deferred tasks. [airflow]

Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on PR #38984:
URL: https://github.com/apache/airflow/pull/38984#issuecomment-2146109601

   > @tirkarthi @dstandish do we want to merge this PR for 2.9.2?
   
   I think we cannot merge this anymore now that https://github.com/apache/airflow/pull/39336 is merged. At least not into main.  I think the only way this could be released is if it's patched on a 2.9.2 branch specifically.  WDYT?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Increment try_number while clearing deferred tasks. [airflow]

Posted by "tirkarthi (via GitHub)" <gi...@apache.org>.
tirkarthi commented on code in PR #38984:
URL: https://github.com/apache/airflow/pull/38984#discussion_r1564470882


##########
airflow/models/taskinstance.py:
##########
@@ -274,6 +274,12 @@ def clear_task_instances(
                 ti.state = TaskInstanceState.RESTARTING
                 job_ids.append(ti.job_id)
         else:
+            # When the task is deferred the try_number is decremented so that the same try
+            # number is used when the task resumes execution to process the  event. But in case of
+            # clearing the try number should be incremented so that the next run doesn't reuse the same try_number
+            if ti.state == TaskInstanceState.DEFERRED:

Review Comment:
   @dirrao Sorry, I don't understand your comment. When the task moves from running to deferred the try number is decremented so that when task resumes from the trigger to running the try number is incremented so that same try number is used and helps with logging etc. When the task is cleared then the try number is incremented but since it's in deferred state with try number decremented it causes issues where same log file is used, retry count mismatch etc. for different attempts.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Increment try_number while clearing deferred tasks. [airflow]

Posted by "bbovenzi (via GitHub)" <gi...@apache.org>.
bbovenzi commented on PR #38984:
URL: https://github.com/apache/airflow/pull/38984#issuecomment-2090944403

   cc: @dstandish 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Increment try_number while clearing deferred tasks. [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #38984:
URL: https://github.com/apache/airflow/pull/38984#issuecomment-2144355862

   @tirkarthi @dstandish do we want to merge this PR for 2.9.2?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Increment try_number while clearing deferred tasks. [airflow]

Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on PR #38984:
URL: https://github.com/apache/airflow/pull/38984#issuecomment-2091215899

   this should be obsoleted by this PR https://github.com/apache/airflow/pull/39336
   
   i'm just working through bunch of test fixes cus we make a _lot_ of assertions about try_number because it's always been so problematic


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Increment try_number while clearing deferred tasks. [airflow]

Posted by "dstandish (via GitHub)" <gi...@apache.org>.
dstandish commented on PR #38984:
URL: https://github.com/apache/airflow/pull/38984#issuecomment-2091959568

   > @dstandish I agree the linked PR is a good long term solution but this PR is good enough for a bugfix in 2.9.x series with 2.10 release few months away.
   
   Sure no problem 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Increment try_number while clearing deferred tasks. [airflow]

Posted by "tirkarthi (via GitHub)" <gi...@apache.org>.
tirkarthi commented on PR #38984:
URL: https://github.com/apache/airflow/pull/38984#issuecomment-2146329579

   I am okay with closing this since main has a long term fix and 2.10 will be released in few months hopefully. We had this patch since we found this during our upgrade to 2.7.0 using deferrable operators and the patch has been working well.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Increment try_number while clearing deferred tasks. [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed pull request #38984: Increment try_number while clearing deferred tasks.
URL: https://github.com/apache/airflow/pull/38984


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Increment try_number while clearing deferred tasks. [airflow]

Posted by "dirrao (via GitHub)" <gi...@apache.org>.
dirrao commented on code in PR #38984:
URL: https://github.com/apache/airflow/pull/38984#discussion_r1564417518


##########
airflow/models/taskinstance.py:
##########
@@ -274,6 +274,12 @@ def clear_task_instances(
                 ti.state = TaskInstanceState.RESTARTING
                 job_ids.append(ti.job_id)
         else:
+            # When the task is deferred the try_number is decremented so that the same try
+            # number is used when the task resumes execution to process the  event. But in case of
+            # clearing the try number should be incremented so that the next run doesn't reuse the same try_number
+            if ti.state == TaskInstanceState.DEFERRED:

Review Comment:
   I believe clear  action is meant to quickly restart things. However, here we are increasing the try_number nothing else. 



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Increment try_number while clearing deferred tasks. [airflow]

Posted by "eladkal (via GitHub)" <gi...@apache.org>.
eladkal commented on PR #38984:
URL: https://github.com/apache/airflow/pull/38984#issuecomment-2146115372

   Then its up to @ephraimbuddy 
   I think the real question here is if do we think there is value? If so then @tirkarthi will need to change the base branch of this PR from main to v2.9 stable


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Increment try_number while clearing deferred tasks. [airflow]

Posted by "tirkarthi (via GitHub)" <gi...@apache.org>.
tirkarthi commented on PR #38984:
URL: https://github.com/apache/airflow/pull/38984#issuecomment-2091941227

   @dstandish I agree the linked PR is a good long term solution but this PR is good enough for a bugfix in 2.9.x series with 2.10 release few months away.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Increment try_number while clearing deferred tasks. [airflow]

Posted by "ephraimbuddy (via GitHub)" <gi...@apache.org>.
ephraimbuddy commented on PR #38984:
URL: https://github.com/apache/airflow/pull/38984#issuecomment-2146174283

   > Then its up to @ephraimbuddy I think the real question here is if do we think there is value? If so then @tirkarthi will need to change the base branch of this PR from main to v2.9 stable
   
   We can only PR it to v2-9-test but not stable. However, we don't usually do that. We only PR against the test branch if we have issues during the release process and it's mostly CI changes. cc @potiuk 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org