[GitHub] [airflow] weiwenlan commented on a diff in pull request #30571: Issue on the try_number is not in the loop

["weiwenlan (via GitHub)" <gi...@apache.org>]

weiwenlan commented on code in PR #30571:
URL: https://github.com/apache/airflow/pull/30571#discussion_r1163632571


##########
airflow/sensors/base.py:
##########
@@ -251,7 +251,7 @@ def run_duration() -> float:
                 raise AirflowRescheduleException(reschedule_date)
             else:
                 time.sleep(self._get_next_poke_interval(started_at, run_duration, try_number))
-                try_number += 1
+            try_number += 1

Review Comment:
   Hi @hussein-awala, this is just put the try_number outside the if logic. For now, if reschedule is true, the `try_number` will always be 1 and the exponential_backoff will be of no use. The exponential_backoff parameter is used in the function: `_get_next_poke_interval`  and it will use `try_number` to generate the backoff interval. If the `try_number`  remains 1, the interval will not change at all, even if the exponential_backoff is set to True. Show some code scripts below.
   
   ```
   # code in the execute()
   if self.reschedule:
       next_poke_interval = self._get_next_poke_interval(started_at, run_duration, try_number)
       reschedule_date = timezone.utcnow() + timedelta(seconds=next_poke_interval)
       if _is_metadatabase_mysql() and reschedule_date > _MYSQL_TIMESTAMP_MAX:
           raise AirflowSensorTimeout(
               f"Cannot reschedule DAG {log_dag_id} to {reschedule_date.isoformat()} "
               f"since it is over MySQL's TIMESTAMP storage limit."
           )
       raise AirflowRescheduleException(reschedule_date)
   else:
       time.sleep(self._get_next_poke_interval(started_at, run_duration, try_number))
       try_number += 1
   ```
   
   ```
   # code about determine the poke interval
   def _get_next_poke_interval(
           self,
           started_at: datetime.datetime | float,
           run_duration: Callable[[], float],
           try_number: int,
       ) -> float:
           """Using the similar logic which is used for exponential backoff retry delay for operators."""
           if not self.exponential_backoff:
               return self.poke_interval
   
           min_backoff = int(self.poke_interval * (2 ** (try_number - 2)))
   ```
   



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org