[GitHub] [airflow] elchao96 opened a new issue, #29903: Task-level retries overrides from the DAG-level default args are not respected when using `partial`

["elchao96 (via GitHub)" <gi...@apache.org>]

elchao96 opened a new issue, #29903:
URL: https://github.com/apache/airflow/issues/29903

   ### Apache Airflow version
   
   Other Airflow 2 version (please specify below)
   
   ### What happened
   
   When running a DAG that is structured like:
   ```
   @dag{dag_id="my_dag", default_args={"retries":0"}}
   def dag():
       op = MyOperator.partial(task_id="my_task", retries=3).expand(...)
   ```
   
   The following test fails:
   ```
       def test_retries(self) -> None:
           dag_bag = DagBag(dag_folder=DAG_FOLDER, include_examples=False)
           dag = dag_bag.dags["my_dag"]
           for task in dag.tasks:
               if "my_task" in task.task_id:
                   self.assertEqual(3, task.retries) # fails - this is 0
   ```
   
   When printing out `task.partial_kwargs`, and looking at how the default args and partial args are merged, it seems like the default args are always taking precedence, even though in the `partial` global function, the `retries` do get set later on with the task-level parameter value. This doesn't seem to be respected though.
   
   ### What you think should happen instead
   
   _No response_
   
   ### How to reproduce
   
   If you run my above unit test for a test DAG, on version 2.4.3, it should show up as a test failure.
   
   ### Operating System
   
   OS Ventura
   
   ### Versions of Apache Airflow Providers
   
   _No response_
   
   ### Deployment
   
   Google Cloud Composer
   
   ### Deployment details
   
   _No response_
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] boring-cyborg[bot] commented on issue #29903: Task-level retries overrides from the DAG-level default args are not respected when using `partial`

["boring-cyborg[bot] (via GitHub)" <gi...@apache.org>]

boring-cyborg[bot] commented on issue #29903:
URL: https://github.com/apache/airflow/issues/29903#issuecomment-1454014615

   Thanks for opening your first issue here! Be sure to follow the issue template!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] hussein-awala closed issue #29903: Task-level retries overrides from the DAG-level default args are not respected when using `partial`

["hussein-awala (via GitHub)" <gi...@apache.org>]

hussein-awala closed issue #29903: Task-level retries overrides from the DAG-level default args are not respected when using `partial`
URL: https://github.com/apache/airflow/issues/29903


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org

[GitHub] [airflow] hussein-awala commented on issue #29903: Task-level retries overrides from the DAG-level default args are not respected when using `partial`

["hussein-awala (via GitHub)" <gi...@apache.org>]

hussein-awala commented on issue #29903:
URL: https://github.com/apache/airflow/issues/29903#issuecomment-1454256057

   I can reproduce on master:
   ```python
   import pendulum
   from airflow.decorators import dag
   from airflow.operators.bash import BashOperator
   
   
   @dag(
       dag_id="test_default_args_with_partial",
       start_date=pendulum.datetime(2023, 1, 1),
       catchup=False,
       default_args={"retries": 0},
   )
   def dag():
       op = BashOperator.partial(task_id="my_task", retries=3).expand(bash_command=["exit 1", "ps"])
   
   
   dag()
   ```
   The retries of the mapped tasks is 0.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org