You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "eladkal (via GitHub)" <gi...@apache.org> on 2025/04/28 10:47:26 UTC

[I] Improve error message for Python decorator invoking function with same name as the task name [airflow]

eladkal opened a new issue, #49875:
URL: https://github.com/apache/airflow/issues/49875

   ### Body
   
   Given code:
   
   ```
   from datetime import datetime
   from airflow import DAG
   from airflow.decorators import task
   from time import sleep
   
   default_args = {
       'owner': 'airflow',
       'start_date': datetime(2023, 2, 1)
   }
   
   
   with DAG('scheduler', schedule="@daily", catchup=True, default_args=default_args):
   
       @task()
       def sleep():
           sleep(3600)
       sleep()
   ```
   
   It fails with:
   
   ```
   TypeError: too many positional arguments
   
   File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/task_runner.py", line 829 in run
   
   File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/task_runner.py", line 1092 in _execute_task
   
   File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 408 in wrapper
   
   File "/opt/airflow/task-sdk/src/airflow/sdk/bases/decorator.py", line 251 in execute
   
   File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 408 in wrapper
   
   File "/opt/airflow/providers/standard/src/airflow/providers/standard/operators/python.py", line 212 in execute
   
   File "/opt/airflow/providers/standard/src/airflow/providers/standard/operators/python.py", line 235 in execute_callable
   
   File "/opt/airflow/task-sdk/src/airflow/sdk/execution_time/callback_runner.py", line 81 in run
   
   File "/files/dags/scheduler.py", line 15 in sleep
   
   File "/opt/airflow/task-sdk/src/airflow/sdk/bases/decorator.py", line 363 in __call__
   
   File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 513 in apply_defaults
   
   File "/opt/airflow/providers/standard/src/airflow/providers/standard/decorators/python.py", line 58 in __init__
   
   File "/opt/airflow/task-sdk/src/airflow/sdk/bases/operator.py", line 513 in apply_defaults
   
   File "/opt/airflow/task-sdk/src/airflow/sdk/bases/decorator.py", line 239 in __init__
   
   File "/usr/local/lib/python3.9/inspect.py", line 3045 in bind
   
   File "/usr/local/lib/python3.9/inspect.py", line 2966 in _bind
   
   ```
   
   
   The reason for this failure is because the python task has the exact name as the python function it executes thus Airflow thinks it's recursion. Obviously this is wrong coding but I expected the error to be more informative. I think we need to catch this case and raise user friendly error. My real code was more complex than the example I share here.. it took me several minutes to figure out what was the issue.
   
   <img width="1610" alt="Image" src="https://github.com/user-attachments/assets/6d56a97f-fc77-4743-913a-d34b16654bb0" />
   
   ### Committer
   
   - [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Improve error message for Python decorator invoking function with same name as the task name [airflow]

Posted by "gitbita99 (via GitHub)" <gi...@apache.org>.
gitbita99 commented on issue #49875:
URL: https://github.com/apache/airflow/issues/49875#issuecomment-4209032678

   > > [@ashb](https://github.com/ashb) - Can I take a shot at this?
   > > My plan is to implement it by intercepting function calls in TaskDecorator.**call**, and inspecting the call stack to detect potential self-invocations caused by naming collisions between the function and the task object. If detected, I'll log a clear warning or optionally raise an informative error to help users debug it faster. Let me know if you have any suggestions before I start
   > 
   > Ideally though we do that without inspecting the callstack in "happy" path cases, as looking at the call stack is expensive.
   
   Hi @ashb , I would like to work in this issue
   
   **Problem**: When users shadow imports with task name, they get a cryptic `TypeError: too many positional arguments`.
   
   **Solution**: Add exception handling in **`DecoratedOperator.__init__`** to provide a helpful hint message - only when the error occurs (No overhead).
   
   The fix catches the TypeError during signature binding, adds guidance / hint and returns it.
   
   I have a working implementation ready. Could you please assign this issue to me?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Improve error message for Python decorator invoking function with same name as the task name [airflow]

Posted by "suman-himanshu (via GitHub)" <gi...@apache.org>.
suman-himanshu commented on issue #49875:
URL: https://github.com/apache/airflow/issues/49875#issuecomment-3236634539

   I am able to reproduce this issue at my end.
   
   <img width="1856" height="1074" alt="Image" src="https://github.com/user-attachments/assets/735ff254-8884-4c9d-824d-03dfc737bcb1" />


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Improve error message for Python decorator invoking function with same name as the task name [airflow]

Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on issue #49875:
URL: https://github.com/apache/airflow/issues/49875#issuecomment-3327175654

   > [@ashb](https://github.com/ashb) - Can I take a shot at this?
   > 
   > My plan is to implement it by intercepting function calls in TaskDecorator.**call**, and inspecting the call stack to detect potential self-invocations caused by naming collisions between the function and the task object. If detected, I'll log a clear warning or optionally raise an informative error to help users debug it faster. Let me know if you have any suggestions before I start
   
   Ideally though we do that without inspecting the callstack in "happy" path cases, as looking at the call stack is expensive.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Improve error message for Python decorator invoking function with same name as the task name [airflow]

Posted by "vatsrahul1001 (via GitHub)" <gi...@apache.org>.
vatsrahul1001 commented on issue #49875:
URL: https://github.com/apache/airflow/issues/49875#issuecomment-3233415044

   I can repro this as well


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Improve error message for Python decorator invoking function with same name as the task name [airflow]

Posted by "Gourigupta2004 (via GitHub)" <gi...@apache.org>.
Gourigupta2004 commented on issue #49875:
URL: https://github.com/apache/airflow/issues/49875#issuecomment-4825520706

   Hi @eladkal, 
   
   I would like to investigate and work on this issue. Please assign this to me
   
   Thanks
    


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Improve error message for Python decorator invoking function with same name as the task name [airflow]

Posted by "giannistbs (via GitHub)" <gi...@apache.org>.
giannistbs commented on issue #49875:
URL: https://github.com/apache/airflow/issues/49875#issuecomment-2834929843

   Hello! Can i work on this?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Improve error message for Python decorator invoking function with same name as the task name [airflow]

Posted by "vatsrahul1001 (via GitHub)" <gi...@apache.org>.
vatsrahul1001 closed issue #49875: Improve error message for Python decorator invoking function with same name as the task name
URL: https://github.com/apache/airflow/issues/49875


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Improve error message for Python decorator invoking function with same name as the task name [airflow]

Posted by "ashb (via GitHub)" <gi...@apache.org>.
ashb commented on issue #49875:
URL: https://github.com/apache/airflow/issues/49875#issuecomment-2835899373

   > Hello! Can i work on this?
   
   What is your plan?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Improve error message for Python decorator invoking function with same name as the task name [airflow]

Posted by "kiran2706 (via GitHub)" <gi...@apache.org>.
kiran2706 commented on issue #49875:
URL: https://github.com/apache/airflow/issues/49875#issuecomment-2904382582

   @ashb - Can I take a shot at this?
   
   My plan is to implement it by intercepting function calls in TaskDecorator.__call__, and inspecting the call stack to detect potential self-invocations caused by naming collisions between the function and the task object.
   If detected, I'll log a clear warning or optionally raise an informative error to help users debug it faster.
   Let me know if you have any suggestions before I start
   
   cc: @potiuk 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [I] Improve error message for Python decorator invoking function with same name as the task name [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #49875:
URL: https://github.com/apache/airflow/issues/49875#issuecomment-4829226213

   We do not assign issues (see contributing guides) - but feel free to work on it.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org