You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "jabbera (via GitHub)" <gi...@apache.org> on 2025/09/11 17:29:04 UTC
[I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
jabbera opened a new issue, #55524:
URL: https://github.com/apache/airflow/issues/55524
### Apache Airflow version
3.0.6
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
The following code:
```
try:
from airflow.sdk import task, DAG
except ImportError:
from airflow.decorators import task
from airflow.models import DAG
@task()
def test_task(start_date, required_arg) -> None:
pass
with DAG(dag_id="example_dag") as dag:
test_task("2023-01-01", "value")
```
Generates this error:
```
The function signature broke while assigning defaults to context key parameters.
The decorator is replacing the signature
> test_task(start_date, required_arg)
with
> test_task(start_date=None, required_arg)
which isn't valid: non-default argument follows default argument.
```
This works fine in airflow 2.10.
Note: we happen to have a function argument named start_date here, we are NOT trying to use the template variable here.
The error is generated because the decorator indiscriminately sets a default (None) for every parameter whose name matches a template variable:
https://github.com/apache/airflow/blob/c9100bbbc60fd886691309b4bafc9069b7abbf0f/task-sdk/src/airflow/sdk/bases/decorator.py#L209-L212
IMO: The default should only be set if there isn't a positional or named parameter that already sets a value for a given parameter.
Fixing this on our end would be monumental because it would require the rename of quite a few parameters across our entire ecosystem.
### What you think should happen instead?
The dag should behave like airflow 2.X
### How to reproduce
See above
### Operating System
Ubuntu 24,.04 or WSL debian
### Versions of Apache Airflow Providers
_No response_
### Deployment
Virtualenv installation
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "jabbera (via GitHub)" <gi...@apache.org>.
jabbera commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3282213380
Thanks! I'll see which approach is easier for us.
> Now, I'm not sure why this is first popping up in Airflow 3.0
It's because start_date is a new template field as of 3.0. Like I said, probably a bad choice since any template field essentially reserves that parameter name and it's incredibly likely to collide with others. Given it's use task_start_date probably would have been a choice that would have been less likely to collide.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3286281710
BTW. Same reason why you would not name your python module `email` for example - our docs also suggest to use your specific packages (namespaces) specific for your case.
It's really the qustion - shoudl **everything be more complex** to make it easier to add "commonly named" parameters occasionally or rather make **everything less complex** at the expense of occasionally breaking things that is super easy to fix (and have good error message).
I am definitely for the latter.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3282121545
I would suggest something like `first_date` or `beginning_date`, since `start_date` cannot be used.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3282210442
@jabbera - `start_date` was added to `KNOWN_CONTEXT_KEYS` as a part of this commit (https://github.com/apache/airflow/commit/0047a6886a12478dc30fe76e7192fc837b118001), which was eventually brought into Airflow 3.0. That's why you're now seeing this issue in Airflow 3.0.6.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "jerryqhyu (via GitHub)" <gi...@apache.org>.
jerryqhyu commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3285762702
Echoing @jabbera here. There’s no reason why any airflow context variables should collide with user code. They can all be named arbitrarily like __airflow_task_start_date_☺ or something and you would never have problems like these.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3286273031
> Echoing [@jabbera](https://github.com/jabbera) here. There’s no reason why any airflow context variables should collide with user code. They can all be named arbitrarily like __airflow_task_start_date_☺ or something and you would never have problems like these.
One could also argue that anyone introducing their argument with "their" namespace. For example prefix it with their business specific prefix: `my_test_start_date` in this case for example.
One of the reson reason is that it's historically like that and way easier to use and difficult to change otherwise. Occasional namespace breakage are not worth adding such complexity IMHO. But if you have a way how to introduce it without breaking millions of Dags here and have a good UX, you are absolutely welcome to contribute such change.
But I highly doubt it's possible. For now possibly a PR to make error more obvious is a good idea (and also contributions are welcome).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3282112466
@jabbera, `start_date` is a "reserved" field. It's used to determine the `start_date` for that Task (believe it or not, each Task can have its own `start_date`). I would suggest changing the parameter name.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3282094532
@jabbera, are you trying to specify a `start_date` for the Task, or just have a parameter called `start_date`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3282233317
Happy to help - I'm with you, I don't love that this logic is somewhat hidden and has a misleading error message. I'm thinking about opening up a PR to see if we can create a more clear exception.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3282178409
@jabbera, I'm going to reproduce this on my side. But it seems like switching the order of `start_date` and `required_arg` would do the trick. Reproducing now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "jabbera (via GitHub)" <gi...@apache.org>.
jabbera commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3282121870
It's not super collision resistant. task_start_date might have been a better choice but I guess that ship has sailed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "jabbera (via GitHub)" <gi...@apache.org>.
jabbera closed issue #55524: Airflow 3.0 taskflow can't have a positional argument named after a template variable
URL: https://github.com/apache/airflow/issues/55524
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "jroachgolf84 (via GitHub)" <gi...@apache.org>.
jroachgolf84 commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3282199454
Switching the order of `start_date` and `required_arg` seems to have done the trick. Here's what's going on:
* That code snippet you sent is actually making `start_date` a keyword argument vs. a positional argument.
* Python has a convention that positional arguments proceed keyword arguments.
Now, I'm not sure why this is first popping up in Airflow 3.0... What would the lift be to make a change like this? There are two options:
1. Flip the order of `start_date` and `required_arg`.
2. Add a default value for `required_arg` (which, makes it no longer required). However, you could add logic to catch that in the function.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "jabbera (via GitHub)" <gi...@apache.org>.
jabbera commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3282106191
@jroachgolf84 it's a parameter start_date that is an xcom from another task.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Airflow 3.0 taskflow can't have a positional argument named after a template variable [airflow]
Posted by "jabbera (via GitHub)" <gi...@apache.org>.
jabbera commented on issue #55524:
URL: https://github.com/apache/airflow/issues/55524#issuecomment-3282131372
> I would suggest something like `first_date` or `beginning_date`, since `start_date` cannot be used.
I've got 50k lines of code with 100's of methods that have the parameter start_date. It's not that easy.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org