You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "m8719-github (via GitHub)" <gi...@apache.org> on 2026/01/02 19:52:18 UTC
[I] Operator template fields via callable serialization caused unstable DAG serialization [airflow]
m8719-github opened a new issue, #60064:
URL: https://github.com/apache/airflow/issues/60064
### Apache Airflow version
3.1.5
### If "Other Airflow 3 version" selected, which one?
_No response_
### What happened?
Operator template fields may be provided via callable of the form: `def fn(context, jinja_env)`
These template field callables are serialized via default `str` which includes current address of the callable: `<function fn at 0x7592eb0cb380>`
The address of the loaded callable changes between iterations of the dag-processor DAG refresh loop and since DAG versioning depends on stable serialization, this leads to DAG version explosion.
### What you think should happen instead?
Operator template fields serialization is treated separately [here](https://github.com/apache/airflow/blob/07ef94e6e966a2bf5d1a159e084a602b30263737/airflow-core/src/airflow/serialization/serialized_objects.py#L1124) and [serialize_template_field](https://github.com/apache/airflow/blob/07ef94e6e966a2bf5d1a159e084a602b30263737/airflow-core/src/airflow/serialization/helpers.py#L32) does not handle callable case.
Instead template field callables should be serialized via already existing logic [here](https://github.com/apache/airflow/blob/07ef94e6e966a2bf5d1a159e084a602b30263737/airflow-core/src/airflow/serialization/serialized_objects.py#L617).
### How to reproduce
Create a simple DAG with a single task with template fields, configure any of the template fields to be provided via callable: `def fn(context, jinja_env)`
Watch DAG versions increment indefinitely.
### Operating System
Ubuntu 24.04.3 LTS
### Versions of Apache Airflow Providers
apache-airflow-providers-amazon==9.18.1
apache-airflow-providers-cncf-kubernetes==10.11.1
apache-airflow-providers-common-compat==1.10.1
apache-airflow-providers-common-io==1.7.0
apache-airflow-providers-common-sql==1.30.1
apache-airflow-providers-databricks==7.8.1
apache-airflow-providers-datadog==3.10.1
apache-airflow-providers-ftp==3.14.0
apache-airflow-providers-http==5.6.1
apache-airflow-providers-mysql==6.4.0
apache-airflow-providers-openlineage==2.9.1
apache-airflow-providers-pagerduty==5.2.1
apache-airflow-providers-sftp==5.5.1
apache-airflow-providers-slack==9.6.1
apache-airflow-providers-smtp==2.4.1
apache-airflow-providers-ssh==4.2.1
apache-airflow-providers-standard==1.10.1
apache-airflow-providers-trino==6.4.1
### Deployment
Virtualenv installation
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [x] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Operator template fields via callable serialization causes unstable DAG serialization [airflow]
Posted by "m8719-github (via GitHub)" <gi...@apache.org>.
m8719-github commented on issue #60064:
URL: https://github.com/apache/airflow/issues/60064#issuecomment-3707257759
> And please - provide a very specific example of the DAG that causes the issue you are mentioning.
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Operator template fields via callable serialization causes unstable DAG serialization [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #60064:
URL: https://github.com/apache/airflow/issues/60064#issuecomment-3707493214
> My approach for https://github.com/apache/airflow/pull/60056 was to have lowest footprint solution, reusing as much as possible of what's already there.
This should not be goal on it's own. Sometimes you need more "footprint" to solve things well
> I do like what's been mentioned in this https://github.com/apache/airflow/discussions/59595#discussioncomment-15373896 where the approach would be to hash the source code and have a compact representation in the serialized DAG, something like callable <MD5_hash_value_of_source_code>
Yes. That looks way more reasonable. especially that often you will not be able to serialize the function because it will have non-serializable local imports for example.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Operator template fields via callable serialization causes unstable DAG serialization [airflow]
Posted by "m8719-github (via GitHub)" <gi...@apache.org>.
m8719-github commented on issue #60064:
URL: https://github.com/apache/airflow/issues/60064#issuecomment-3706194069
@potiuk Could you please take a look at this issue as well as the PR. This is somewhat related to [#59465](https://github.com/apache/airflow/issues/59465)
Much appreciated!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Operator template fields via callable serialization causes unstable DAG serialization [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #60064:
URL: https://github.com/apache/airflow/issues/60064#issuecomment-3706891945
And please - provide a very specific example of the DAG that causes the issue you are mentioning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Operator template fields via callable serialization causes unstable DAG serialization [airflow]
Posted by "m8719-github (via GitHub)" <gi...@apache.org>.
m8719-github commented on issue #60064:
URL: https://github.com/apache/airflow/issues/60064#issuecomment-3707442393
My approach for #60056 was to have lowest footprint solution, reusing as much as possible of what's already there.
However, I do agree that it's likely not the best solution due to the fact that these template var callables can be arbitrarily large and shoving all of their source code into a serialized DAG structure is probably not the best idea; for example: our production DAGs with KPO tasks require a large list of K8s pod annotations to be injected for integration with our Kubernetes observability platform, we inject these annotations via a callable and its source code representation is not small, unnecessarily bloating serialized DAG structure.
I do like what's been mentioned in this [comment](https://github.com/apache/airflow/discussions/59595#discussioncomment-15373896) where the approach would be to hash the source code and have a compact representation in the serialized DAG, something like `callable <MD5_hash_value_of_source_code>`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Operator template fields via callable serialization causes unstable DAG serialization [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #60064:
URL: https://github.com/apache/airflow/issues/60064#issuecomment-3706891458
> [@potiuk](https://github.com/potiuk) Could you please take a look at this issue as well as the PR. This is somewhat related to [#59465](https://github.com/apache/airflow/issues/59465) Much appreciated!
Please don't ping people - they might be (like me) on holidays and by pinging and singling out maintainers, you are making it less likely that someone else will take a look at your PR.
If you look at the contributing guide - there is at least 72 hours when you would expect any reaction of anyone to any PR. So please do not ping people because you are impatient.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Operator template fields via callable serialization caused unstable DAG serialization [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on issue #60064:
URL: https://github.com/apache/airflow/issues/60064#issuecomment-3706123193
Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Operator template fields via callable serialization causes unstable DAG serialization [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on issue #60064:
URL: https://github.com/apache/airflow/issues/60064#issuecomment-3707279606
cc: @ephraimbuddy @jedcunningham @wjddn279: I think that one is an important thing to fix for 3.1.6 -> there does not seem to be any strange patterns or dynamic usage here that should cause instability in serialization - using callable for template var is quite a normal pattern:
From: https://airflow.apache.org/docs/apache-airflow/stable/core-concepts/operators.html#jinja-templating
> You can also pass in a callable instead when Python is more readable than a Jinja template. The callable must accept two named arguments context and jinja_env
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] Operator template fields via callable serialization causes unstable DAG serialization [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #60064: Operator template fields via callable serialization causes unstable DAG serialization
URL: https://github.com/apache/airflow/issues/60064
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org