You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "kyungjunleeme (via GitHub)" <gi...@apache.org> on 2025/08/12 07:02:26 UTC
[D] Is the `json` override in `airflow_local_settings.py` still relevant? [airflow]
GitHub user kyungjunleeme created a discussion: Is the `json` override in `airflow_local_settings.py` still relevant?
Hello. I’m wondering if this approach
https://airflow.apache.org/docs/apache-airflow/stable/administration-and-deployment/dag-serialization.html#using-a-different-json-library
is still relevant or useful in real-world Airflow deployments.
## Why it might have limited value
1. **Limited scope** – This override only affects Airflow components that import `json` from `airflow_local_settings`. Many operators, hooks, and provider packages import Python’s built-in `json` module directly, so they wouldn’t be impacted.
2. **Minimal performance gains** – In most Airflow environments, performance bottlenecks are related to I/O (e.g., DB queries, network calls, XCom storage) rather than JSON parsing speed. Switching to something like `ujson` may not yield noticeable improvements.
3. **Compatibility risks** – Alternative libraries can behave differently (e.g., handling of `datetime`, `Decimal`, `NaN`), which might cause subtle bugs or inconsistencies, especially in XCom serialization.
## Questions for the community
- Is this override mechanism still actively used or tested in Airflow core?
- Should we consider clarifying in the docs that it only affects a limited part of the codebase?
- Would it make sense to deprecate this recommendation in favor of patching JSON handling directly in custom hooks/operators if needed?
I’d love to hear thoughts from the community and maintainers.
If the consensus is that this is no longer valuable, perhaps we could update the docs to reflect that.
GitHub link: https://github.com/apache/airflow/discussions/54387
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Is the `json` override in `airflow_local_settings.py` still relevant? [airflow]
Posted by "kyungjunleeme (via GitHub)" <gi...@apache.org>.
GitHub user kyungjunleeme added a comment to the discussion: Is the `json` override in `airflow_local_settings.py` still relevant?
I’ve looked into this and confirmed that this part is valid. Rather, I think it would be better to add an explanation in the documentation about where it can be used, so that users can understand it more easily.
- https://github.com/apache/airflow/pull/6630/
- https://github.com/penguinolog/sqlalchemy_jsonfield
GitHub link: https://github.com/apache/airflow/discussions/54387#discussioncomment-14171554
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Is the `json` override in `airflow_local_settings.py` still relevant? [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
GitHub user potiuk added a comment to the discussion: Is the `json` override in `airflow_local_settings.py` still relevant?
I don't even know why it's there. IMHO we should remove it - maybe you could track the history of it by git history and you will find more explanations and you can ask the right person who did it and why - because I doubt there is even a general knowledge that this paragraph is there.
GitHub link: https://github.com/apache/airflow/discussions/54387#discussioncomment-14082005
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org
Re: [D] Is the `json` override in `airflow_local_settings.py` still relevant? [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
GitHub user potiuk added a comment to the discussion: Is the `json` override in `airflow_local_settings.py` still relevant?
> I’ve looked into this and confirmed that this part is valid. Rather, I think it would be better to add an explanation in the documentation about where it can be used, so that users can understand it more easily.
>
> * [[AIRFLOW-5947] Make the json backend pluggable for DAG Serialization #6630](https://github.com/apache/airflow/pull/6630)
> * https://github.com/penguinolog/sqlalchemy_jsonfield
Sounds good :)
GitHub link: https://github.com/apache/airflow/discussions/54387#discussioncomment-14180430
----
This is an automatically sent email for commits@airflow.apache.org.
To unsubscribe, please send an email to: commits-unsubscribe@airflow.apache.org