You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "Anuragp22 (via GitHub)" <gi...@apache.org> on 2026/05/26 03:15:44 UTC
[PR] Lazy load neo4j driver imports [airflow]
Anuragp22 opened a new pull request, #67522:
URL: https://github.com/apache/airflow/pull/67522
Lazy-load `neo4j` SDK imports to reduce memory footprint and import time of `airflow.providers.neo4j.hooks.neo4j`, following the same pattern as #62365 (snowflake) and #67519 (elasticsearch).
### Hook changes
- Removed the top-level `from neo4j import Driver, GraphDatabase`.
- `Driver` is only used as a type annotation, so it now lives under `if TYPE_CHECKING:`.
- `GraphDatabase.driver(...)` is only invoked inside `_create_driver`, so the import is moved into that method body.
### Test changes
- Updated five `mock.patch` targets in `test_neo4j.py` from `airflow.providers.neo4j.hooks.neo4j.GraphDatabase` to `neo4j.GraphDatabase` (and `...GraphDatabase.driver` for the parametrized test), since `GraphDatabase` is no longer a module-level symbol in the hook module.
### Validation
Local smoke check (Windows host; full unit suite requires the Linux-only `fcntl` import that Airflow's test conftest pulls in transitively):
- Importing `Neo4jHook` no longer loads the `neo4j` package into `sys.modules`.
- `Driver` and `GraphDatabase` are no longer attributes of the hook module.
- `neo4j.GraphDatabase` and `neo4j.GraphDatabase.driver` resolve as `mock.patch` targets.
The unit tests in `providers/neo4j/tests/unit/neo4j/hooks/test_neo4j.py` should now pass against the moved import; CI will be the source of truth here.
* related: #67515
---
##### Was generative AI tooling used to co-author this PR?
- [x] Yes — Claude (Opus) was used for the mechanical edits and the announcement comment on #67515. Reviewed by hand against the pattern in #62365 and #67519.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Lazy load neo4j driver imports [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil closed pull request #67522: Lazy load neo4j driver imports
URL: https://github.com/apache/airflow/pull/67522
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Lazy load neo4j driver imports [airflow]
Posted by "boring-cyborg[bot] (via GitHub)" <gi...@apache.org>.
boring-cyborg[bot] commented on PR #67522:
URL: https://github.com/apache/airflow/pull/67522#issuecomment-4539663893
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our [Contributors' Guide](https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
Here are some useful points:
- Pay attention to the quality of your code (ruff, mypy and type annotations). Our [prek-hooks]( https://github.com/apache/airflow/blob/main/contributing-docs/08_static_code_checks.rst#prerequisites-for-prek-hooks) will help you with that.
- In case of a new feature add useful documentation (in docstrings or in `docs/` directory). Adding a new operator? Check this short [guide](https://github.com/apache/airflow/blob/main/airflow-core/docs/howto/custom-operator.rst) Consider adding an example Dag that shows how users should use it.
- Consider using [Breeze environment](https://github.com/apache/airflow/blob/main/dev/breeze/doc/README.rst) for testing locally, it's a heavy docker but it ships with a working Airflow and a lot of integrations.
- Be patient and persistent. It might take some time to get a review or get the final approval from Committers.
- Please follow [ASF Code of Conduct](https://www.apache.org/foundation/policies/conduct) for all communication including (but not limited to) comments on Pull Requests, Mailing list and Slack.
- Be sure to read the [Airflow Coding style]( https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#coding-style-and-best-practices).
- Always keep your Pull Requests rebased, otherwise your build might fail due to changes not related to your commits.
Apache Airflow is a community-driven project and together we are making it better 🚀.
In case of doubts contact the developers at:
Mailing List: dev@airflow.apache.org
Slack: https://s.apache.org/airflow-slack
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Lazy load neo4j driver imports [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on PR #67522:
URL: https://github.com/apache/airflow/pull/67522#issuecomment-4548350137
Hi @Anuragp22, thank you for the focused work here -- the engineering on this PR is clean.
I'm closing this along with the other PRs targeting #67515 after thinking through the actual savings more carefully. Full reasoning here: https://github.com/apache/airflow/issues/67515#issuecomment-4548272652
Short version: an Operator imports a Hook which imports the SDK, so moving the SDK import inside a method mostly shifts *when* the import happens rather than saving it -- workers running the task pay the same cost. It also loses the parse-time `import_error` surface that today reports install/version issues to users. PEP 810 (lazy imports, Python 3.15) is the cleaner path and the polyfill works alongside it without source-level refactors.
Genuinely sorry for the wasted cycles -- on us for not flagging the strategy concern before contributors started shipping.
Two areas that *are* worth the work if you want something else to pick up:
1. `TYPE_CHECKING` guards on type-only imports across providers. Pure win, no trade-off.
2. Lazy-loading conditionally-used heavy deps (e.g. `pandas_gbq` inside `BigQueryHook.get_pandas_df` rather than at module level) -- the fail-fast loss is scoped to methods users may never call.
Thanks again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org