You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "potiuk (via GitHub)" <gi...@apache.org> on 2026/03/11 23:26:47 UTC

[PR] Fix datafusion import breaking provider yaml validation [airflow]

potiuk opened a new pull request, #63395:
URL: https://github.com/apache/airflow/pull/63395

   The `object_storage_provider.py` in the `common.sql` provider has top-level imports from `datafusion.object_store` which is not a required dependency. This breaks the `Validate provider.yaml files` CI check because the module fails to import when `datafusion` is not installed.
   
   Move the `from datafusion.object_store import AmazonS3` and `from datafusion.object_store import LocalFileSystem` imports inside their respective `create_object_store()` methods where they are actually used, following the lazy-import pattern already used elsewhere in the codebase.
   
   ---
   
   ##### Was generative AI tooling used to co-author this PR?
   
   - [X] Yes — Claude Code (Claude Opus 4.6)
   
   Generated-by: Claude Code (Claude Opus 4.6) following [the guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#gen-ai-assisted-contributions)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix datafusion import breaking provider yaml validation [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #63395:
URL: https://github.com/apache/airflow/pull/63395#issuecomment-4042924329

   > yeah good to move this but not sure that does fix the issue, because there is session context imported in the `DataFusionEngine` class .. so we likely move Engine import complete to inside property field . https://github.com/apache/airflow/pull/63393/changes
   
   Let's see :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix datafusion import breaking provider yaml validation [airflow]

Posted by "gopidesupavan (via GitHub)" <gi...@apache.org>.
gopidesupavan commented on PR #63395:
URL: https://github.com/apache/airflow/pull/63395#issuecomment-4042921292

   yeah good to move this but not sure that does fix the issue, because there is session context imported in the `DataFusionEngine` class ..
   so we likely move Engine import complete to inside property field . https://github.com/apache/airflow/pull/63393/changes


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix datafusion import breaking provider yaml validation [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #63395:
URL: https://github.com/apache/airflow/pull/63395#issuecomment-4042946752

   Yeah. not helping .. @gopidesupavan - yours is better :)


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org


Re: [PR] Fix datafusion import breaking provider yaml validation [airflow]

Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed pull request #63395: Fix datafusion import breaking provider yaml validation
URL: https://github.com/apache/airflow/pull/63395


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org

For queries about this service, please contact Infrastructure at:
users@infra.apache.org