You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "mik-laj (via GitHub)" <gi...@apache.org> on 2025/04/07 11:25:51 UTC
[I] check-imports-in-providers doesn't detect invalid imports [airflow]
mik-laj opened a new issue, #48870:
URL: https://github.com/apache/airflow/issues/48870
### Apache Airflow version
main (development)
### If "Other Airflow 2 version" selected, which one?
_No response_
### What happened?
Hello,
I have the impression that hook `check-imports-in-providers` does not work properly and does not detect any problems after we started using uv workspace. Internally, this hook uses `ruff analyze graph` command and this command doesn't support uv workspace.
```
$ ruff analyze graph | jq '.[$K]' --arg K "providers/snowflake/src/airflow/providers/snowflake/utils/openlineage.py"
[]
$ ruff analyze graph | jq '.[$K]' --arg K "providers/snowflake/src/airflow/providers/snowflake/version_compat.py"
[
"providers/cohere/src/airflow/__init__.py"
]
```
### What you think should happen instead?
_No response_
### How to reproduce
Make changes as below:
```
diff --git a/providers/snowflake/src/airflow/providers/snowflake/utils/openlineage.py b/providers/snowflake/src/airflow/providers/snowflake/utils/openlineage.py
index 1a3d3f0183..6ed07fb535 100644
--- a/providers/snowflake/src/airflow/providers/snowflake/utils/openlineage.py
+++ b/providers/snowflake/src/airflow/providers/snowflake/utils/openlineage.py
@@ -23,7 +23,7 @@ from typing import TYPE_CHECKING
from urllib.parse import quote, urlparse, urlunparse
from airflow.providers.common.compat.openlineage.check import require_openlineage_version
-from airflow.providers.snowflake.version_compat import AIRFLOW_V_2_10_PLUS, AIRFLOW_V_3_0_PLUS
+from airflow.providers.databricks.version_compat import AIRFLOW_V_2_10_PLUS, AIRFLOW_V_3_0_PLUS
from airflow.utils import timezone
from airflow.utils.state import TaskInstanceState
```
and run
### Operating System
GitHub Codepsaces
### Versions of Apache Airflow Providers
_No response_
### Deployment
Other
### Deployment details
_No response_
### Anything else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [x] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] check-imports-in-providers doesn't detect invalid imports [airflow]
Posted by "mik-laj (via GitHub)" <gi...@apache.org>.
mik-laj commented on issue #48870:
URL: https://github.com/apache/airflow/issues/48870#issuecomment-2783061788
Explicitly passing the filename allows us to detect additional imports, but it still doesn't solve the main problem.
```
$ ruff analyze graph /opt/airflow/providers/snowflake/src/airflow/providers/snowflake/utils/openlineage.py
warning: `ruff analyze graph` is experimental and may change without warning
{
"providers/snowflake/src/airflow/providers/snowflake/utils/openlineage.py": [
"providers/snowflake/src/airflow/providers/snowflake/hooks/snowflake.py",
"providers/snowflake/src/airflow/providers/snowflake/version_compat.py"
]
}```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] check-imports-in-providers doesn't detect invalid imports [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on issue #48870:
URL: https://github.com/apache/airflow/issues/48870#issuecomment-2783125047
Good seeing you old friend!!
And yes we should fix it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] check-imports-in-providers doesn't detect invalid imports [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk closed issue #48870: check-imports-in-providers doesn't detect invalid imports
URL: https://github.com/apache/airflow/issues/48870
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] check-imports-in-providers doesn't detect invalid imports [airflow]
Posted by "mik-laj (via GitHub)" <gi...@apache.org>.
mik-laj commented on issue #48870:
URL: https://github.com/apache/airflow/issues/48870#issuecomment-2783036691
I suspect there may be more problems in `ruff analyze graph`.
For example, this command returns empty list
```
$ ruff analyze graph | jq '.[$K]' --arg K "providers/snowflake/src/airflow/providers/snowflake/utils/openlineage.py"
warning: `ruff analyze graph` is experimental and may change without warning
[]
```
But we have imports in this file:
```
from __future__ import annotations
import datetime
import logging
from contextlib import closing
from typing import TYPE_CHECKING
from urllib.parse import quote, urlparse, urlunparse
from airflow.providers.common.compat.openlineage.check import require_openlineage_version
from airflow.providers.snowflake.version_compat import AIRFLOW_V_2_10_PLUS, AIRFLOW_V_3_0_PLUS
from airflow.utils import timezone
from airflow.utils.state import TaskInstanceState
if TYPE_CHECKING:
from openlineage.client.event_v2 import RunEvent
from openlineage.client.facet_v2 import JobFacet
from airflow.providers.snowflake.hooks.snowflake import SnowflakeHook
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org