You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by "kaxil (via GitHub)" <gi...@apache.org> on 2025/10/18 01:57:07 UTC
[PR] Simplify version-specific imports in the Google provider [airflow]
kaxil opened a new pull request, #56793:
URL: https://github.com/apache/airflow/pull/56793
Simplify version-specific imports in the Google provider by using lazy_compat from common.compat provider added in https://github.com/apache/airflow/pull/56790
for components that are available since Airflow 3.0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #56793:
URL: https://github.com/apache/airflow/pull/56793#discussion_r2442143494
##########
providers/common/compat/pyproject.toml:
##########
@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
[project]
name = "apache-airflow-providers-common-compat"
-version = "1.7.4"
+version = "1.8.0"
Review Comment:
We discussed (and followed) with @eladkal that non-released should **NOT** upgrade version of common.provider in other places than during release prepaaration. While we do not have a final "robust" mechanism for that, I propose that we do it this way now:
* keep 1.7.4 here (and in the version below)
* in the google provider add ">=1.7.4 # TODO: bump to next version"
Then release manager (we can automate that) release manager will have to remember to bump all such version to the next version of common provider.
The main reason for that is that it makes it difficult for the release manager who manually bumps all the versions of providers and prepares release notes - once the version is dumped, our mechanism of deriving the next version to bump and generate properly grouped release notes does not work well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #56793:
URL: https://github.com/apache/airflow/pull/56793#discussion_r2442159166
##########
providers/common/compat/pyproject.toml:
##########
@@ -76,7 +76,6 @@ dev = [
"apache-airflow-task-sdk",
"apache-airflow-devel-common",
"apache-airflow-providers-openlineage",
- "apache-airflow-providers-standard",
Review Comment:
Ah. Because of providers/common/compat/src/airflow/providers/common/compat/standard/operators.py cleanup. All good.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #56793:
URL: https://github.com/apache/airflow/pull/56793#issuecomment-3418623434
> > If so, will we create meta issue for migration and invite the community to help?
>
> Not yet :) But eventually after this PR is merged and probably one or two critical ones.
Yeah.... Do it once, do it twice, figure out if repeatable and document -> then fire it in massive parallel way. Parallelising too early might result in parallleling wrong solution.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #56793:
URL: https://github.com/apache/airflow/pull/56793#discussion_r2442433644
##########
providers/google/pyproject.toml:
##########
@@ -58,7 +58,7 @@ requires-python = ">=3.10"
# After you modify the dependencies, and rebuild your Breeze CI image with ``breeze ci-image build``
dependencies = [
"apache-airflow>=2.10.0",
- "apache-airflow-providers-common-compat>=1.4.0",
+ "apache-airflow-providers-common-compat>=1.8.0",
Review Comment:
Done. fyi @eladkal for when you release the provider
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on PR #56793:
URL: https://github.com/apache/airflow/pull/56793#issuecomment-3418452035
>If so, will we create meta issue for migration and invite the community to help?
Not yet :) But eventually after this PR is merged and probably one or two critical ones.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #56793:
URL: https://github.com/apache/airflow/pull/56793#discussion_r2442154893
##########
providers/common/compat/pyproject.toml:
##########
@@ -76,7 +76,6 @@ dev = [
"apache-airflow-task-sdk",
"apache-airflow-devel-common",
"apache-airflow-providers-openlineage",
- "apache-airflow-providers-standard",
Review Comment:
This is interestting -> I am not sure why standard is not detected any more as a dependency (this is aiutomatically generated) ... Strange :) ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #56793:
URL: https://github.com/apache/airflow/pull/56793#discussion_r2442143494
##########
providers/common/compat/pyproject.toml:
##########
@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
[project]
name = "apache-airflow-providers-common-compat"
-version = "1.7.4"
+version = "1.8.0"
Review Comment:
We discussed (and followed) with @eladkal that non-released should **NOT** upgrade version of common.provider in other places than during release prepaaration. While we do not have a final "robust" mechanism for that, I propose that we do it this way now:
* keep 1.7.4 here (and in all the versions below
* in the google provider use `apache-airflow-providers-common-compat >=1.7.4", # TODO: bump to next version`
Then release manager (we can automate that) release manager will have to remember to bump all such version to the next version of common provider.
The main reason for that is that it makes it difficult for the release manager who manually bumps all the versions of providers and prepares release notes - once the version is dumped, our mechanism of deriving the next version to bump and generate properly grouped release notes does not work well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #56793:
URL: https://github.com/apache/airflow/pull/56793#discussion_r2442143494
##########
providers/common/compat/pyproject.toml:
##########
@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
[project]
name = "apache-airflow-providers-common-compat"
-version = "1.7.4"
+version = "1.8.0"
Review Comment:
We discussed (and followed) with @eladkal that non-released should **NOT** upgrade version of common.provider in other places than during release prepaaration. While we do not have a final "robust" mechanism for that, I propose that we do it this way now:
* keep 1.7.4 here (and in the version below)
* in the google provider use `apache-airflow-providers-common-compat >=1.7.4", # TODO: bump to next version`
Then release manager (we can automate that) release manager will have to remember to bump all such version to the next version of common provider.
The main reason for that is that it makes it difficult for the release manager who manually bumps all the versions of providers and prepares release notes - once the version is dumped, our mechanism of deriving the next version to bump and generate properly grouped release notes does not work well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #56793:
URL: https://github.com/apache/airflow/pull/56793#discussion_r2454110293
##########
providers/google/src/airflow/providers/google/cloud/links/base.py:
##########
@@ -20,17 +20,8 @@
from typing import TYPE_CHECKING, ClassVar
from urllib.parse import urlparse
-from airflow.providers.google.version_compat import (
- AIRFLOW_V_3_0_PLUS,
- BaseOperator,
- BaseOperatorLink,
- BaseSensorOperator,
-)
-
-if AIRFLOW_V_3_0_PLUS:
- from airflow.sdk.execution_time.xcom import XCom
-else:
- from airflow.models.xcom import XCom # type: ignore[no-redef]
+from airflow.providers.common.compat.lazy_compat import BaseOperatorLink, BaseSensorOperator, XCom
Review Comment:
I remember now :)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #56793:
URL: https://github.com/apache/airflow/pull/56793#discussion_r2442434476
##########
providers/common/compat/pyproject.toml:
##########
@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
[project]
name = "apache-airflow-providers-common-compat"
-version = "1.7.4"
+version = "1.8.0"
Review Comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil merged PR #56793:
URL: https://github.com/apache/airflow/pull/56793
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #56793:
URL: https://github.com/apache/airflow/pull/56793#discussion_r2442143494
##########
providers/common/compat/pyproject.toml:
##########
@@ -25,7 +25,7 @@ build-backend = "flit_core.buildapi"
[project]
name = "apache-airflow-providers-common-compat"
-version = "1.7.4"
+version = "1.8.0"
Review Comment:
We discussed (and followed) with @eladkal that non-released should **NOT** upgrade version of common.provider in other places than during release prepaaration. While we do not have a final "robust" mechanism for that, I propose that we do it this way now:
* keep 1.7.4 here (and in the version below)
* in the google provider add ">=1.7.4 # +next version"
Then release manager (we can automate that) release manager will have to remember to bump all such version to the next version of common provider.
The main reason for that is that it makes it difficult for the release manager who manually bumps all the versions of providers and prepares release notes - once the version is dumped, our mechanism of deriving the next version to bump and generate properly grouped release notes does not work well
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on code in PR #56793:
URL: https://github.com/apache/airflow/pull/56793#discussion_r2442164589
##########
providers/google/pyproject.toml:
##########
@@ -58,7 +58,7 @@ requires-python = ">=3.10"
# After you modify the dependencies, and rebuild your Breeze CI image with ``breeze ci-image build``
dependencies = [
"apache-airflow>=2.10.0",
- "apache-airflow-providers-common-compat>=1.4.0",
+ "apache-airflow-providers-common-compat>=1.8.0",
Review Comment:
```suggestion
"apache-airflow-providers-common-compat>=1.7.4", # + TODO: bump to next version
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "potiuk (via GitHub)" <gi...@apache.org>.
potiuk commented on PR #56793:
URL: https://github.com/apache/airflow/pull/56793#issuecomment-3418220716
> If so, will we create meta issue for migration and invite the community to help?
Once we get this one in as an example I guess :) ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "amoghrajesh (via GitHub)" <gi...@apache.org>.
amoghrajesh commented on code in PR #56793:
URL: https://github.com/apache/airflow/pull/56793#discussion_r2450490515
##########
providers/google/src/airflow/providers/google/cloud/links/base.py:
##########
@@ -20,17 +20,8 @@
from typing import TYPE_CHECKING, ClassVar
from urllib.parse import urlparse
-from airflow.providers.google.version_compat import (
- AIRFLOW_V_3_0_PLUS,
- BaseOperator,
- BaseOperatorLink,
- BaseSensorOperator,
-)
-
-if AIRFLOW_V_3_0_PLUS:
- from airflow.sdk.execution_time.xcom import XCom
-else:
- from airflow.models.xcom import XCom # type: ignore[no-redef]
+from airflow.providers.common.compat.lazy_compat import BaseOperatorLink, BaseSensorOperator, XCom
Review Comment:
Can we not import BaseOperator here too?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [PR] Simplify version-specific imports in the Google provider [airflow]
Posted by "kaxil (via GitHub)" <gi...@apache.org>.
kaxil commented on code in PR #56793:
URL: https://github.com/apache/airflow/pull/56793#discussion_r2451940072
##########
providers/google/src/airflow/providers/google/cloud/links/base.py:
##########
@@ -20,17 +20,8 @@
from typing import TYPE_CHECKING, ClassVar
from urllib.parse import urlparse
-from airflow.providers.google.version_compat import (
- AIRFLOW_V_3_0_PLUS,
- BaseOperator,
- BaseOperatorLink,
- BaseSensorOperator,
-)
-
-if AIRFLOW_V_3_0_PLUS:
- from airflow.sdk.execution_time.xcom import XCom
-else:
- from airflow.models.xcom import XCom # type: ignore[no-redef]
+from airflow.providers.common.compat.lazy_compat import BaseOperatorLink, BaseSensorOperator, XCom
Review Comment:
No, BaseOperator is special cased for 3.1 due to xcom_push bug -- there should be a comment about it in version_compat file
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@airflow.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org