You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@airflow.apache.org by GitBox <gi...@apache.org> on 2021/04/01 09:07:08 UTC
[GitHub] [airflow] dinigo opened a new issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
dinigo opened a new issue #15129:
URL: https://github.com/apache/airflow/issues/15129
**Apache Airflow version**:
1.10.14 and 1.10.15
**Environment**
Official Docker image python 3.6
**What happened**:
The backport package `apache-airflow-backport-providers-google` has a dependency on `google-api-core`:
* [apache-airflow-backport-providers-google](https://github.com/apache/airflow/blob/a7a558eade11ff43a7b96544dcfb84be2441fdc0/setup.py#L285) — `>=1.25.1,<2.0.0`
* [airflow 1.10.14 constraints for python 3.6](https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.6.txt) — `google-api-core==1.23.0`
Those two do not overlap in any version. Setting `google-api-core==1.26.3` will make it solve the dependencies again
**What you expected to happen**:
Should be posible to install airflow with the current constraints.
```
pip install apache-airflow-backport-providers-google -c https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.6.txt
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk commented on issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #15129:
URL: https://github.com/apache/airflow/issues/15129#issuecomment-812369833
@dinigo. Everything is as expected.
Constraints are frozen at the time of creating the `apache-airflow` package. And they are there to be able to install the given version of `apache-airflow` in a consistent way. No more, no less. They have nothing to do with the version of provider that you can install afterwards. Constraints finish their role when you have apache-airflow installed, and then you should manage your additional packages (including backport providers) on your own.
There is (and cannot be) absolutely no relation of the constraints with version 1.10.14 to the providers (which are released on a completely dfferent schedule).
You should not use constraints to install providers, this is neither endorsed not supported. You should install providers (or backport providers) separately
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] ashb commented on issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
ashb commented on issue #15129:
URL: https://github.com/apache/airflow/issues/15129#issuecomment-811855818
> The `>=1.25.1,<2.0.0` is in master branch not in 1.10 branch.
> I would imagine that the constrains for 1.10.14 are not created from master but i might be wrong here.
You are correct.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dinigo commented on issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
dinigo commented on issue #15129:
URL: https://github.com/apache/airflow/issues/15129#issuecomment-811825510
I've placed a link to the `setup.py` with a range of versions, and then the constrain generated lays outside of this range. I thought they where generated from the `setup.py` until I saw this:
* [setup.py](https://github.com/apache/airflow/blob/a7a558eade11ff43a7b96544dcfb84be2441fdc0/setup.py#L285) — `>=1.25.1,<2.0.0`
* [airflow 1.10.14 constraints for python 3.6](https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.6.txt) — `==1.23.0`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dinigo commented on issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
dinigo commented on issue #15129:
URL: https://github.com/apache/airflow/issues/15129#issuecomment-812658691
@potiuk much clearer. Thanks
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #15129:
URL: https://github.com/apache/airflow/issues/15129#issuecomment-812369833
@dinigo. Everything is as expected.
Constraints are frozen at the time of creating the `apache-airflow` package. And they are there to be able to install the given version of `apache-airflow` in a consistent way. No more, no less. They have nothing to do with the version of provider that you can install afterwards. Constraints finish their role when you have apache-airflow installed, and then you should manage your additional packages (including backport providers) on your own.
There is (and cannot be) absolutely no relation of the constraints with version 1.10.14 to the providers/backport providers (which are released on a completely dfferent schedule).
You should not use constraints to install providers, this is neither endorsed not supported. You should install providers (or backport providers) separately, without constraints.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dinigo commented on issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
dinigo commented on issue #15129:
URL: https://github.com/apache/airflow/issues/15129#issuecomment-811777104
Same issue with `snowflake-connector-python==2.3.6` in the constraints but `snowflake-connector-python>=2.4.1` in the `apache-airflow-backport-providers-snowflake` package. If the providers are going to be decoupled, then each one should have it's own constraints, so it's easier to keep them up to date
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] RosterIn commented on issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
RosterIn commented on issue #15129:
URL: https://github.com/apache/airflow/issues/15129#issuecomment-811838587
The `>=1.25.1,<2.0.0` is in master branch not in 1.10 branch.
I would imagine that the constrains for 1.10.14 are not created from master but i might be wrong here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dinigo edited a comment on issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
dinigo edited a comment on issue #15129:
URL: https://github.com/apache/airflow/issues/15129#issuecomment-811825510
I've placed a link to the `setup.py` with a range of versions, and then the constrain generated lays outside of this range. I thought they where generated from the `setup.py` until I saw this:
* [setup.py](https://github.com/apache/airflow/blob/a7a558eade11ff43a7b96544dcfb84be2441fdc0/setup.py#L285) — `>=1.25.1,<2.0.0`
* [airflow 1.10.14 constraints for python 3.6](https://raw.githubusercontent.com/apache/airflow/constraints-1.10.14/constraints-3.6.txt) — `==1.23.0`
It has to be generated somewhere else. Otherwise constrain version would be between `1.25.1` and `2.0.0`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #15129:
URL: https://github.com/apache/airflow/issues/15129#issuecomment-812369833
@dinigo. Everything is as expected.
Constraints are frozen at the time of creating the `apache-airflow` package. And they are there to be able to install the given version of `apache-airflow` in a consistent way. No more, no less. They have nothing to do with the version of provider that you can install afterwards. Constraints finish their role when you have apache-airflow installed, and then you should manage your additional packages (including backport providers) on your own.
There is (and cannot be) absolutely no relation of the constraints with version 1.10.14 to the providers (which are released on a completely dfferent schedule).
You should not use constraints to install providers, this is neither endorsed not supported. You should install providers (or backport providers) separately, without constraints.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk edited a comment on issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
potiuk edited a comment on issue #15129:
URL: https://github.com/apache/airflow/issues/15129#issuecomment-812369833
@dinigo. Everything is as expected.
Constraints are frozen at the time of creating the `apache-airflow` package. And they are there to be able to install the given version of `apache-airflow` in a consistent way. No more, no less. They have nothing to do with the version of provider that you can install afterwards. Constraints finish their role when you have apache-airflow installed, and then you should manage your additional packages (including backport providers) on your own.
There is (and cannot be) absolutely no relation of the constraints with version 1.10.14 to the providers/backport providers (which are released on a completely dfferent schedule).
You should not use constraints to install providers/backports, this is neither endorsed not supported. You should install providers (or backport providers) separately, without constraints.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] uranusjr commented on issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
uranusjr commented on issue #15129:
URL: https://github.com/apache/airflow/issues/15129#issuecomment-811815926
The constraint files are automatically generated from CI, and you shouldn’t be modifying them directly.
I believe the way to do it is to change the minimal version in `setup.py`; CI will pick them up automatically and update the constraints files accordingly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] potiuk closed issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
potiuk closed issue #15129:
URL: https://github.com/apache/airflow/issues/15129
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dinigo commented on issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
dinigo commented on issue #15129:
URL: https://github.com/apache/airflow/issues/15129#issuecomment-811927068
> The >=1.25.1,<2.0.0 is in master branch not in 1.10 branch.
You are right, in the 1.10.14 branch is [not in the setup.py](https://github.com/apache/airflow/blob/1.10.14/setup.py#L267). How do we get to this version? I think I'm missing something.
I see a couple of reasons to close this issue:
1. Backport providers [not maintained anymore](https://github.com/apache/airflow/blame/master/dev/README.md#L104),
> _Note that Backport Providers for Airflow 1.10.* series are not released any more. The last release
of Backport Providers was done on March 17, 2021._
2. Python [3.6 about to die](https://endoflife.date/python) — only 8 months left
According to the /dev/README each package has its dependencies. But in the master they seem to be managed from the `setup.py`. I see how different providers (overall transfer operators) might over time use incompatible versions if they where completely decoupled.
(I hope Google updates their Composer image to python3.6 soon)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [airflow] dinigo commented on issue #15129: Bump google-api-core package version to make apache-airflow-providers-google compatible with python3.6
Posted by GitBox <gi...@apache.org>.
dinigo commented on issue #15129:
URL: https://github.com/apache/airflow/issues/15129#issuecomment-811769091
I haven't found where the constraint files are created, because they seem to be dynamically created. Where can I find it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org