You are viewing a plain text version of this content. The canonical link for it is here.
Posted to notifications@libcloud.apache.org by GitBox <gi...@apache.org> on 2021/07/11 12:15:04 UTC
[GitHub] [libcloud] potiuk opened a new issue #1594: Migrate to the next version of Python requests when released
potiuk opened a new issue #1594:
URL: https://github.com/apache/libcloud/issues/1594
Hello Maintainers,
I am a PMC member of Apache Airflow, and I wanted to give you a bit of heads-up with rather important migration to the upcoming version of requests library in your Python release.
Since you are using requests library in your project, you are affected.
As discussed at length in https://issues.apache.org/jira/browse/LEGAL-572 we found out that the 'chardet` library used by `requests` library was a mandatory dependency to requests and since it has LGPL licence, we should not release any Apache Software with it.
Since then (and since in Airflow we rely on requests heavily) we have been working with the requests maintainers and "charset-normalizer" maintainer to make it possible to replace chardet with MIT-licensed `charset-normalizer` instead so that requests library can be used in Python releases by Apache projects.
This was a bumpy road but finally the PR by @ashb has been merged: https://github.com/psf/requests/pull/5797 and we hope soon a new version of requests library will be released.
This is just a heads-up. I will let you know when it is released, but I have a kind requests as well - I might ask the maintainers to release a release candidate of requests and maybe you could help to test it before it is released, that would be some re-assurance for the maintainers of requests who are very concerned about stability of their releases.
Let me know if you need any more information and whether you would like to help in testing the candidate when it is out.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] potiuk commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-966695247
See my comment here: https://issues.apache.org/jira/browse/LEGAL-572?focusedCommentId=17442504&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17442504 - I quite agree with @Kami
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] potiuk commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-966696598
And just to add to that - Airlfow is both a library AND application :). Which makes us take even a bit different approach (we have as open dependencies as we can afford, but we also have constraint files to allow installation to be reproducible: https://airflow.apache.org/docs/apache-airflow/stable/installation/installing-from-pypi.html#constraints-files
We added the "requests" limitation, but I definitely agree library such as libcloud should not do it unless they are really broken by some library versions.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] Kami closed issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
Kami closed issue #1594:
URL: https://github.com/apache/libcloud/issues/1594
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] Kami commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
Kami commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-966700137
@potiuk Thanks for your input and additional context (and of course many thanks for bringing this issue to our and other projects attention and coordinating with the requests project, etc.)
@justinmclean Also thanks for your input. With all the info above, we would like to get some additional clarify on how to proceed here.
We will of course comply with any legal requirements, but we need to know how to proceed so we can plan and execute accordingly.
If there is no other way, but to update our setup.py metadata file but to require ``requests >= 2.26.0`` (which will likely cause issues with a lot of our users and they won't be able to upgrade), we will also need to drop support for Python 3.5 since that version is only available for Python >= 3.6 (that's something we have been planning to do in the future anyway, but not just yet).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] justinmclean edited a comment on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
justinmclean edited a comment on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-966681292
It not fine from an ASF Policy point of view you can not have a dependancy that has a Category X license.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] Kami edited a comment on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
Kami edited a comment on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-950398383
Since Python 3.5 is EOL for more than a year now (https://endoflife.date/python), I think it's reasonable for us to drop support for it in the release after the next one.
Next one will be 3.4.0 and should go out this year and the one after that will be 3.5.0 and will likely go out some time next year.
What do other people think? /cc @c-w @vdloo @tonybaloney @micafer @RunOrVeith
EDIT: Per package download stats (https://pypistats.org/packages/apache-libcloud), Python 3.5, on average, represents less than 2% of daily downloads.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] Kami commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
Kami commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-966627964
@potiuk and others - Just a heads up - I decided to remove that requests pinning change all together for now - edfb4c5b9302ef74bedb63d53192a458bc466aa2.
The reasoning behind it is here - https://github.com/apache/libcloud/pull/1627#issuecomment-966625020.
In short, we are a library and not application and strict version pinning like that has a big chance to brake many user installations due to conflicting dependency versions (e.g. user app where they rely on Libcloud among many other libraries relies on requests <= 2.26.0, but we require >= 2.25.0)
In fact, it's already braking some places where we rely on Libcloud, but can't use newer version of requests - aka requests is pinned to an older version which is not compatible with latest version Libcloud version due to that strict version pinning.
And per my commit message, I think that should also be fine from licensing perspective since we don't bundle ``requests`` directly and in most cases when installing libcloud, latest version of requests will be correctly picked and installed (unless user already has request pinned to an older version).
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] Kami commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
Kami commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-966689653
@justinmclean I read the ASF policy (https://www.apache.org/legal/resolved.html) and some of the related JIRA issues (https://issues.apache.org/jira/browse/LEGAL-572, https://issues.apache.org/jira/browse/LEGAL-220) and it doesn't appear that the answer is that straight and clear cut in this scenario (or maybe my interpretation is wrong or I missed some ticket). I haven't really found much prior art either.
There are a couple of things at play here:
1. We don't bundle any dependency directly with the library. Dependencies are installed when installing library from PyPi using pip on install time.
2. Library with a problematic dependency in this case is a transitive dependency of requests on which Libcloud relies on and not a direct dependency.
I'm of course all for being on the "safe side as much as possible", but sadly the issue is more complicated, because as mentioned above, Libcloud is a library and not an application. Libcloud is usually used as one of the many build blocks among many other libraries to build different application on top and similar.
This means we should use as loose requirements as possible and not pin dependencies to specific versions since this will prevent many users from using the library if some other dependency in their stack depends on an older or a conflicting version (which is very likely the case with requests since 2.26.0 was released just recently and I'm personally aware of multiple situations like that myself).
Apache Airflow is a bit different and doesn't have the same problem since they are an application and not a library. And applications are usually encouraged (and it's also a best practice for reproducibility, secure and other reasons) to pin all the dependencies to specific versions.
In addition to that, we still support Python 3.5 and requests 2.2.26 only supports Python >= 3.6.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] Kami commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
Kami commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-950398383
Since Python 3.5 is EOL for more than a year now (https://endoflife.date/python), I think it's reasonable for us to drop support for it in the release after the next one.
Next one will be 3.4.0 and should go out this year and the one after that will be 3.5.0 and will likely go out some time next year.
What do other people think? /cc @c-w @vdloo @tonybaloney @micafer @RunOrVeith
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] potiuk commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
potiuk commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-879195369
Requests 2.26.0 released without the mandatory chardet dependency:
https://pypi.org/project/requests/2.26.0/
Instead of chardet, use the MIT-licensed charset_normalizer for Python3 to remove license ambiguity for projects bundling requests. If chardet is already installed on your machine it will be used instead of charset_normalizer to keep backwards compatibility. (#5797)
You can also install chardet while installing requests by specifying [use_chardet_on_py3] extra as follows:
shell pip install "requests[use_chardet_on_py3]"
Python2 still depends upon the chardet module.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] tonybaloney commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
tonybaloney commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-950398517
> Since Python 3.5 is EOL for more than a year now (https://endoflife.date/python), I think it's reasonable for us to drop support for it in the release after the next one.
>
>
>
> Next one will be 3.4.0 and should go out this year and the one after that will be 3.5.0 and will likely go out some time next year.
>
>
>
> What do other people think? /cc @c-w @vdloo @tonybaloney @micafer @RunOrVeith
100%
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] RunOrVeith commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
RunOrVeith commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-951097862
I also agree
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] justinmclean commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
justinmclean commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-966681292
It not fine from an ASF Policy point of view you can have a dependancy that has a Category X license.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] Kami commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
Kami commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-962520345
A lot of the libraries and tooling has already dropped support for Python 3.5 a while back so my recent PRs and changes have been quite a pain since I needed to implement a bunch of workarounds to get it to work under Python 3.5.
So the sooner we drop support for it, the better.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] micafer commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
micafer commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-950565412
I fully agree.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] vdloo commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
vdloo commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-950539639
sounds good, people can pin the 3.4.0 release if they still need to run on python 3.5 :+1:
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] Kami commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
Kami commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-950389032
Sorry for the delay and thank for notifying us about this issue / change.
I will look into updating setup.py metadata so we require ``requests>=2.26.0`` and including that change in the next release (I will bump a minor version to be on the safe side).
We don't support Python 2 anymore so hopefully there won't be any problems, but I will also go over requests changelog to make sure we avoid any possible surprises.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] Kami commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
Kami commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-950392456
Looks like latest version if requests also dropped support for Python 3.5 which means this change will be more involved.
For now, I will add a conditional to setup.py to use older version of requests under Python 3.5, but in the future we should probably also consider dropping support for Python 3.5 to simplify things.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [libcloud] Kami commented on issue #1594: Migrate to the next version of Python requests when released
Posted by GitBox <gi...@apache.org>.
Kami commented on issue #1594:
URL: https://github.com/apache/libcloud/issues/1594#issuecomment-952066115
#1619 should mostly address this issue (will be fully addressed when dropping support for Python 3.5).
It's also worth noting that we don't have a fixed version specified in setup.py so in most cases, latest version of requests will already be installed when installing Libcloud.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscribe@libcloud.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org