You are viewing a plain text version of this content. The canonical link for it is here.

Posted to dev@beam.apache.org by Udi Meiri <eh...@google.com> on 2020/06/10 17:59:31 UTC

python precommit error - google-auth depenedency?

Hi,
I'm trying to understand these "pip check" failures:

ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll
have rsa 4.1 which is incompatible


https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console

However, when I do
pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]

locally, the google-auth package is not installed at all.
Any ideas on how to debug where this requirement is coming from?

Re: python precommit error - google-auth depenedency?

Posted by Valentyn Tymofieiev <va...@google.com>.

Also: https://github.com/sybrenstuvel/python-rsa/blob/master/CHANGELOG.md

On Mon, Jun 15, 2020 at 2:10 PM Valentyn Tymofieiev <va...@google.com>
wrote:

> They have been fixing an unclean turndown of Py2 support, there are some
> details in https://github.com/sybrenstuvel/python-rsa/issues/152.
>
> On Mon, Jun 15, 2020 at 1:59 PM Ahmet Altay <al...@google.com> wrote:
>
>> Thank you all for working on this.
>>
>> I have a curiosity question. rsa package released 8 versions in the past
>> 5 days after 2 years. Does anyone have visibility into what is happening?
>> Which version is stable?
>>
>> On Mon, Jun 15, 2020 at 1:32 PM Valentyn Tymofieiev <va...@google.com>
>> wrote:
>>
>>> FYI, Beam dependency checker is still complaining currently since
>>> google-auth currently sets the restriction to rsa<4.1 for Py2.
>>>
>>> There are 2 independent changes in flight, which should fix this:
>>> https://github.com/googleapis/google-auth-library-python/pull/535,
>>> https://github.com/apache/beam/pull/11996
>>>
>>> On Thu, Jun 11, 2020 at 4:28 PM Udi Meiri <eh...@google.com> wrote:
>>>
>>>> BTW, the new pip resolver seems to do the right thing by installing
>>>> rsa==4.0 instead of 4.2 in this case (upgrade pip to 20.1.1):
>>>>
>>>> pip --unstable-feature=resolver install apache-beam[gcp]
>>>>
>>>>
>>>> https://discuss.python.org/t/an-update-on-pip-and-dependency-resolution/1898/4
>>>>
>>>> On Thu, Jun 11, 2020 at 2:40 PM Valentyn Tymofieiev <
>>>> valentyn@google.com> wrote:
>>>>
>>>>> > In python 2 oauth2client's rsa>3.14 requirement will resolve to
>>>>> latest python2 supporting version of rsa (4.0?)
>>>>>
>>>>> Unfortunately rsa 4.1 didn't set a python_requires stanza to prevent
>>>>> the breakage of Py2 users, opened:
>>>>> https://github.com/sybrenstuvel/python-rsa/issues/152.
>>>>>
>>>>> On Wed, Jun 10, 2020 at 7:14 PM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 10, 2020 at 7:11 PM Bu Sun Kim <bu...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> google-auth has been released (with the wider pin
>>>>>>> <https://github.com/googleapis/google-auth-library-python/blob/6350834ee25295e0754b6611fdff257668a0b0c4/setup.py#L24> on
>>>>>>> rsa).
>>>>>>>
>>>>>>
>>>>>> Thank you! Much appreciated!
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 10, 2020 at 6:07 PM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 10, 2020 at 4:07 PM Kyle Weaver <kc...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> The fix to google-auth has been merged. Is the plan just to wait
>>>>>>>>> until a new version of google-auth is released and ignore the failing tests
>>>>>>>>> until then? (btw I filed a JIRA for this before I realized it was already
>>>>>>>>> being discussed here:
>>>>>>>>> https://issues.apache.org/jira/browse/BEAM-10232)
>>>>>>>>>
>>>>>>>>
>>>>>>>> Could we add it as a test dependency? Or if that is not possible,
>>>>>>>> add it but remove it before next release?
>>>>>>>>
>>>>>>>> It seems like there is a release PR on google-auth (
>>>>>>>> https://github.com/googleapis/google-auth-library-python/pull/525).
>>>>>>>> I asked +Bu Sun Kim <bu...@google.com> on the PR, they usually
>>>>>>>> release pretty quickly.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jun 10, 2020 at 3:21 PM Udi Meiri <eh...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Yes you're right, Py2 envs are still using 4.0.
>>>>>>>>>>
>>>>>>>>>> On Wed, Jun 10, 2020 at 3:03 PM Ahmet Altay <al...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri <eh...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> 4.1 drops Python 2 support, so I'm not sure if we're ready for
>>>>>>>>>>>> that yet.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Wouldn't that work by default? In python 2 oauth2client's
>>>>>>>>>>> rsa>3.14 requirement will resolve to latest python2 supporting version of
>>>>>>>>>>> rsa (4.0?)
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <al...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Looks like there is an attempt to fix this:
>>>>>>>>>>>>> https://github.com/googleapis/google-auth-library-python/pull/524
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <
>>>>>>>>>>>>>>> kenn@apache.org> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> You may be interested in following
>>>>>>>>>>>>>>>> https://github.com/pypa/pip/issues/988 if you are not
>>>>>>>>>>>>>>>> already.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Kenn
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <
>>>>>>>>>>>>>>>> ehudm@google.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Seems like manually installing rsa==4.0 satisfies deps,
>>>>>>>>>>>>>>>>> but pip doesn't do transitive deps well.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Would it be right to put a direct dependency
>>>>>>>>>>>>>>>>> on rsa<4.1,>=3.1.4 in setup.py?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Did you find where the google-auth dependency is coming
>>>>>>>>>>>>>>> from? We might try to fix the problem at the source of that dependency
>>>>>>>>>>>>>>> instead of adding rsa to beam's setup.py.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> oauth2client depends on rsa>=3.14 with no upper limit. rsa
>>>>>>>>>>>>>> 4.1 was released today.
>>>>>>>>>>>>>> The places that require rsa<4.1 are deeper in the dependency
>>>>>>>>>>>>>> tree. For example:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> google-cloud-bigquery==1.24.0
>>>>>>>>>>>>>>   - google-api-core [required: >=1.15.0,<2.0dev, installed:
>>>>>>>>>>>>>> 1.20.0]
>>>>>>>>>>>>>>     - google-auth [required: >=1.14.0,<2.0dev, installed:
>>>>>>>>>>>>>> 1.16.1]
>>>>>>>>>>>>>>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <
>>>>>>>>>>>>>>>>> ehudm@google.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Thanks, that helped in an unexpected way. :)
>>>>>>>>>>>>>>>>>> I should have used the "gcp" extra instead of "cloud" in
>>>>>>>>>>>>>>>>>> my pip install command above.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>>>>>>>>>>>>>>> valentyn@google.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> > Any ideas on how to debug where this requirement is
>>>>>>>>>>>>>>>>>>> coming from?
>>>>>>>>>>>>>>>>>>> You could try installing and calling pipdeptree [1] from
>>>>>>>>>>>>>>>>>>> a Jenkins job, and see if it helps.
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <
>>>>>>>>>>>>>>>>>>> ehudm@google.com> wrote:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> However, when I do
>>>>>>>>>>>>>>>>>>>> pip install
>>>>>>>>>>>>>>>>>>>> dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>> locally, the google-auth package is not installed at
>>>>>>>>>>>>>>>>>>>> all.
>>>>>>>>>>>>>>>>>>>> Any ideas on how to debug where this requirement is
>>>>>>>>>>>>>>>>>>>> coming from?
>>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Valentyn Tymofieiev <va...@google.com>.

They have been fixing an unclean turndown of Py2 support, there are some
details in https://github.com/sybrenstuvel/python-rsa/issues/152.

On Mon, Jun 15, 2020 at 1:59 PM Ahmet Altay <al...@google.com> wrote:

> Thank you all for working on this.
>
> I have a curiosity question. rsa package released 8 versions in the past 5
> days after 2 years. Does anyone have visibility into what is happening?
> Which version is stable?
>
> On Mon, Jun 15, 2020 at 1:32 PM Valentyn Tymofieiev <va...@google.com>
> wrote:
>
>> FYI, Beam dependency checker is still complaining currently since
>> google-auth currently sets the restriction to rsa<4.1 for Py2.
>>
>> There are 2 independent changes in flight, which should fix this:
>> https://github.com/googleapis/google-auth-library-python/pull/535,
>> https://github.com/apache/beam/pull/11996
>>
>> On Thu, Jun 11, 2020 at 4:28 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> BTW, the new pip resolver seems to do the right thing by installing
>>> rsa==4.0 instead of 4.2 in this case (upgrade pip to 20.1.1):
>>>
>>> pip --unstable-feature=resolver install apache-beam[gcp]
>>>
>>>
>>> https://discuss.python.org/t/an-update-on-pip-and-dependency-resolution/1898/4
>>>
>>> On Thu, Jun 11, 2020 at 2:40 PM Valentyn Tymofieiev <va...@google.com>
>>> wrote:
>>>
>>>> > In python 2 oauth2client's rsa>3.14 requirement will resolve to
>>>> latest python2 supporting version of rsa (4.0?)
>>>>
>>>> Unfortunately rsa 4.1 didn't set a python_requires stanza to prevent
>>>> the breakage of Py2 users, opened:
>>>> https://github.com/sybrenstuvel/python-rsa/issues/152.
>>>>
>>>> On Wed, Jun 10, 2020 at 7:14 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 10, 2020 at 7:11 PM Bu Sun Kim <bu...@google.com>
>>>>> wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> google-auth has been released (with the wider pin
>>>>>> <https://github.com/googleapis/google-auth-library-python/blob/6350834ee25295e0754b6611fdff257668a0b0c4/setup.py#L24> on
>>>>>> rsa).
>>>>>>
>>>>>
>>>>> Thank you! Much appreciated!
>>>>>
>>>>>
>>>>>>
>>>>>> On Wed, Jun 10, 2020 at 6:07 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 10, 2020 at 4:07 PM Kyle Weaver <kc...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> The fix to google-auth has been merged. Is the plan just to wait
>>>>>>>> until a new version of google-auth is released and ignore the failing tests
>>>>>>>> until then? (btw I filed a JIRA for this before I realized it was already
>>>>>>>> being discussed here:
>>>>>>>> https://issues.apache.org/jira/browse/BEAM-10232)
>>>>>>>>
>>>>>>>
>>>>>>> Could we add it as a test dependency? Or if that is not possible,
>>>>>>> add it but remove it before next release?
>>>>>>>
>>>>>>> It seems like there is a release PR on google-auth (
>>>>>>> https://github.com/googleapis/google-auth-library-python/pull/525).
>>>>>>> I asked +Bu Sun Kim <bu...@google.com> on the PR, they usually
>>>>>>> release pretty quickly.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 10, 2020 at 3:21 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>>
>>>>>>>>> Yes you're right, Py2 envs are still using 4.0.
>>>>>>>>>
>>>>>>>>> On Wed, Jun 10, 2020 at 3:03 PM Ahmet Altay <al...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri <eh...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> 4.1 drops Python 2 support, so I'm not sure if we're ready for
>>>>>>>>>>> that yet.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Wouldn't that work by default? In python 2 oauth2client's
>>>>>>>>>> rsa>3.14 requirement will resolve to latest python2 supporting version of
>>>>>>>>>> rsa (4.0?)
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <al...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Looks like there is an attempt to fix this:
>>>>>>>>>>>> https://github.com/googleapis/google-auth-library-python/pull/524
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <
>>>>>>>>>>>>>> kenn@apache.org> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> You may be interested in following
>>>>>>>>>>>>>>> https://github.com/pypa/pip/issues/988 if you are not
>>>>>>>>>>>>>>> already.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Kenn
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Seems like manually installing rsa==4.0 satisfies deps, but
>>>>>>>>>>>>>>>> pip doesn't do transitive deps well.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Would it be right to put a direct dependency
>>>>>>>>>>>>>>>> on rsa<4.1,>=3.1.4 in setup.py?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Did you find where the google-auth dependency is coming from?
>>>>>>>>>>>>>> We might try to fix the problem at the source of that dependency instead of
>>>>>>>>>>>>>> adding rsa to beam's setup.py.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1
>>>>>>>>>>>>> was released today.
>>>>>>>>>>>>> The places that require rsa<4.1 are deeper in the dependency
>>>>>>>>>>>>> tree. For example:
>>>>>>>>>>>>>
>>>>>>>>>>>>> google-cloud-bigquery==1.24.0
>>>>>>>>>>>>>   - google-api-core [required: >=1.15.0,<2.0dev, installed:
>>>>>>>>>>>>> 1.20.0]
>>>>>>>>>>>>>     - google-auth [required: >=1.14.0,<2.0dev, installed:
>>>>>>>>>>>>> 1.16.1]
>>>>>>>>>>>>>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <
>>>>>>>>>>>>>>>> ehudm@google.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks, that helped in an unexpected way. :)
>>>>>>>>>>>>>>>>> I should have used the "gcp" extra instead of "cloud" in
>>>>>>>>>>>>>>>>> my pip install command above.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>>>>>>>>>>>>>> valentyn@google.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> > Any ideas on how to debug where this requirement is
>>>>>>>>>>>>>>>>>> coming from?
>>>>>>>>>>>>>>>>>> You could try installing and calling pipdeptree [1] from
>>>>>>>>>>>>>>>>>> a Jenkins job, and see if it helps.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <
>>>>>>>>>>>>>>>>>> ehudm@google.com> wrote:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> However, when I do
>>>>>>>>>>>>>>>>>>> pip install
>>>>>>>>>>>>>>>>>>> dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>> locally, the google-auth package is not installed at all.
>>>>>>>>>>>>>>>>>>> Any ideas on how to debug where this requirement is
>>>>>>>>>>>>>>>>>>> coming from?
>>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Ahmet Altay <al...@google.com>.

Thank you all for working on this.

I have a curiosity question. rsa package released 8 versions in the past 5
days after 2 years. Does anyone have visibility into what is happening?
Which version is stable?

On Mon, Jun 15, 2020 at 1:32 PM Valentyn Tymofieiev <va...@google.com>
wrote:

> FYI, Beam dependency checker is still complaining currently since
> google-auth currently sets the restriction to rsa<4.1 for Py2.
>
> There are 2 independent changes in flight, which should fix this:
> https://github.com/googleapis/google-auth-library-python/pull/535,
> https://github.com/apache/beam/pull/11996
>
> On Thu, Jun 11, 2020 at 4:28 PM Udi Meiri <eh...@google.com> wrote:
>
>> BTW, the new pip resolver seems to do the right thing by installing
>> rsa==4.0 instead of 4.2 in this case (upgrade pip to 20.1.1):
>>
>> pip --unstable-feature=resolver install apache-beam[gcp]
>>
>>
>> https://discuss.python.org/t/an-update-on-pip-and-dependency-resolution/1898/4
>>
>> On Thu, Jun 11, 2020 at 2:40 PM Valentyn Tymofieiev <va...@google.com>
>> wrote:
>>
>>> > In python 2 oauth2client's rsa>3.14 requirement will resolve to
>>> latest python2 supporting version of rsa (4.0?)
>>>
>>> Unfortunately rsa 4.1 didn't set a python_requires stanza to prevent the
>>> breakage of Py2 users, opened:
>>> https://github.com/sybrenstuvel/python-rsa/issues/152.
>>>
>>> On Wed, Jun 10, 2020 at 7:14 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>>>
>>>>
>>>> On Wed, Jun 10, 2020 at 7:11 PM Bu Sun Kim <bu...@google.com> wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> google-auth has been released (with the wider pin
>>>>> <https://github.com/googleapis/google-auth-library-python/blob/6350834ee25295e0754b6611fdff257668a0b0c4/setup.py#L24> on
>>>>> rsa).
>>>>>
>>>>
>>>> Thank you! Much appreciated!
>>>>
>>>>
>>>>>
>>>>> On Wed, Jun 10, 2020 at 6:07 PM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 10, 2020 at 4:07 PM Kyle Weaver <kc...@google.com>
>>>>>> wrote:
>>>>>>
>>>>>>> The fix to google-auth has been merged. Is the plan just to wait
>>>>>>> until a new version of google-auth is released and ignore the failing tests
>>>>>>> until then? (btw I filed a JIRA for this before I realized it was already
>>>>>>> being discussed here:
>>>>>>> https://issues.apache.org/jira/browse/BEAM-10232)
>>>>>>>
>>>>>>
>>>>>> Could we add it as a test dependency? Or if that is not possible, add
>>>>>> it but remove it before next release?
>>>>>>
>>>>>> It seems like there is a release PR on google-auth (
>>>>>> https://github.com/googleapis/google-auth-library-python/pull/525).
>>>>>> I asked +Bu Sun Kim <bu...@google.com> on the PR, they usually
>>>>>> release pretty quickly.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 10, 2020 at 3:21 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>
>>>>>>>> Yes you're right, Py2 envs are still using 4.0.
>>>>>>>>
>>>>>>>> On Wed, Jun 10, 2020 at 3:03 PM Ahmet Altay <al...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri <eh...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> 4.1 drops Python 2 support, so I'm not sure if we're ready for
>>>>>>>>>> that yet.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Wouldn't that work by default? In python 2 oauth2client's rsa>3.14
>>>>>>>>> requirement will resolve to latest python2 supporting version of rsa (4.0?)
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <al...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Looks like there is an attempt to fix this:
>>>>>>>>>>> https://github.com/googleapis/google-auth-library-python/pull/524
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <
>>>>>>>>>>>>> kenn@apache.org> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> You may be interested in following
>>>>>>>>>>>>>> https://github.com/pypa/pip/issues/988 if you are not
>>>>>>>>>>>>>> already.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Kenn
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Seems like manually installing rsa==4.0 satisfies deps, but
>>>>>>>>>>>>>>> pip doesn't do transitive deps well.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Would it be right to put a direct dependency
>>>>>>>>>>>>>>> on rsa<4.1,>=3.1.4 in setup.py?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>> Did you find where the google-auth dependency is coming from?
>>>>>>>>>>>>> We might try to fix the problem at the source of that dependency instead of
>>>>>>>>>>>>> adding rsa to beam's setup.py.
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1
>>>>>>>>>>>> was released today.
>>>>>>>>>>>> The places that require rsa<4.1 are deeper in the dependency
>>>>>>>>>>>> tree. For example:
>>>>>>>>>>>>
>>>>>>>>>>>> google-cloud-bigquery==1.24.0
>>>>>>>>>>>>   - google-api-core [required: >=1.15.0,<2.0dev, installed:
>>>>>>>>>>>> 1.20.0]
>>>>>>>>>>>>     - google-auth [required: >=1.14.0,<2.0dev, installed:
>>>>>>>>>>>> 1.16.1]
>>>>>>>>>>>>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks, that helped in an unexpected way. :)
>>>>>>>>>>>>>>>> I should have used the "gcp" extra instead of "cloud" in my
>>>>>>>>>>>>>>>> pip install command above.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>>>>>>>>>>>>> valentyn@google.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> > Any ideas on how to debug where this requirement is
>>>>>>>>>>>>>>>>> coming from?
>>>>>>>>>>>>>>>>> You could try installing and calling pipdeptree [1] from a
>>>>>>>>>>>>>>>>> Jenkins job, and see if it helps.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <
>>>>>>>>>>>>>>>>> ehudm@google.com> wrote:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> However, when I do
>>>>>>>>>>>>>>>>>> pip install
>>>>>>>>>>>>>>>>>> dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> locally, the google-auth package is not installed at all.
>>>>>>>>>>>>>>>>>> Any ideas on how to debug where this requirement is
>>>>>>>>>>>>>>>>>> coming from?
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Valentyn Tymofieiev <va...@google.com>.

FYI, Beam dependency checker is still complaining currently since
google-auth currently sets the restriction to rsa<4.1 for Py2.

There are 2 independent changes in flight, which should fix this:
https://github.com/googleapis/google-auth-library-python/pull/535,
https://github.com/apache/beam/pull/11996

On Thu, Jun 11, 2020 at 4:28 PM Udi Meiri <eh...@google.com> wrote:

> BTW, the new pip resolver seems to do the right thing by installing
> rsa==4.0 instead of 4.2 in this case (upgrade pip to 20.1.1):
>
> pip --unstable-feature=resolver install apache-beam[gcp]
>
>
> https://discuss.python.org/t/an-update-on-pip-and-dependency-resolution/1898/4
>
> On Thu, Jun 11, 2020 at 2:40 PM Valentyn Tymofieiev <va...@google.com>
> wrote:
>
>> > In python 2 oauth2client's rsa>3.14 requirement will resolve to
>> latest python2 supporting version of rsa (4.0?)
>>
>> Unfortunately rsa 4.1 didn't set a python_requires stanza to prevent the
>> breakage of Py2 users, opened:
>> https://github.com/sybrenstuvel/python-rsa/issues/152.
>>
>> On Wed, Jun 10, 2020 at 7:14 PM Ahmet Altay <al...@google.com> wrote:
>>
>>>
>>>
>>> On Wed, Jun 10, 2020 at 7:11 PM Bu Sun Kim <bu...@google.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> google-auth has been released (with the wider pin
>>>> <https://github.com/googleapis/google-auth-library-python/blob/6350834ee25295e0754b6611fdff257668a0b0c4/setup.py#L24> on
>>>> rsa).
>>>>
>>>
>>> Thank you! Much appreciated!
>>>
>>>
>>>>
>>>> On Wed, Jun 10, 2020 at 6:07 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 10, 2020 at 4:07 PM Kyle Weaver <kc...@google.com>
>>>>> wrote:
>>>>>
>>>>>> The fix to google-auth has been merged. Is the plan just to wait
>>>>>> until a new version of google-auth is released and ignore the failing tests
>>>>>> until then? (btw I filed a JIRA for this before I realized it was already
>>>>>> being discussed here:
>>>>>> https://issues.apache.org/jira/browse/BEAM-10232)
>>>>>>
>>>>>
>>>>> Could we add it as a test dependency? Or if that is not possible, add
>>>>> it but remove it before next release?
>>>>>
>>>>> It seems like there is a release PR on google-auth (
>>>>> https://github.com/googleapis/google-auth-library-python/pull/525). I
>>>>> asked +Bu Sun Kim <bu...@google.com> on the PR, they usually
>>>>> release pretty quickly.
>>>>>
>>>>>
>>>>>>
>>>>>> On Wed, Jun 10, 2020 at 3:21 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>
>>>>>>> Yes you're right, Py2 envs are still using 4.0.
>>>>>>>
>>>>>>> On Wed, Jun 10, 2020 at 3:03 PM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>>
>>>>>>>>> 4.1 drops Python 2 support, so I'm not sure if we're ready for
>>>>>>>>> that yet.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Wouldn't that work by default? In python 2 oauth2client's rsa>3.14
>>>>>>>> requirement will resolve to latest python2 supporting version of rsa (4.0?)
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <al...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Looks like there is an attempt to fix this:
>>>>>>>>>> https://github.com/googleapis/google-auth-library-python/pull/524
>>>>>>>>>>
>>>>>>>>>> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <
>>>>>>>>>>>> kenn@apache.org> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> You may be interested in following
>>>>>>>>>>>>> https://github.com/pypa/pip/issues/988 if you are not already.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Kenn
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Seems like manually installing rsa==4.0 satisfies deps, but
>>>>>>>>>>>>>> pip doesn't do transitive deps well.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Would it be right to put a direct dependency
>>>>>>>>>>>>>> on rsa<4.1,>=3.1.4 in setup.py?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> Did you find where the google-auth dependency is coming from?
>>>>>>>>>>>> We might try to fix the problem at the source of that dependency instead of
>>>>>>>>>>>> adding rsa to beam's setup.py.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1
>>>>>>>>>>> was released today.
>>>>>>>>>>> The places that require rsa<4.1 are deeper in the dependency
>>>>>>>>>>> tree. For example:
>>>>>>>>>>>
>>>>>>>>>>> google-cloud-bigquery==1.24.0
>>>>>>>>>>>   - google-api-core [required: >=1.15.0,<2.0dev, installed:
>>>>>>>>>>> 1.20.0]
>>>>>>>>>>>     - google-auth [required: >=1.14.0,<2.0dev, installed: 1.16.1]
>>>>>>>>>>>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks, that helped in an unexpected way. :)
>>>>>>>>>>>>>>> I should have used the "gcp" extra instead of "cloud" in my
>>>>>>>>>>>>>>> pip install command above.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>>>>>>>>>>>> valentyn@google.com> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> > Any ideas on how to debug where this requirement is
>>>>>>>>>>>>>>>> coming from?
>>>>>>>>>>>>>>>> You could try installing and calling pipdeptree [1] from a
>>>>>>>>>>>>>>>> Jenkins job, and see if it helps.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <
>>>>>>>>>>>>>>>> ehudm@google.com> wrote:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> However, when I do
>>>>>>>>>>>>>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> locally, the google-auth package is not installed at all.
>>>>>>>>>>>>>>>>> Any ideas on how to debug where this requirement is coming
>>>>>>>>>>>>>>>>> from?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Udi Meiri <eh...@google.com>.

BTW, the new pip resolver seems to do the right thing by installing
rsa==4.0 instead of 4.2 in this case (upgrade pip to 20.1.1):

pip --unstable-feature=resolver install apache-beam[gcp]

https://discuss.python.org/t/an-update-on-pip-and-dependency-resolution/1898/4

On Thu, Jun 11, 2020 at 2:40 PM Valentyn Tymofieiev <va...@google.com>
wrote:

> > In python 2 oauth2client's rsa>3.14 requirement will resolve to
> latest python2 supporting version of rsa (4.0?)
>
> Unfortunately rsa 4.1 didn't set a python_requires stanza to prevent the
> breakage of Py2 users, opened:
> https://github.com/sybrenstuvel/python-rsa/issues/152.
>
> On Wed, Jun 10, 2020 at 7:14 PM Ahmet Altay <al...@google.com> wrote:
>
>>
>>
>> On Wed, Jun 10, 2020 at 7:11 PM Bu Sun Kim <bu...@google.com> wrote:
>>
>>> Hi,
>>>
>>> google-auth has been released (with the wider pin
>>> <https://github.com/googleapis/google-auth-library-python/blob/6350834ee25295e0754b6611fdff257668a0b0c4/setup.py#L24> on
>>> rsa).
>>>
>>
>> Thank you! Much appreciated!
>>
>>
>>>
>>> On Wed, Jun 10, 2020 at 6:07 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>>>
>>>>
>>>> On Wed, Jun 10, 2020 at 4:07 PM Kyle Weaver <kc...@google.com>
>>>> wrote:
>>>>
>>>>> The fix to google-auth has been merged. Is the plan just to wait until
>>>>> a new version of google-auth is released and ignore the failing tests until
>>>>> then? (btw I filed a JIRA for this before I realized it was already being
>>>>> discussed here: https://issues.apache.org/jira/browse/BEAM-10232)
>>>>>
>>>>
>>>> Could we add it as a test dependency? Or if that is not possible, add
>>>> it but remove it before next release?
>>>>
>>>> It seems like there is a release PR on google-auth (
>>>> https://github.com/googleapis/google-auth-library-python/pull/525). I
>>>> asked +Bu Sun Kim <bu...@google.com> on the PR, they usually
>>>> release pretty quickly.
>>>>
>>>>
>>>>>
>>>>> On Wed, Jun 10, 2020 at 3:21 PM Udi Meiri <eh...@google.com> wrote:
>>>>>
>>>>>> Yes you're right, Py2 envs are still using 4.0.
>>>>>>
>>>>>> On Wed, Jun 10, 2020 at 3:03 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>
>>>>>>>> 4.1 drops Python 2 support, so I'm not sure if we're ready for that
>>>>>>>> yet.
>>>>>>>>
>>>>>>>
>>>>>>> Wouldn't that work by default? In python 2 oauth2client's rsa>3.14
>>>>>>> requirement will resolve to latest python2 supporting version of rsa (4.0?)
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <al...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Looks like there is an attempt to fix this:
>>>>>>>>> https://github.com/googleapis/google-auth-library-python/pull/524
>>>>>>>>>
>>>>>>>>> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <ke...@apache.org>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> You may be interested in following
>>>>>>>>>>>> https://github.com/pypa/pip/issues/988 if you are not already.
>>>>>>>>>>>>
>>>>>>>>>>>> Kenn
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Seems like manually installing rsa==4.0 satisfies deps, but
>>>>>>>>>>>>> pip doesn't do transitive deps well.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Would it be right to put a direct dependency
>>>>>>>>>>>>> on rsa<4.1,>=3.1.4 in setup.py?
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>> Did you find where the google-auth dependency is coming from? We
>>>>>>>>>>> might try to fix the problem at the source of that dependency instead of
>>>>>>>>>>> adding rsa to beam's setup.py.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1
>>>>>>>>>> was released today.
>>>>>>>>>> The places that require rsa<4.1 are deeper in the dependency
>>>>>>>>>> tree. For example:
>>>>>>>>>>
>>>>>>>>>> google-cloud-bigquery==1.24.0
>>>>>>>>>>   - google-api-core [required: >=1.15.0,<2.0dev, installed:
>>>>>>>>>> 1.20.0]
>>>>>>>>>>     - google-auth [required: >=1.14.0,<2.0dev, installed: 1.16.1]
>>>>>>>>>>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks, that helped in an unexpected way. :)
>>>>>>>>>>>>>> I should have used the "gcp" extra instead of "cloud" in my
>>>>>>>>>>>>>> pip install command above.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>>>>>>>>>>> valentyn@google.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> > Any ideas on how to debug where this requirement is coming
>>>>>>>>>>>>>>> from?
>>>>>>>>>>>>>>> You could try installing and calling pipdeptree [1] from a
>>>>>>>>>>>>>>> Jenkins job, and see if it helps.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com>
>>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> However, when I do
>>>>>>>>>>>>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> locally, the google-auth package is not installed at all.
>>>>>>>>>>>>>>>> Any ideas on how to debug where this requirement is coming
>>>>>>>>>>>>>>>> from?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Valentyn Tymofieiev <va...@google.com>.

> In python 2 oauth2client's rsa>3.14 requirement will resolve to
latest python2 supporting version of rsa (4.0?)

Unfortunately rsa 4.1 didn't set a python_requires stanza to prevent the
breakage of Py2 users, opened:
https://github.com/sybrenstuvel/python-rsa/issues/152.

On Wed, Jun 10, 2020 at 7:14 PM Ahmet Altay <al...@google.com> wrote:

>
>
> On Wed, Jun 10, 2020 at 7:11 PM Bu Sun Kim <bu...@google.com> wrote:
>
>> Hi,
>>
>> google-auth has been released (with the wider pin
>> <https://github.com/googleapis/google-auth-library-python/blob/6350834ee25295e0754b6611fdff257668a0b0c4/setup.py#L24> on
>> rsa).
>>
>
> Thank you! Much appreciated!
>
>
>>
>> On Wed, Jun 10, 2020 at 6:07 PM Ahmet Altay <al...@google.com> wrote:
>>
>>>
>>>
>>> On Wed, Jun 10, 2020 at 4:07 PM Kyle Weaver <kc...@google.com> wrote:
>>>
>>>> The fix to google-auth has been merged. Is the plan just to wait until
>>>> a new version of google-auth is released and ignore the failing tests until
>>>> then? (btw I filed a JIRA for this before I realized it was already being
>>>> discussed here: https://issues.apache.org/jira/browse/BEAM-10232)
>>>>
>>>
>>> Could we add it as a test dependency? Or if that is not possible, add it
>>> but remove it before next release?
>>>
>>> It seems like there is a release PR on google-auth (
>>> https://github.com/googleapis/google-auth-library-python/pull/525). I
>>> asked +Bu Sun Kim <bu...@google.com> on the PR, they usually release
>>> pretty quickly.
>>>
>>>
>>>>
>>>> On Wed, Jun 10, 2020 at 3:21 PM Udi Meiri <eh...@google.com> wrote:
>>>>
>>>>> Yes you're right, Py2 envs are still using 4.0.
>>>>>
>>>>> On Wed, Jun 10, 2020 at 3:03 PM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>
>>>>>>> 4.1 drops Python 2 support, so I'm not sure if we're ready for that
>>>>>>> yet.
>>>>>>>
>>>>>>
>>>>>> Wouldn't that work by default? In python 2 oauth2client's rsa>3.14
>>>>>> requirement will resolve to latest python2 supporting version of rsa (4.0?)
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Looks like there is an attempt to fix this:
>>>>>>>> https://github.com/googleapis/google-auth-library-python/pull/524
>>>>>>>>
>>>>>>>> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <ke...@apache.org>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> You may be interested in following
>>>>>>>>>>> https://github.com/pypa/pip/issues/988 if you are not already.
>>>>>>>>>>>
>>>>>>>>>>> Kenn
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Seems like manually installing rsa==4.0 satisfies deps, but pip
>>>>>>>>>>>> doesn't do transitive deps well.
>>>>>>>>>>>>
>>>>>>>>>>>> Would it be right to put a direct dependency on rsa<4.1,>=3.1.4
>>>>>>>>>>>> in setup.py?
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> Did you find where the google-auth dependency is coming from? We
>>>>>>>>>> might try to fix the problem at the source of that dependency instead of
>>>>>>>>>> adding rsa to beam's setup.py.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1 was
>>>>>>>>> released today.
>>>>>>>>> The places that require rsa<4.1 are deeper in the dependency tree.
>>>>>>>>> For example:
>>>>>>>>>
>>>>>>>>> google-cloud-bigquery==1.24.0
>>>>>>>>>   - google-api-core [required: >=1.15.0,<2.0dev, installed: 1.20.0]
>>>>>>>>>     - google-auth [required: >=1.14.0,<2.0dev, installed: 1.16.1]
>>>>>>>>>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks, that helped in an unexpected way. :)
>>>>>>>>>>>>> I should have used the "gcp" extra instead of "cloud" in my
>>>>>>>>>>>>> pip install command above.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>>>>>>>>>> valentyn@google.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> > Any ideas on how to debug where this requirement is coming
>>>>>>>>>>>>>> from?
>>>>>>>>>>>>>> You could try installing and calling pipdeptree [1] from a
>>>>>>>>>>>>>> Jenkins job, and see if it helps.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com>
>>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> However, when I do
>>>>>>>>>>>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> locally, the google-auth package is not installed at all.
>>>>>>>>>>>>>>> Any ideas on how to debug where this requirement is coming
>>>>>>>>>>>>>>> from?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Ahmet Altay <al...@google.com>.

On Wed, Jun 10, 2020 at 7:11 PM Bu Sun Kim <bu...@google.com> wrote:

> Hi,
>
> google-auth has been released (with the wider pin
> <https://github.com/googleapis/google-auth-library-python/blob/6350834ee25295e0754b6611fdff257668a0b0c4/setup.py#L24> on
> rsa).
>

Thank you! Much appreciated!


>
> On Wed, Jun 10, 2020 at 6:07 PM Ahmet Altay <al...@google.com> wrote:
>
>>
>>
>> On Wed, Jun 10, 2020 at 4:07 PM Kyle Weaver <kc...@google.com> wrote:
>>
>>> The fix to google-auth has been merged. Is the plan just to wait until a
>>> new version of google-auth is released and ignore the failing tests until
>>> then? (btw I filed a JIRA for this before I realized it was already being
>>> discussed here: https://issues.apache.org/jira/browse/BEAM-10232)
>>>
>>
>> Could we add it as a test dependency? Or if that is not possible, add it
>> but remove it before next release?
>>
>> It seems like there is a release PR on google-auth (
>> https://github.com/googleapis/google-auth-library-python/pull/525). I
>> asked +Bu Sun Kim <bu...@google.com> on the PR, they usually release
>> pretty quickly.
>>
>>
>>>
>>> On Wed, Jun 10, 2020 at 3:21 PM Udi Meiri <eh...@google.com> wrote:
>>>
>>>> Yes you're right, Py2 envs are still using 4.0.
>>>>
>>>> On Wed, Jun 10, 2020 at 3:03 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri <eh...@google.com> wrote:
>>>>>
>>>>>> 4.1 drops Python 2 support, so I'm not sure if we're ready for that
>>>>>> yet.
>>>>>>
>>>>>
>>>>> Wouldn't that work by default? In python 2 oauth2client's rsa>3.14
>>>>> requirement will resolve to latest python2 supporting version of rsa (4.0?)
>>>>>
>>>>>
>>>>>>
>>>>>> On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>
>>>>>>> Looks like there is an attempt to fix this:
>>>>>>> https://github.com/googleapis/google-auth-library-python/pull/524
>>>>>>>
>>>>>>> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <ke...@apache.org>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> You may be interested in following
>>>>>>>>>> https://github.com/pypa/pip/issues/988 if you are not already.
>>>>>>>>>>
>>>>>>>>>> Kenn
>>>>>>>>>>
>>>>>>>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Seems like manually installing rsa==4.0 satisfies deps, but pip
>>>>>>>>>>> doesn't do transitive deps well.
>>>>>>>>>>>
>>>>>>>>>>> Would it be right to put a direct dependency on rsa<4.1,>=3.1.4
>>>>>>>>>>> in setup.py?
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> Did you find where the google-auth dependency is coming from? We
>>>>>>>>> might try to fix the problem at the source of that dependency instead of
>>>>>>>>> adding rsa to beam's setup.py.
>>>>>>>>>
>>>>>>>>
>>>>>>>> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1 was
>>>>>>>> released today.
>>>>>>>> The places that require rsa<4.1 are deeper in the dependency tree.
>>>>>>>> For example:
>>>>>>>>
>>>>>>>> google-cloud-bigquery==1.24.0
>>>>>>>>   - google-api-core [required: >=1.15.0,<2.0dev, installed: 1.20.0]
>>>>>>>>     - google-auth [required: >=1.14.0,<2.0dev, installed: 1.16.1]
>>>>>>>>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Thanks, that helped in an unexpected way. :)
>>>>>>>>>>>> I should have used the "gcp" extra instead of "cloud" in my pip
>>>>>>>>>>>> install command above.
>>>>>>>>>>>>
>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>>>>>>>>> valentyn@google.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> > Any ideas on how to debug where this requirement is coming
>>>>>>>>>>>>> from?
>>>>>>>>>>>>> You could try installing and calling pipdeptree [1] from a
>>>>>>>>>>>>> Jenkins job, and see if it helps.
>>>>>>>>>>>>>
>>>>>>>>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> However, when I do
>>>>>>>>>>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> locally, the google-auth package is not installed at all.
>>>>>>>>>>>>>> Any ideas on how to debug where this requirement is coming
>>>>>>>>>>>>>> from?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Bu Sun Kim <bu...@google.com>.

Hi,

google-auth has been released (with the wider pin
<https://github.com/googleapis/google-auth-library-python/blob/6350834ee25295e0754b6611fdff257668a0b0c4/setup.py#L24>
on
rsa).

On Wed, Jun 10, 2020 at 6:07 PM Ahmet Altay <al...@google.com> wrote:

>
>
> On Wed, Jun 10, 2020 at 4:07 PM Kyle Weaver <kc...@google.com> wrote:
>
>> The fix to google-auth has been merged. Is the plan just to wait until a
>> new version of google-auth is released and ignore the failing tests until
>> then? (btw I filed a JIRA for this before I realized it was already being
>> discussed here: https://issues.apache.org/jira/browse/BEAM-10232)
>>
>
> Could we add it as a test dependency? Or if that is not possible, add it
> but remove it before next release?
>
> It seems like there is a release PR on google-auth (
> https://github.com/googleapis/google-auth-library-python/pull/525). I
> asked +Bu Sun Kim <bu...@google.com> on the PR, they usually release
> pretty quickly.
>
>
>>
>> On Wed, Jun 10, 2020 at 3:21 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> Yes you're right, Py2 envs are still using 4.0.
>>>
>>> On Wed, Jun 10, 2020 at 3:03 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>>>
>>>>
>>>> On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri <eh...@google.com> wrote:
>>>>
>>>>> 4.1 drops Python 2 support, so I'm not sure if we're ready for that
>>>>> yet.
>>>>>
>>>>
>>>> Wouldn't that work by default? In python 2 oauth2client's rsa>3.14
>>>> requirement will resolve to latest python2 supporting version of rsa (4.0?)
>>>>
>>>>
>>>>>
>>>>> On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>> Looks like there is an attempt to fix this:
>>>>>> https://github.com/googleapis/google-auth-library-python/pull/524
>>>>>>
>>>>>> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <ke...@apache.org>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> You may be interested in following
>>>>>>>>> https://github.com/pypa/pip/issues/988 if you are not already.
>>>>>>>>>
>>>>>>>>> Kenn
>>>>>>>>>
>>>>>>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Seems like manually installing rsa==4.0 satisfies deps, but pip
>>>>>>>>>> doesn't do transitive deps well.
>>>>>>>>>>
>>>>>>>>>> Would it be right to put a direct dependency on rsa<4.1,>=3.1.4
>>>>>>>>>> in setup.py?
>>>>>>>>>>
>>>>>>>>>
>>>>>>>> Did you find where the google-auth dependency is coming from? We
>>>>>>>> might try to fix the problem at the source of that dependency instead of
>>>>>>>> adding rsa to beam's setup.py.
>>>>>>>>
>>>>>>>
>>>>>>> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1 was
>>>>>>> released today.
>>>>>>> The places that require rsa<4.1 are deeper in the dependency tree.
>>>>>>> For example:
>>>>>>>
>>>>>>> google-cloud-bigquery==1.24.0
>>>>>>>   - google-api-core [required: >=1.15.0,<2.0dev, installed: 1.20.0]
>>>>>>>     - google-auth [required: >=1.14.0,<2.0dev, installed: 1.16.1]
>>>>>>>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Thanks, that helped in an unexpected way. :)
>>>>>>>>>>> I should have used the "gcp" extra instead of "cloud" in my pip
>>>>>>>>>>> install command above.
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>>>>>>>> valentyn@google.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> > Any ideas on how to debug where this requirement is coming
>>>>>>>>>>>> from?
>>>>>>>>>>>> You could try installing and calling pipdeptree [1] from a
>>>>>>>>>>>> Jenkins job, and see if it helps.
>>>>>>>>>>>>
>>>>>>>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com>
>>>>>>>>>>>> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi,
>>>>>>>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>>>>>>>
>>>>>>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>>>>>>>
>>>>>>>>>>>>> However, when I do
>>>>>>>>>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>>>>>>>
>>>>>>>>>>>>> locally, the google-auth package is not installed at all.
>>>>>>>>>>>>> Any ideas on how to debug where this requirement is coming
>>>>>>>>>>>>> from?
>>>>>>>>>>>>>
>>>>>>>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Ahmet Altay <al...@google.com>.

On Wed, Jun 10, 2020 at 4:07 PM Kyle Weaver <kc...@google.com> wrote:

> The fix to google-auth has been merged. Is the plan just to wait until a
> new version of google-auth is released and ignore the failing tests until
> then? (btw I filed a JIRA for this before I realized it was already being
> discussed here: https://issues.apache.org/jira/browse/BEAM-10232)
>

Could we add it as a test dependency? Or if that is not possible, add it
but remove it before next release?

It seems like there is a release PR on google-auth (
https://github.com/googleapis/google-auth-library-python/pull/525). I asked +Bu
Sun Kim <bu...@google.com> on the PR, they usually release pretty
quickly.


>
> On Wed, Jun 10, 2020 at 3:21 PM Udi Meiri <eh...@google.com> wrote:
>
>> Yes you're right, Py2 envs are still using 4.0.
>>
>> On Wed, Jun 10, 2020 at 3:03 PM Ahmet Altay <al...@google.com> wrote:
>>
>>>
>>>
>>> On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri <eh...@google.com> wrote:
>>>
>>>> 4.1 drops Python 2 support, so I'm not sure if we're ready for that yet.
>>>>
>>>
>>> Wouldn't that work by default? In python 2 oauth2client's rsa>3.14
>>> requirement will resolve to latest python2 supporting version of rsa (4.0?)
>>>
>>>
>>>>
>>>> On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>> Looks like there is an attempt to fix this:
>>>>> https://github.com/googleapis/google-auth-library-python/pull/524
>>>>>
>>>>> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <ke...@apache.org>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> You may be interested in following
>>>>>>>> https://github.com/pypa/pip/issues/988 if you are not already.
>>>>>>>>
>>>>>>>> Kenn
>>>>>>>>
>>>>>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Seems like manually installing rsa==4.0 satisfies deps, but pip
>>>>>>>>> doesn't do transitive deps well.
>>>>>>>>>
>>>>>>>>> Would it be right to put a direct dependency on rsa<4.1,>=3.1.4 in
>>>>>>>>> setup.py?
>>>>>>>>>
>>>>>>>>
>>>>>>> Did you find where the google-auth dependency is coming from? We
>>>>>>> might try to fix the problem at the source of that dependency instead of
>>>>>>> adding rsa to beam's setup.py.
>>>>>>>
>>>>>>
>>>>>> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1 was
>>>>>> released today.
>>>>>> The places that require rsa<4.1 are deeper in the dependency tree.
>>>>>> For example:
>>>>>>
>>>>>> google-cloud-bigquery==1.24.0
>>>>>>   - google-api-core [required: >=1.15.0,<2.0dev, installed: 1.20.0]
>>>>>>     - google-auth [required: >=1.14.0,<2.0dev, installed: 1.16.1]
>>>>>>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks, that helped in an unexpected way. :)
>>>>>>>>>> I should have used the "gcp" extra instead of "cloud" in my pip
>>>>>>>>>> install command above.
>>>>>>>>>>
>>>>>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>>>>>>> valentyn@google.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> > Any ideas on how to debug where this requirement is coming
>>>>>>>>>>> from?
>>>>>>>>>>> You could try installing and calling pipdeptree [1] from a
>>>>>>>>>>> Jenkins job, and see if it helps.
>>>>>>>>>>>
>>>>>>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>>>>>>
>>>>>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>>>>>>
>>>>>>>>>>>> However, when I do
>>>>>>>>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>>>>>>
>>>>>>>>>>>> locally, the google-auth package is not installed at all.
>>>>>>>>>>>> Any ideas on how to debug where this requirement is coming from?
>>>>>>>>>>>>
>>>>>>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Kyle Weaver <kc...@google.com>.

The fix to google-auth has been merged. Is the plan just to wait until a
new version of google-auth is released and ignore the failing tests until
then? (btw I filed a JIRA for this before I realized it was already being
discussed here: https://issues.apache.org/jira/browse/BEAM-10232)

On Wed, Jun 10, 2020 at 3:21 PM Udi Meiri <eh...@google.com> wrote:

> Yes you're right, Py2 envs are still using 4.0.
>
> On Wed, Jun 10, 2020 at 3:03 PM Ahmet Altay <al...@google.com> wrote:
>
>>
>>
>> On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> 4.1 drops Python 2 support, so I'm not sure if we're ready for that yet.
>>>
>>
>> Wouldn't that work by default? In python 2 oauth2client's rsa>3.14
>> requirement will resolve to latest python2 supporting version of rsa (4.0?)
>>
>>
>>>
>>> On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>>> Looks like there is an attempt to fix this:
>>>> https://github.com/googleapis/google-auth-library-python/pull/524
>>>>
>>>> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com> wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <ke...@apache.org>
>>>>>> wrote:
>>>>>>
>>>>>>> You may be interested in following
>>>>>>> https://github.com/pypa/pip/issues/988 if you are not already.
>>>>>>>
>>>>>>> Kenn
>>>>>>>
>>>>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>>
>>>>>>>> Seems like manually installing rsa==4.0 satisfies deps, but pip
>>>>>>>> doesn't do transitive deps well.
>>>>>>>>
>>>>>>>> Would it be right to put a direct dependency on rsa<4.1,>=3.1.4 in
>>>>>>>> setup.py?
>>>>>>>>
>>>>>>>
>>>>>> Did you find where the google-auth dependency is coming from? We
>>>>>> might try to fix the problem at the source of that dependency instead of
>>>>>> adding rsa to beam's setup.py.
>>>>>>
>>>>>
>>>>> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1 was
>>>>> released today.
>>>>> The places that require rsa<4.1 are deeper in the dependency tree. For
>>>>> example:
>>>>>
>>>>> google-cloud-bigquery==1.24.0
>>>>>   - google-api-core [required: >=1.15.0,<2.0dev, installed: 1.20.0]
>>>>>     - google-auth [required: >=1.14.0,<2.0dev, installed: 1.16.1]
>>>>>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thanks, that helped in an unexpected way. :)
>>>>>>>>> I should have used the "gcp" extra instead of "cloud" in my pip
>>>>>>>>> install command above.
>>>>>>>>>
>>>>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>>>>>> valentyn@google.com> wrote:
>>>>>>>>>
>>>>>>>>>> > Any ideas on how to debug where this requirement is coming from?
>>>>>>>>>> You could try installing and calling pipdeptree [1] from a
>>>>>>>>>> Jenkins job, and see if it helps.
>>>>>>>>>>
>>>>>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi,
>>>>>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>>>>>
>>>>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>>>>>
>>>>>>>>>>> However, when I do
>>>>>>>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>>>>>
>>>>>>>>>>> locally, the google-auth package is not installed at all.
>>>>>>>>>>> Any ideas on how to debug where this requirement is coming from?
>>>>>>>>>>>
>>>>>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Udi Meiri <eh...@google.com>.

Yes you're right, Py2 envs are still using 4.0.

On Wed, Jun 10, 2020 at 3:03 PM Ahmet Altay <al...@google.com> wrote:

>
>
> On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri <eh...@google.com> wrote:
>
>> 4.1 drops Python 2 support, so I'm not sure if we're ready for that yet.
>>
>
> Wouldn't that work by default? In python 2 oauth2client's rsa>3.14
> requirement will resolve to latest python2 supporting version of rsa (4.0?)
>
>
>>
>> On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <al...@google.com> wrote:
>>
>>> Looks like there is an attempt to fix this:
>>> https://github.com/googleapis/google-auth-library-python/pull/524
>>>
>>> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com> wrote:
>>>
>>>>
>>>>
>>>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com> wrote:
>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <ke...@apache.org>
>>>>> wrote:
>>>>>
>>>>>> You may be interested in following
>>>>>> https://github.com/pypa/pip/issues/988 if you are not already.
>>>>>>
>>>>>> Kenn
>>>>>>
>>>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com> wrote:
>>>>>>
>>>>>>> Seems like manually installing rsa==4.0 satisfies deps, but pip
>>>>>>> doesn't do transitive deps well.
>>>>>>>
>>>>>>> Would it be right to put a direct dependency on rsa<4.1,>=3.1.4 in
>>>>>>> setup.py?
>>>>>>>
>>>>>>
>>>>> Did you find where the google-auth dependency is coming from? We might
>>>>> try to fix the problem at the source of that dependency instead of adding
>>>>> rsa to beam's setup.py.
>>>>>
>>>>
>>>> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1 was
>>>> released today.
>>>> The places that require rsa<4.1 are deeper in the dependency tree. For
>>>> example:
>>>>
>>>> google-cloud-bigquery==1.24.0
>>>>   - google-api-core [required: >=1.15.0,<2.0dev, installed: 1.20.0]
>>>>     - google-auth [required: >=1.14.0,<2.0dev, installed: 1.16.1]
>>>>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>>>>
>>>>
>>>>>
>>>>>>
>>>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com> wrote:
>>>>>>>
>>>>>>>> Thanks, that helped in an unexpected way. :)
>>>>>>>> I should have used the "gcp" extra instead of "cloud" in my pip
>>>>>>>> install command above.
>>>>>>>>
>>>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>>>>> valentyn@google.com> wrote:
>>>>>>>>
>>>>>>>>> > Any ideas on how to debug where this requirement is coming from?
>>>>>>>>> You could try installing and calling pipdeptree [1] from a Jenkins
>>>>>>>>> job, and see if it helps.
>>>>>>>>>
>>>>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>>>>
>>>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>>>>
>>>>>>>>>> However, when I do
>>>>>>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>>>>
>>>>>>>>>> locally, the google-auth package is not installed at all.
>>>>>>>>>> Any ideas on how to debug where this requirement is coming from?
>>>>>>>>>>
>>>>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Ahmet Altay <al...@google.com>.

On Wed, Jun 10, 2020 at 2:25 PM Udi Meiri <eh...@google.com> wrote:

> 4.1 drops Python 2 support, so I'm not sure if we're ready for that yet.
>

Wouldn't that work by default? In python 2 oauth2client's rsa>3.14
requirement will resolve to latest python2 supporting version of rsa (4.0?)


>
> On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <al...@google.com> wrote:
>
>> Looks like there is an attempt to fix this:
>> https://github.com/googleapis/google-auth-library-python/pull/524
>>
>> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com> wrote:
>>
>>>
>>>
>>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com> wrote:
>>>
>>>>
>>>>
>>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <ke...@apache.org>
>>>> wrote:
>>>>
>>>>> You may be interested in following
>>>>> https://github.com/pypa/pip/issues/988 if you are not already.
>>>>>
>>>>> Kenn
>>>>>
>>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com> wrote:
>>>>>
>>>>>> Seems like manually installing rsa==4.0 satisfies deps, but pip
>>>>>> doesn't do transitive deps well.
>>>>>>
>>>>>> Would it be right to put a direct dependency on rsa<4.1,>=3.1.4 in
>>>>>> setup.py?
>>>>>>
>>>>>
>>>> Did you find where the google-auth dependency is coming from? We might
>>>> try to fix the problem at the source of that dependency instead of adding
>>>> rsa to beam's setup.py.
>>>>
>>>
>>> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1 was
>>> released today.
>>> The places that require rsa<4.1 are deeper in the dependency tree. For
>>> example:
>>>
>>> google-cloud-bigquery==1.24.0
>>>   - google-api-core [required: >=1.15.0,<2.0dev, installed: 1.20.0]
>>>     - google-auth [required: >=1.14.0,<2.0dev, installed: 1.16.1]
>>>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>>>
>>>
>>>>
>>>>>
>>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com> wrote:
>>>>>>
>>>>>>> Thanks, that helped in an unexpected way. :)
>>>>>>> I should have used the "gcp" extra instead of "cloud" in my pip
>>>>>>> install command above.
>>>>>>>
>>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>>>> valentyn@google.com> wrote:
>>>>>>>
>>>>>>>> > Any ideas on how to debug where this requirement is coming from?
>>>>>>>> You could try installing and calling pipdeptree [1] from a Jenkins
>>>>>>>> job, and see if it helps.
>>>>>>>>
>>>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Hi,
>>>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>>>
>>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>>>
>>>>>>>>> However, when I do
>>>>>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>>>
>>>>>>>>> locally, the google-auth package is not installed at all.
>>>>>>>>> Any ideas on how to debug where this requirement is coming from?
>>>>>>>>>
>>>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Udi Meiri <eh...@google.com>.

4.1 drops Python 2 support, so I'm not sure if we're ready for that yet.

On Wed, Jun 10, 2020 at 2:20 PM Ahmet Altay <al...@google.com> wrote:

> Looks like there is an attempt to fix this:
> https://github.com/googleapis/google-auth-library-python/pull/524
>
> On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com> wrote:
>
>>
>>
>> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com> wrote:
>>
>>>
>>>
>>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <ke...@apache.org> wrote:
>>>
>>>> You may be interested in following
>>>> https://github.com/pypa/pip/issues/988 if you are not already.
>>>>
>>>> Kenn
>>>>
>>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com> wrote:
>>>>
>>>>> Seems like manually installing rsa==4.0 satisfies deps, but pip
>>>>> doesn't do transitive deps well.
>>>>>
>>>>> Would it be right to put a direct dependency on rsa<4.1,>=3.1.4 in
>>>>> setup.py?
>>>>>
>>>>
>>> Did you find where the google-auth dependency is coming from? We might
>>> try to fix the problem at the source of that dependency instead of adding
>>> rsa to beam's setup.py.
>>>
>>
>> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1 was
>> released today.
>> The places that require rsa<4.1 are deeper in the dependency tree. For
>> example:
>>
>> google-cloud-bigquery==1.24.0
>>   - google-api-core [required: >=1.15.0,<2.0dev, installed: 1.20.0]
>>     - google-auth [required: >=1.14.0,<2.0dev, installed: 1.16.1]
>>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>>
>>
>>>
>>>>
>>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com> wrote:
>>>>>
>>>>>> Thanks, that helped in an unexpected way. :)
>>>>>> I should have used the "gcp" extra instead of "cloud" in my pip
>>>>>> install command above.
>>>>>>
>>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>>> valentyn@google.com> wrote:
>>>>>>
>>>>>>> > Any ideas on how to debug where this requirement is coming from?
>>>>>>> You could try installing and calling pipdeptree [1] from a Jenkins
>>>>>>> job, and see if it helps.
>>>>>>>
>>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>>
>>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>>
>>>>>>>> However, when I do
>>>>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>>
>>>>>>>> locally, the google-auth package is not installed at all.
>>>>>>>> Any ideas on how to debug where this requirement is coming from?
>>>>>>>>
>>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Ahmet Altay <al...@google.com>.

Looks like there is an attempt to fix this:
https://github.com/googleapis/google-auth-library-python/pull/524

On Wed, Jun 10, 2020 at 2:07 PM Udi Meiri <eh...@google.com> wrote:

>
>
> On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com> wrote:
>
>>
>>
>> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <ke...@apache.org> wrote:
>>
>>> You may be interested in following
>>> https://github.com/pypa/pip/issues/988 if you are not already.
>>>
>>> Kenn
>>>
>>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com> wrote:
>>>
>>>> Seems like manually installing rsa==4.0 satisfies deps, but pip doesn't
>>>> do transitive deps well.
>>>>
>>>> Would it be right to put a direct dependency on rsa<4.1,>=3.1.4 in
>>>> setup.py?
>>>>
>>>
>> Did you find where the google-auth dependency is coming from? We might
>> try to fix the problem at the source of that dependency instead of adding
>> rsa to beam's setup.py.
>>
>
> oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1 was
> released today.
> The places that require rsa<4.1 are deeper in the dependency tree. For
> example:
>
> google-cloud-bigquery==1.24.0
>   - google-api-core [required: >=1.15.0,<2.0dev, installed: 1.20.0]
>     - google-auth [required: >=1.14.0,<2.0dev, installed: 1.16.1]
>       - rsa [required: >=3.1.4,<4.1, installed: 4.1]
>
>
>>
>>>
>>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com> wrote:
>>>>
>>>>> Thanks, that helped in an unexpected way. :)
>>>>> I should have used the "gcp" extra instead of "cloud" in my pip
>>>>> install command above.
>>>>>
>>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>>> valentyn@google.com> wrote:
>>>>>
>>>>>> > Any ideas on how to debug where this requirement is coming from?
>>>>>> You could try installing and calling pipdeptree [1] from a Jenkins
>>>>>> job, and see if it helps.
>>>>>>
>>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>>
>>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>>
>>>>>>>
>>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>>
>>>>>>> However, when I do
>>>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>>
>>>>>>> locally, the google-auth package is not installed at all.
>>>>>>> Any ideas on how to debug where this requirement is coming from?
>>>>>>>
>>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Udi Meiri <eh...@google.com>.

On Wed, Jun 10, 2020 at 1:59 PM Ahmet Altay <al...@google.com> wrote:

>
>
> On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <ke...@apache.org> wrote:
>
>> You may be interested in following https://github.com/pypa/pip/issues/988 if
>> you are not already.
>>
>> Kenn
>>
>> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com> wrote:
>>
>>> Seems like manually installing rsa==4.0 satisfies deps, but pip doesn't
>>> do transitive deps well.
>>>
>>> Would it be right to put a direct dependency on rsa<4.1,>=3.1.4 in
>>> setup.py?
>>>
>>
> Did you find where the google-auth dependency is coming from? We might try
> to fix the problem at the source of that dependency instead of adding rsa
> to beam's setup.py.
>

oauth2client depends on rsa>=3.14 with no upper limit. rsa 4.1 was released
today.
The places that require rsa<4.1 are deeper in the dependency tree. For
example:

google-cloud-bigquery==1.24.0
  - google-api-core [required: >=1.15.0,<2.0dev, installed: 1.20.0]
    - google-auth [required: >=1.14.0,<2.0dev, installed: 1.16.1]
      - rsa [required: >=3.1.4,<4.1, installed: 4.1]


>
>>
>>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com> wrote:
>>>
>>>> Thanks, that helped in an unexpected way. :)
>>>> I should have used the "gcp" extra instead of "cloud" in my pip install
>>>> command above.
>>>>
>>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>>> valentyn@google.com> wrote:
>>>>
>>>>> > Any ideas on how to debug where this requirement is coming from?
>>>>> You could try installing and calling pipdeptree [1] from a Jenkins
>>>>> job, and see if it helps.
>>>>>
>>>>> [1] https://pypi.org/project/pipdeptree/
>>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com> wrote:
>>>>>
>>>>>> Hi,
>>>>>> I'm trying to understand these "pip check" failures:
>>>>>>
>>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>>
>>>>>>
>>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>>
>>>>>> However, when I do
>>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>>
>>>>>> locally, the google-auth package is not installed at all.
>>>>>> Any ideas on how to debug where this requirement is coming from?
>>>>>>
>>>>>

Re: python precommit error - google-auth depenedency?

Posted by Ahmet Altay <al...@google.com>.

On Wed, Jun 10, 2020 at 1:29 PM Kenneth Knowles <ke...@apache.org> wrote:

> You may be interested in following https://github.com/pypa/pip/issues/988 if
> you are not already.
>
> Kenn
>
> On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com> wrote:
>
>> Seems like manually installing rsa==4.0 satisfies deps, but pip doesn't
>> do transitive deps well.
>>
>> Would it be right to put a direct dependency on rsa<4.1,>=3.1.4 in
>> setup.py?
>>
>
Did you find where the google-auth dependency is coming from? We might try
to fix the problem at the source of that dependency instead of adding rsa
to beam's setup.py.


>
>> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com> wrote:
>>
>>> Thanks, that helped in an unexpected way. :)
>>> I should have used the "gcp" extra instead of "cloud" in my pip install
>>> command above.
>>>
>>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <
>>> valentyn@google.com> wrote:
>>>
>>>> > Any ideas on how to debug where this requirement is coming from?
>>>> You could try installing and calling pipdeptree [1] from a Jenkins job,
>>>> and see if it helps.
>>>>
>>>> [1] https://pypi.org/project/pipdeptree/
>>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com> wrote:
>>>>
>>>>> Hi,
>>>>> I'm trying to understand these "pip check" failures:
>>>>>
>>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>>
>>>>>
>>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>>
>>>>> However, when I do
>>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>>
>>>>> locally, the google-auth package is not installed at all.
>>>>> Any ideas on how to debug where this requirement is coming from?
>>>>>
>>>>

Re: python precommit error - google-auth depenedency?

Posted by Kenneth Knowles <ke...@apache.org>.

You may be interested in following https://github.com/pypa/pip/issues/988 if
you are not already.

Kenn

On Wed, Jun 10, 2020 at 12:17 PM Udi Meiri <eh...@google.com> wrote:

> Seems like manually installing rsa==4.0 satisfies deps, but pip doesn't do
> transitive deps well.
>
> Would it be right to put a direct dependency on rsa<4.1,>=3.1.4 in
> setup.py?
>
> On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com> wrote:
>
>> Thanks, that helped in an unexpected way. :)
>> I should have used the "gcp" extra instead of "cloud" in my pip install
>> command above.
>>
>> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <va...@google.com>
>> wrote:
>>
>>> > Any ideas on how to debug where this requirement is coming from?
>>> You could try installing and calling pipdeptree [1] from a Jenkins job,
>>> and see if it helps.
>>>
>>> [1] https://pypi.org/project/pipdeptree/
>>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com> wrote:
>>>
>>>> Hi,
>>>> I'm trying to understand these "pip check" failures:
>>>>
>>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>>
>>>>
>>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>>
>>>> However, when I do
>>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>>
>>>> locally, the google-auth package is not installed at all.
>>>> Any ideas on how to debug where this requirement is coming from?
>>>>
>>>

Re: python precommit error - google-auth depenedency?

Posted by Udi Meiri <eh...@google.com>.

Seems like manually installing rsa==4.0 satisfies deps, but pip doesn't do
transitive deps well.

Would it be right to put a direct dependency on rsa<4.1,>=3.1.4 in setup.py?

On Wed, Jun 10, 2020 at 11:48 AM Udi Meiri <eh...@google.com> wrote:

> Thanks, that helped in an unexpected way. :)
> I should have used the "gcp" extra instead of "cloud" in my pip install
> command above.
>
> On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <va...@google.com>
> wrote:
>
>> > Any ideas on how to debug where this requirement is coming from?
>> You could try installing and calling pipdeptree [1] from a Jenkins job,
>> and see if it helps.
>>
>> [1] https://pypi.org/project/pipdeptree/
>> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com> wrote:
>>
>>> Hi,
>>> I'm trying to understand these "pip check" failures:
>>>
>>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>>
>>>
>>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>>
>>> However, when I do
>>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>>
>>> locally, the google-auth package is not installed at all.
>>> Any ideas on how to debug where this requirement is coming from?
>>>
>>

Re: python precommit error - google-auth depenedency?

Posted by Udi Meiri <eh...@google.com>.

Thanks, that helped in an unexpected way. :)
I should have used the "gcp" extra instead of "cloud" in my pip install
command above.

On Wed, Jun 10, 2020 at 11:37 AM Valentyn Tymofieiev <va...@google.com>
wrote:

> > Any ideas on how to debug where this requirement is coming from?
> You could try installing and calling pipdeptree [1] from a Jenkins job,
> and see if it helps.
>
> [1] https://pypi.org/project/pipdeptree/
> On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com> wrote:
>
>> Hi,
>> I'm trying to understand these "pip check" failures:
>>
>> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>>
>>
>> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>>
>> However, when I do
>> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>>
>> locally, the google-auth package is not installed at all.
>> Any ideas on how to debug where this requirement is coming from?
>>
>

Re: python precommit error - google-auth depenedency?

Posted by Valentyn Tymofieiev <va...@google.com>.

> Any ideas on how to debug where this requirement is coming from?
You could try installing and calling pipdeptree [1] from a Jenkins job, and
see if it helps.

[1] https://pypi.org/project/pipdeptree/
On Wed, Jun 10, 2020 at 11:00 AM Udi Meiri <eh...@google.com> wrote:

> Hi,
> I'm trying to understand these "pip check" failures:
>
> ERROR: google-auth 1.16.1 has requirement rsa<4.1,>=3.1.4, but you'll have rsa 4.1 which is incompatible
>
>
> https://builds.apache.org/job/beam_PreCommit_Python_Cron/2860/console
>
> However, when I do
> pip install dist/apache-beam-2.23.0.dev0.tar.gz[test,cloud]
>
> locally, the google-auth package is not installed at all.
> Any ideas on how to debug where this requirement is coming from?
>