You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/07/11 11:27:36 UTC
[GitHub] [beam] NicAlis opened a new issue, #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build Cloud Build
NicAlis opened a new issue, #22218:
URL: https://github.com/apache/beam/issues/22218
Cloud Build keeps timing out while trying to install apache beam during docker build due to continuous warnings: google-api-core 2.8.2 does not provide the extra 'grpcgcp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] richardcann commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
richardcann commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1180554672
As a temporary fix Ive had to use `poetry` instead of `pip` and it works fine. Would like to know why the dep tree for this using `pip` forces the installation of `google-api-core 2.8.2` which I can see has removed the extra `grpcgcp` in [this pull request of latest version](https://github.com/googleapis/python-api-core/pull/401)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jkukul commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
jkukul commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1182024641
Another workaround is to use an older version of `pip`. Older versions (before `21.3`) somehow don't force the installation of `google-api-core 2.8.2`. Or, rather, an older `pip` seems to be able to faster figure out the working version of dependencies.
In your `Dockerfile` change:
```
RUN pip install --upgrade pip
````
to:
```
pip install pip==21.2.4
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #22218: Installation of apache-beam[gcp] in a clean environment gets stuck in pip's dependency resolver.
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1182642891
We are working closely with GCP client libraries maintainers to mitigate the issue. Another workaround is to use apache-beam[gcp]==2.33.0 or earlier that are not as adversely affected.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #22218: Installation of apache-beam[gcp] in a clean environment gets stuck in pip's dependency resolver.
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1183543540
This is mitigated for now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #22218: Installation of apache-beam[gcp] in a clean environment gets stuck in pip's dependency resolver.
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1182601148
As a workaround, users can give pip some hints for which dependencies to install to satisfy the constraints. Currently, the below commands suffice:
```
pip install google-api-core==1.31.6
pip install google-cloud-pubsub==2.13.1
google-cloud-bigquery-storage==2.13.2
pip install apache-beam[gcp]
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1180607473
This is due to an upstream change of google-cloud-bigquery-storage which bumped google-cloud-core version requirements. Needs to patch docker file as in https://github.com/apache/beam/pull/22205
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] juanan-hernandez commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
juanan-hernandez commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1180313609
+1 here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1180813351
Do we know what recent release of google cloud libraries have caused this change?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] NicAlis commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
NicAlis commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1182003015
I noticed there was something wrong again today but ended up giving up. Any possible fixes for this? a bit annoying that they keep rolling out changes that break stuff, as its holding back some important work for us.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1181790661
Same problem happened again, this time is due to the rollout of google-cloud-pubsub 2.13.2: https://github.com/googleapis/python-pubsub/pull/726
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1180649095
> As a temporary fix Ive had to use `poetry` instead of `pip` and it works fine. Would like to know why the dep tree for this using `pip` forces the installation of `google-api-core 2.8.2` which I can see has removed the extra `grpcgcp` in [this pull request of latest version](https://github.com/googleapis/python-api-core/pull/401)
For released versions, a workaround could be manually install google-cloud-bigquery-storage<2.14 prior installing apache beam:
```
RUN pip install google-cloud-bigquery-storage==2.13.0
RUN pip install apache-beam[gcp]==2.38.0
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1180814548
is it https://pypi.org/project/google-cloud-bigquery-storage/#history ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #22218: Installation of apache-beam[gcp] in a clean environment gets stuck in pip's dependency resolver.
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1196014115
The issue is still reproducible on some older versions of pip, for example on `pip==21.1.1`. Make sure to update to newer versions of pip (`pip install --upgrade pip`) if you encounter it.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] NicAlis commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
NicAlis commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1180669980
> RUN pip install google-cloud-bigquery-storage==2.13.0
This worked for me. Thanks @Abacn.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] NicAlis commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
NicAlis commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1180807519
maybe slightly unrelated but if launching the above image via Dataflow job via Flex Template launcher, the GCE instance setting up the job runs into the same issue (google-api-core 2.8.2 does not provide the extra 'grpcgcp'), even though the SDK image I have built, using the above fix, does not use `google-api-core 2.8.2`. I have raised an issue with the Dataflow team: https://issuetracker.google.com/issues/238658546.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] NicAlis commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
NicAlis commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1182182527
worked, thanks @jkukul!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #22218: Installation of apache-beam[gcp] in a clean environment gets stuck in pip's dependency resolver.
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1182651943
> Another workaround is to use an older version of pip
Older pip version just install conflicting dependencies without resolving the conflicts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn closed issue #22218: Installation of apache-beam[gcp] in a clean environment gets stuck in pip's dependency resolver.
Posted by GitBox <gi...@apache.org>.
tvalentyn closed issue #22218: Installation of apache-beam[gcp] in a clean environment gets stuck in pip's dependency resolver.
URL: https://github.com/apache/beam/issues/22218
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1180819057
nvm, it's answered above
> This is due to an upstream change of google-cloud-bigquery-storage which bumped google-cloud-core version requirements
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] josejmacias commented on issue #22218: Installation of apache-beam[gcp] in a clean environment gets stuck in pip's dependency resolver.
Posted by GitBox <gi...@apache.org>.
josejmacias commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1182987289
Only degrading to v2.33.0 was possible in my case.
I also had to remove from installing any lib considered as dependencies by apache-beam and limit thoses that are not such as storage (google-cloud-storage==2.4.0) or secret-manager (google-cloud-secret-manager==2.11.1).
Hope it helps
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #22218: google-api-core 2.8.2 does not provide the extra 'grpcgcp' warning while installing apache-beam[gcp] during Docker build in Cloud Build
Posted by GitBox <gi...@apache.org>.
Abacn commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1180622784
The 'grpcgcp' warning is popped when installing google-cloud-spanner v1.x, which used this extra for google-api-core that recently been removed: https://github.com/googleapis/python-spanner/blob/v1.7.1/setup.py#L32
This warning does not causing docker build timeout. It rather indicates that PyPI is failing to resolve dependency and doing crazy backtrackings
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #22218: Installation of apache-beam[gcp] in a clean environment gets stuck in pip's dependency resolver.
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #22218:
URL: https://github.com/apache/beam/issues/22218#issuecomment-1226602196
Retrospective: https://docs.google.com/document/d/1gxQF8mciRYgACNpCy1wlR7TBa8zN-Tl6PebW-U8QvBk
Discussion on dev@: https://lists.apache.org/thread/3sk5x70plw0s2ttb3lmh816q9hl60kp5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org