You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by GitBox <gi...@apache.org> on 2022/11/04 13:28:10 UTC
[GitHub] [beam] cypreess opened a new issue, #23982: [Bug]: Could not find a version that matches protobuf
cypreess opened a new issue, #23982:
URL: https://github.com/apache/beam/issues/23982
### What happened?
Cannot install latest apache-beam[gcp]==2.42.0 because there is a conflict of protobuf version dependencies.
```
Could not find a version that matches protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<4,<4.0.0dev,<5.0.0dev,>=3.12.0,>=3.12.2,>=3.15.0,>=3.19.0,>=3.19.5,>=4.21.6 (from proto-plus==1.22.1->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
Tried: 2.0.3, 2.3.0, 2.4.1, 2.5.0, 2.6.0, 2.6.1, 3.0.0, 3.0.0, 3.1.0, 3.1.0.post1, 3.1.0.post1, 3.2.0, 3.2.0, 3.3.0, 3.4.0, 3.4.0, 3.5.0.post1, 3.5.0.post1, 3.5.1, 3.5.1, 3.5.2, 3.5.2, 3.5.2.post1, 3.5.2.post1, 3.6.0, 3.6.0, 3.6.1, 3.6.1, 3.7.0, 3.7.0, 3.7.1, 3.7.1, 3.8.0, 3.8.0, 3.9.0, 3.9.0, 3.9.1, 3.9.1, 3.9.2, 3.9.2, 3.10.0, 3.10.0, 3.11.0, 3.11.0, 3.11.1, 3.11.1, 3.11.2, 3.11.2, 3.11.3, 3.11.3, 3.12.2, 3.12.2, 3.12.4, 3.12.4, 3.13.0, 3.13.0, 3.14.0, 3.14.0, 3.15.0, 3.15.0, 3.15.1, 3.15.1, 3.15.2, 3.15.2, 3.15.3, 3.15.3, 3.15.4, 3.15.4, 3.15.5, 3.15.5, 3.15.6, 3.15.6, 3.15.7, 3.15.7, 3.15.8, 3.15.8, 3.16.0, 3.16.0, 3.17.0, 3.17.0, 3.17.0, 3.17.1, 3.17.1, 3.17.1, 3.17.2, 3.17.2, 3.17.2, 3.17.3, 3.17.3, 3.17.3, 3.18.0, 3.18.0, 3.18.0, 3.18.1, 3.18.1, 3.18.1, 3.18.3, 3.18.3, 3.18.3, 3.19.0, 3.19.0, 3.19.0, 3.19.1, 3.19.1, 3.19.1, 3.19.2, 3.19.2, 3.19.2, 3.19.3, 3.19.3, 3.19.3, 3.19.4, 3.19.4, 3.19.4, 3.19.5, 3.19.5, 3.19.5, 3.19.6, 3.19.6, 3.19.6, 3.20.0, 3.20.0, 3.20.0, 3.20.1,
3.20.1, 3.20.1, 3.20.2, 3.20.2, 3.20.2, 3.20.3, 3.20.3, 3.20.3, 4.21.0, 4.21.0, 4.21.0, 4.21.0, 4.21.1, 4.21.1, 4.21.1, 4.21.1, 4.21.2, 4.21.2, 4.21.2, 4.21.2, 4.21.3, 4.21.3, 4.21.3, 4.21.3, 4.21.4, 4.21.4, 4.21.4, 4.21.4, 4.21.5, 4.21.5, 4.21.5, 4.21.5, 4.21.6, 4.21.6, 4.21.6, 4.21.6, 4.21.7, 4.21.7, 4.21.7, 4.21.7, 4.21.8, 4.21.8, 4.21.8, 4.21.8, 4.21.9, 4.21.9, 4.21.9, 4.21.9
Skipped pre-versions: 2.0.0b0, 3.0.0a2, 3.0.0a3, 3.0.0b1, 3.0.0b1.post1, 3.0.0b1.post2, 3.0.0b2, 3.0.0b2, 3.0.0b2.post1, 3.0.0b2.post1, 3.0.0b2.post2, 3.0.0b2.post2, 3.0.0b3, 3.0.0b4, 3.0.0b4, 3.2.0rc1, 3.2.0rc1, 3.2.0rc1.post1, 3.2.0rc1.post1, 3.2.0rc2, 3.2.0rc2, 3.7.0rc2, 3.7.0rc2, 3.7.0rc3, 3.7.0rc3, 3.8.0rc1, 3.8.0rc1, 3.9.0rc1, 3.9.0rc1, 3.10.0rc1, 3.10.0rc1, 3.11.0rc1, 3.11.0rc1, 3.11.0rc2, 3.11.0rc2, 3.13.0rc3, 3.13.0rc3, 3.14.0rc1, 3.14.0rc1, 3.14.0rc2, 3.14.0rc2, 3.14.0rc3, 3.14.0rc3, 3.15.0rc1, 3.15.0rc1, 3.15.0rc2, 3.15.0rc2, 3.16.0rc1, 3.16.0rc1, 3.16.0rc2, 3.16.0rc2, 3.17.0rc1, 3.17.0rc1, 3.17.0rc1, 3.17.0rc2, 3.17.0rc2, 3.17.0rc2, 3.18.0rc1, 3.18.0rc1, 3.18.0rc1, 3.18.0rc2, 3.18.0rc2, 3.18.0rc2, 3.19.0rc1, 3.19.0rc1, 3.19.0rc1, 3.19.0rc2, 3.19.0rc2, 3.19.0rc2, 3.20.0rc1, 3.20.0rc1, 3.20.0rc1, 3.20.0rc2, 3.20.0rc2, 3.20.0rc2, 3.20.1rc1, 3.20.1rc1, 3.20.1rc1, 4.0.0rc1, 4.0.0rc1, 4.0.0rc2, 4.0.0rc2, 4.21.0rc1, 4.21.0rc1, 4.21.0rc1, 4.21.0rc1, 4.21.0rc2, 4.21.0rc2, 4.21.
0rc2, 4.21.0rc2
There are incompatible versions in the resolved dependencies:
protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 (from google-api-core[grpc,grpcgcp]==2.10.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf<5.0.0dev,>=3.19.0 (from google-cloud-recommendations-ai==0.7.1->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 (from google-cloud-dlp==3.9.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf>=4.21.6 (from grpcio-status==1.50.0->google-cloud-pubsub==2.13.10->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf<4.0.0dev (from google-cloud-bigtable==1.7.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf<4.0.0dev,>=3.12.0 (from google-cloud-bigquery==2.34.4->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf<5.0.0dev,>=3.15.0 (from googleapis-common-protos==1.56.4->google-api-core==2.10.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf<4.0.0dev (from google-cloud-datastore==1.15.5->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf<4.0.0dev,>=3.19.0 (from google-cloud-bigquery-storage==2.13.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf<4.0.0dev (from google-cloud-language==1.3.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf<4.0.0dev (from google-cloud-videointelligence==1.16.3->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 (from google-cloud-pubsub==2.13.10->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf<4.0.0dev (from google-cloud-spanner==1.19.3->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf<5.0.0dev,>=3.19.0 (from proto-plus==1.22.1->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf<4.0.0dev (from google-cloud-vision==1.0.2->apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
protobuf<4,>=3.12.2 (from apache-beam[gcp]==2.42.0->uni-bi-export (setup.py))
```
### Issue Priority
Priority: 1
### Issue Component
Component: io-py-gcp
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
AnandInguva commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1341296362
> @AnandInguva do we still need this mac-os only restriction?
>
> https://github.com/apache/beam/blob/da100f967a6fbc86a2cc4ae6c9fe69180c70e74f/sdks/python/setup.py#L189
>
> IIRC we added b/c MacOS github action tests were failing. but if that not an issue anymore, it would be good to relax that.
Yes, I will take a look at this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
AnandInguva commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1341294272
```
#
# This file is autogenerated by pip-compile with Python 3.8
# by the following command:
#
# pip-compile --resolver=backtracking
#
--extra-index-url https://pypi.ngc.nvidia.com
--trusted-host pypi.ngc.nvidia.com
apache-beam[gcp]==2.43.0
# via -r requirements.in
cachetools==4.2.4
# via
# apache-beam
# google-auth
certifi==2022.9.24
# via requests
charset-normalizer==2.1.1
# via requests
cloudpickle==2.2.0
# via apache-beam
crcmod==1.7
# via apache-beam
dill==0.3.1.1
# via apache-beam
docopt==0.6.2
# via hdfs
fastavro==1.7.0
# via apache-beam
fasteners==0.18
# via
# apache-beam
# google-apitools
google-api-core[grpc]==2.11.0
# via
# google-cloud-bigquery
# google-cloud-bigquery-storage
# google-cloud-bigtable
# google-cloud-core
# google-cloud-datastore
# google-cloud-dlp
# google-cloud-language
# google-cloud-pubsub
# google-cloud-pubsublite
# google-cloud-recommendations-ai
# google-cloud-secret-manager
# google-cloud-spanner
# google-cloud-videointelligence
# google-cloud-vision
google-apitools==0.5.31
# via apache-beam
google-auth==2.15.0
# via
# apache-beam
# google-api-core
# google-auth-httplib2
# google-cloud-core
google-auth-httplib2==0.1.0
# via apache-beam
google-cloud-bigquery==3.4.0
# via apache-beam
google-cloud-bigquery-storage==2.13.2
# via
# apache-beam
# google-cloud-bigquery
google-cloud-bigtable==1.7.3
# via apache-beam
google-cloud-core==2.3.2
# via
# apache-beam
# google-cloud-bigquery
# google-cloud-bigtable
# google-cloud-datastore
# google-cloud-spanner
google-cloud-datastore==1.15.5
# via apache-beam
google-cloud-dlp==3.9.2
# via apache-beam
google-cloud-language==1.3.2
# via apache-beam
google-cloud-pubsub==2.13.11
# via
# apache-beam
# google-cloud-pubsublite
google-cloud-pubsublite==1.6.0
# via apache-beam
google-cloud-recommendations-ai==0.7.1
# via apache-beam
google-cloud-secret-manager==2.12.6
# via -r requirements.in
google-cloud-spanner==3.24.0
# via apache-beam
google-cloud-videointelligence==1.16.3
# via apache-beam
google-cloud-vision==1.0.2
# via apache-beam
google-crc32c==1.5.0
# via google-resumable-media
google-resumable-media==2.4.0
# via google-cloud-bigquery
googleapis-common-protos[grpc]==1.57.0
# via
# google-api-core
# grpc-google-iam-v1
# grpcio-status
grpc-google-iam-v1==0.12.4
# via
# google-cloud-bigtable
# google-cloud-pubsub
# google-cloud-secret-manager
# google-cloud-spanner
grpcio==1.51.1
# via
# apache-beam
# google-api-core
# google-cloud-bigquery
# google-cloud-pubsub
# google-cloud-pubsublite
# googleapis-common-protos
# grpc-google-iam-v1
# grpcio-status
grpcio-status==1.48.2
# via
# google-api-core
# google-cloud-pubsub
# google-cloud-pubsublite
hdfs==2.7.0
# via apache-beam
httplib2==0.20.4
# via
# apache-beam
# google-apitools
# google-auth-httplib2
# oauth2client
idna==3.4
# via requests
numpy==1.22.4
# via
# apache-beam
# pyarrow
oauth2client==4.1.3
# via google-apitools
objsize==0.5.2
# via apache-beam
orjson==3.8.3
# via apache-beam
overrides==6.5.0
# via google-cloud-pubsublite
packaging==21.3
# via
# google-cloud-bigquery
# google-cloud-spanner
proto-plus==1.22.1
# via
# apache-beam
# google-cloud-bigquery
# google-cloud-bigquery-storage
# google-cloud-dlp
# google-cloud-pubsub
# google-cloud-recommendations-ai
# google-cloud-secret-manager
# google-cloud-spanner
protobuf==3.20.3
# via
# apache-beam
# google-api-core
# google-cloud-bigquery
# google-cloud-bigquery-storage
# google-cloud-bigtable
# google-cloud-datastore
# google-cloud-dlp
# google-cloud-language
# google-cloud-pubsub
# google-cloud-recommendations-ai
# google-cloud-secret-manager
# google-cloud-spanner
# google-cloud-videointelligence
# google-cloud-vision
# googleapis-common-protos
# grpcio-status
# proto-plus
pyarrow==9.0.0
# via
# apache-beam
# google-cloud-bigquery
pyasn1==0.4.8
# via
# oauth2client
# pyasn1-modules
# rsa
pyasn1-modules==0.2.8
# via
# google-auth
# oauth2client
pydot==1.4.2
# via apache-beam
pymongo==3.13.0
# via apache-beam
pyparsing==3.0.9
# via
# httplib2
# packaging
# pydot
python-dateutil==2.8.2
# via
# apache-beam
# google-cloud-bigquery
pytz==2022.6
# via apache-beam
regex==2022.10.31
# via apache-beam
requests==2.28.1
# via
# apache-beam
# google-api-core
# google-cloud-bigquery
# hdfs
rsa==4.9
# via
# google-auth
# oauth2client
six==1.16.0
# via
# google-apitools
# google-auth
# google-auth-httplib2
# hdfs
# oauth2client
# python-dateutil
sqlparse==0.4.3
# via google-cloud-spanner
typing-extensions==4.4.0
# via apache-beam
urllib3==1.26.13
# via requests
zstandard==0.19.0
# via apache-beam
```
Output with `pip-compile --resolver backtracking`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jhoareau commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
jhoareau commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1387116407
Indeed @tvalentyn @AnandInguva https://pypi.org/pypi/apache-beam/2.43.0/json is the source of truth for Poetry, and this source of truth seems to be overly restrictive. Platform constraints sound like a reasonable approach.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] LaurenzReitsam commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
LaurenzReitsam commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1341089003
@tvalentyn yes it is a problem with pip-tools.
Problem can be fixed with adding `grpcio-status==1.48.2` as dependency... 🙄
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] LaurenzReitsam commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
LaurenzReitsam commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1341182804
`grpcio-status` is not an apache-beam dependency. This is just a workaround for me to fix issues with pipenv.
Just pinning `protobuf<4.0.0` does not solve the issue.
---
Just tried the same procedure with `poetry` as an alternative dependency solver. Seems like the same error:
```shell
pip install poetry
mkdir foo && cd ./foo
poetry init
poetry add apache-beam[gcp] google-cloud-secret-manager
```
```text
Because no versions of google-cloud-secret-manager match >2.12.6,<3.0.0
and google-cloud-secret-manager (2.12.6) depends on protobuf (>=3.19.5,<3.20.0 || >3.20.0,<3.20.1 || >3.20.1,<4.21.0 || >4.21.0,<4.21.1 || >4.21.1,<4.21.2 || >4.21.2,<4.21.3 || >4.21.3,<4.21.4 || >4.21.4,<4.21.5 || >4.21.5,<5.0.0dev), google-cloud-secret-manager (>=2.12.6,<3.0.0) requires protobuf (>=3.19.5,<3.20.0 || >3.20.0,<3.20.1 || >3.20.1,<4.21.0 || >4.21.0,<4.21.1 || >4.21.1,<4.21.2 || >4.21.2,<4.21.3 || >4.21.3,<4.21.4 || >4.21.4,<4.21.5 || >4.21.5,<5.0.0dev).
And because apache-beam (2.43.0) depends on protobuf (>3.12.2,<3.19.5)
and no versions of apache-beam match >2.43.0,<3.0.0, google-cloud-secret-manager (>=2.12.6,<3.0.0) is incompatible with apache-beam (>=2.43.0,<3.0.0).
So, because foo depends on both apache-beam (^2.43.0) and google-cloud-secret-manager (^2.12.6), version solving failed.
```
Adding `protobuf<4.0.0` does not change result.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jhoareau commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
jhoareau commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1380366709
Hi @AnandInguva we are blocked because of the above Protobuf restriction ('protobuf>3.12.2,<3.19.5') applies to all environments (also non-MacOS) when using Poetry.
The reason is that the PyPI repository has this quite restrictive requirement in place: https://pypi.org/pypi/apache-beam/2.43.0/json instead of the looser "<4" for non-macOS environments.
It'd be great if you could either remove the condition from `setup.py` or fix the PyPI API requirements!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
AnandInguva commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1341292945
> ```shell
> echo -e "apache-beam[gcp]\ngoogle-cloud-secret-manager" > requirements.in
> pip-compile
> ```
It is due to pip-compile resolver.
You can try using
`pip-compile --resolver backtracking` with latest version of `pip-tools`. You can find more details at https://github.com/jazzband/pip-tools/issues/1190
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1319276443
@cypreess I don't reproduce the issue in a clean virutal environment. Please add relevant details if you have and reopen the issue if still present.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] LaurenzReitsam commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
LaurenzReitsam commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1341346390
@AnandInguva you are right `pip-compile --resolver backtracking` works for me on **Linux + Python3.9**.
Just tried this on **Windows + Python3.10** until now what resulted in several errors...
```
...
numpy_linalg_lapack_lite:
FOUND:
language = c
define_macros = [('HAVE_BLAS_ILP64', None), ('BLAS_SYMBOL_SUFFIX', '64_')]
C:\Users\laurenz.reitsam\.pyenv\pyenv-win\versions\3.10.4\lib\distutils\dist.py:274: UserWarning: Unknown distribution option: 'define_macros'
warnings.warn(msg)
running dist_info
running build_src
build_src
building py_modules sources
creating build
creating build\src.win-amd64-3.10
creating build\src.win-amd64-3.10\numpy
creating build\src.win-amd64-3.10\numpy\distutils
building library "npymath" sources
error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio": https://visualstudio.microsoft.com/downloads/
...
```
Concerning poetry I had the issue in both environments.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1386289034
intresting. is poetry using : the metadata from https://packaging.python.org/en/latest/specifications/core-metadata/#requires-dist-multiple-use / https://pypi.org/pypi/apache-beam/2.43.0/json as SOT?
it appears to be generated incorrectly. @AnandInguva I think we could fix it by adding a platform constraint (see https://peps.python.org/pep-0508/) instead of evaluating the platform dynamically in https://github.com/apache/beam/blob/f40057b7fb68ca556be7fbd0cc420311937dcf80/sdks/python/setup.py#L183
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] jhoareau commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
jhoareau commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1381749656
I understand that. However, the setup.py file seems to have the loose bound (protobuf>3.12.2,<4), while the PyPI bound is tighter, like the macOS build one.
Since PyPI usecases usually do not include building but simply installing from wheel, I think it would make sense to relax the PyPI bound to be the same as the loose setup.py bound. Please correct me if I'm wrong!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
AnandInguva commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1380795361
Yes, protobuf is tightly bounded and as of now, there are efforts going on for upgrading protobuf to 4.x.x version in Apache Beam.
We need to have a tight restriction in protobuf because protobuf is a build dependency of beam and minor changes on protobuf version needs to be validated so we couldn't have a loose bound.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] kennknowles commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
kennknowles commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1314198191
This appears to be caused by other dependencies in your project, perhaps `proto-plus`. Or if there is an incompatibility in the GCP dependencies, we would certainly like to have a set of dependencies that works together. And of course it would be useful for Beam to be compatible with as many things as possible.
CC @tvalentyn
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn closed issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
tvalentyn closed issue #23982: [Bug]: Could not find a version that matches protobuf
URL: https://github.com/apache/beam/issues/23982
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] LaurenzReitsam commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
LaurenzReitsam commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1340479712
I have a similar issue. Seems like `apache-beam[gcp]` dependencies (``protobuf==3.*``) are not up to date with other google packages (``protobuf==4.*``).
```shell
python -m venv .venv
source .venv/bin/activate
pip install pip-tools
echo "apache-beam[gcp] google-cloud-secret-manager" > requirements.in
pip-compile
```
Out:
```
There are incompatible versions in the resolved dependencies:
protobuf<5.0.0dev,>=3.19.0 (from google-cloud-recommendations-ai==0.7.1->apache-beam[gcp]==2.43.0->-r requirements.in (line 5))
protobuf<4.0.0dev (from google-cloud-bigtable==1.7.3->apache-beam[gcp]==2.43.0->-r requirements.in (line 5))
protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 (from google-cloud-dlp==3.9.2->apache-beam[gcp]==2.43.0->-r requirements.in (line 5))
protobuf<4.0.0dev (from google-cloud-language==1.3.2->apache-beam[gcp]==2.43.0->-r requirements.in (line 5))
protobuf<4,>3.12.2 (from apache-beam[gcp]==2.43.0->-r requirements.in (line 5))
protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 (from google-api-core[grpc]==2.11.0->google-cloud-secret-manager==2.12.6->-r requirements.in (line 6))
protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 (from google-cloud-secret-manager==2.12.6->-r requirements.in (line 6))
protobuf!=3.20.0,!=3.20.1,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 (from googleapis-common-protos[grpc]==1.57.0->grpc-google-iam-v1==0.12.4->google-cloud-secret-manager==2.12.6->-r requirements.in (line 6))
protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 (from google-cloud-spanner==3.24.0->apache-beam[gcp]==2.43.0->-r requirements.in (line 5))
protobuf<4.0.0dev (from google-cloud-vision==1.0.2->apache-beam[gcp]==2.43.0->-r requirements.in (line 5))
protobuf>=4.21.6 (from grpcio-status==1.51.1->google-cloud-pubsub==2.13.11->apache-beam[gcp]==2.43.0->-r requirements.in (line 5))
protobuf!=3.20.0,!=3.20.1,!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.19.5 (from google-cloud-pubsub==2.13.11->apache-beam[gcp]==2.43.0->-r requirements.in (line 5))
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1341093349
We prefer not to add indirect dependencies, let alone pin versions thereof. Can this be worked around by preinstalling protobuf v3, or explicitly adding it in the requirements.in ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1341079843
It seems like a bug in dependency resolution in piptools. But it may be also time to try to upgrade our protobuf depdendency. Opened https://github.com/apache/beam/issues/24569
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] tvalentyn commented on issue #23982: [Bug]: Could not find a version that matches protobuf
Posted by GitBox <gi...@apache.org>.
tvalentyn commented on issue #23982:
URL: https://github.com/apache/beam/issues/23982#issuecomment-1341202146
@AnandInguva do we still need this mac-os only restriction? https://github.com/apache/beam/blob/da100f967a6fbc86a2cc4ae6c9fe69180c70e74f/sdks/python/setup.py#L189
IIRC we added b/c MacOS github action tests were failing. but if that not an issue anymore, it would be good to relax that.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org