You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "damccorm (via GitHub)" <gi...@apache.org> on 2023/02/27 22:27:45 UTC
[GitHub] [beam] damccorm opened a new issue, #25649: [Bug]: Building website changes fails with proto issues
damccorm opened a new issue, #25649:
URL: https://github.com/apache/beam/issues/25649
### What happened?
When running `./beam/release/src/main/scripts/build_release_candidate.sh ` per https://beam.apache.org/contribute/release-guide/#run-build_release_candidatesh-to-create-a-release-candidate, the script fails on website generation with the following error:
```
Processing ./target/.tox/.tmp/package/1/apache-beam-2.44.0.tar.gz
Preparing metadata (setup.py) ... error
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> [10 lines of output]
Traceback (most recent call last):
File "<string>", line 2, in <module>
File "<pip-setuptools-caller>", line 34, in <module>
File "/usr/local/google/home/klk/website_update_dir/python_doc/beam/sdks/python/target/.tox/py38-docs/tmp/pip-req-build-0ht9l7p_/setup.py", line 180, in <module>
generate_protos_first()
File "/usr/local/google/home/klk/website_update_dir/python_doc/beam/sdks/python/target/.tox/py38-docs/tmp/pip-req-build-0ht9l7p_/setup.py", line 150, in generate_protos_first
gen_protos.generate_proto_files()
File "/usr/local/google/home/klk/website_update_dir/python_doc/beam/sdks/python/target/.tox/py38-docs/tmp/pip-req-build-0ht9l7p_/gen_protos.py", line 476, in generate_proto_files
raise RuntimeError(error_msg)
RuntimeError: Not in apache git tree, unable to find proto definitions.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
```
Upthread, we also see
```
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
virtualenv 20.17.1 requires distlib<1,>=0.3.6, but you have distlib 0.3.1 which is incompatible.
```
This can be worked around by running `python gen_protos.py` as an intermediate step in the script before running `tox -e py38-docs`, which indicates this probably is a bug with gen_protos in other directories.
### Issue Priority
Priority: 3 (minor)
### Issue Components
- [X] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #25649: [Bug]: Building website changes fails with proto issues
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on issue #25649:
URL: https://github.com/apache/beam/issues/25649#issuecomment-1447232469
What tox version were you using?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #25649: [Bug]: Building website changes fails with proto issues
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on issue #25649:
URL: https://github.com/apache/beam/issues/25649#issuecomment-1447236927
I have encountered this issue before when I was trying to update tox to 4.x.x. Tox 4.x.x uses isolated builds by default and this could be the reason our generated pb_2 files and urn files are not getting copied to the sdist created by tox ( I might be totally wrong here) but the same code worked with tox 3.x in which our tox tasks didn't run on isolated builds.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #25649: [Bug]: Building website changes fails with proto issues
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on issue #25649:
URL: https://github.com/apache/beam/issues/25649#issuecomment-1447277386
Can we try to fix a tox version and investigate what is happening with tox 4.x and gen protos?
> I have encountered this issue before when I was trying to update tox to 4.x.x. Tox 4.x.x uses isolated builds by default and this could be the reason our generated pb_2 files and urn files are not getting copied to the sdist created by tox ( I might be totally wrong here) but the same code worked with tox 3.x in which our tox tasks didn't run on isolated builds.
>
> May be this could be a bug with gen_protos. If it is, we need to figure out what it is since it could be a blocker for #25123
I started to dig into it but this is where I stopped. I can pick it up after Python 3.11 work if its not resolved by then.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #25649: [Bug]: Building website changes fails with proto issues
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on issue #25649:
URL: https://github.com/apache/beam/issues/25649#issuecomment-1464283899
Okay. I figured out what was happening. `*_pb2.py` files are generated during sdist packaging. `setuptools` is not including them during sdist packaging. so we have to include `*_pb2.py` and `*_urns.py` files in `MANIFEST.in`. This solves one error and throws a different error then.
That second error depends on tox isolated builds and I am working on the solution now.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm closed issue #25649: [Bug]: Building website changes fails with proto issues
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #25649: [Bug]: Building website changes fails with proto issues
URL: https://github.com/apache/beam/issues/25649
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on issue #25649: [Bug]: Building website changes fails with proto issues
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #25649:
URL: https://github.com/apache/beam/issues/25649#issuecomment-1447241113
I'd have to check the exact version, but it was definitely tox 4.x.x (which is what now gets installed by default in our local dev setup https://github.com/apache/beam/blob/62e9a1239bb6f9ef2f1469d0c1af192697f8b10c/local-env-setup.sh#L126)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org