You are viewing a plain text version of this content. The canonical link for it is here.
Posted to github@beam.apache.org by "AnandInguva (via GitHub)" <gi...@apache.org> on 2023/03/27 01:38:57 UTC
[GitHub] [beam] AnandInguva opened a new issue, #25985: [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11
AnandInguva opened a new issue, #25985:
URL: https://github.com/apache/beam/issues/25985
### What needs to happen?
### What needs to happen?
Currently, the installation of `python-snappy` from sdist fails. the wheels for python 3.11 is not yet available. Beam doesn't directly depend on `python-snappy` but we would eventually want to include it in the python 3.11 container.
Tracking issue to make sure Beam includes `python-snappy` in the Python 3.11 containers.
python-snappy issue: https://github.com/andrix/python-snappy/issues/124
### Issue Priority
Priority: 2 (default / most normal work should be filed as P2)
### Issue Components
- [X] Component: Python SDK
- [ ] Component: Java SDK
- [ ] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11 [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #25985:
URL: https://github.com/apache/beam/issues/25985#issuecomment-1783365761
Bumping this again since there's no new release. From https://github.com/andrix/python-snappy/issues/124 it looks like this may not actually ever happen. @AnandInguva any reason we need this in the container or can we just close this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11 [beam]
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm closed issue #25985: [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11
URL: https://github.com/apache/beam/issues/25985
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11 [beam]
Posted by "jrmccluskey (via GitHub)" <gi...@apache.org>.
jrmccluskey commented on issue #25985:
URL: https://github.com/apache/beam/issues/25985#issuecomment-1852257270
@tvalentyn do you anticipate finishing this task before the branch cut?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11 [beam]
Posted by "gliptak (via GitHub)" <gi...@apache.org>.
gliptak commented on issue #25985:
URL: https://github.com/apache/beam/issues/25985#issuecomment-2023353046
`python-snappy` released support for `Python 3.11`
https://github.com/intake/python-snappy/issues/124
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on issue #25985: [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on issue #25985:
URL: https://github.com/apache/beam/issues/25985#issuecomment-1662862437
No new releases from python-snappy to support it. Moving to 2.51.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] AnandInguva commented on issue #25985: [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11
Posted by "AnandInguva (via GitHub)" <gi...@apache.org>.
AnandInguva commented on issue #25985:
URL: https://github.com/apache/beam/issues/25985#issuecomment-1551583135
3.11 wheels are not available for this yet. We can change the milestone on this.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11 [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #25985:
URL: https://github.com/apache/beam/issues/25985#issuecomment-1839713483
Actually snappy is installable on Python 3.11 as long as one installs snappy linux headers:
```sudo apt install libsnappy-dev```
Dockerfile already includes this dep, and snappy is installable in Beam Python 3.11 docker containers. However requirement generation fails to process this manual dependency, since on most beam committers machines (and maybe on GHA workers?) libsnappy-dev is not installed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11 [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #25985:
URL: https://github.com/apache/beam/issues/25985#issuecomment-1839717969
> However requirement generation fails to process this manual dependency, since on most beam committers machines (and maybe on GHA workers?) libsnappy-dev is not installed.
One workaround for issues like this (and also to enable macbook users), would be to generate requirements using Beam docker container dockerfile, instead of a virtual environment on a committer's local machine.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
Re: [I] [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11 [beam]
Posted by "tvalentyn (via GitHub)" <gi...@apache.org>.
tvalentyn commented on issue #25985:
URL: https://github.com/apache/beam/issues/25985#issuecomment-1839715854
snappy is optional ; i think it might be used by some fastavro users if they use snappy coder, and it optimizies TFRecordIO, which falls back on less efficient implementation if snappy is not provided. Given that snappy is no longer maintained, there might be more efficient alternatives (zstd?)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] smeet07 commented on issue #25985: [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11
Posted by "smeet07 (via GitHub)" <gi...@apache.org>.
smeet07 commented on issue #25985:
URL: https://github.com/apache/beam/issues/25985#issuecomment-1519118859
.take-issue
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] damccorm commented on issue #25985: [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11
Posted by "damccorm (via GitHub)" <gi...@apache.org>.
damccorm commented on issue #25985:
URL: https://github.com/apache/beam/issues/25985#issuecomment-1727785316
No new release, bumping it again.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] lostluck commented on issue #25985: [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11
Posted by "lostluck (via GitHub)" <gi...@apache.org>.
lostluck commented on issue #25985:
URL: https://github.com/apache/beam/issues/25985#issuecomment-1662852890
2.50 release manager here.
This issue is currently tagged for the 2.50.0 release, which cuts in a week on August 9th.
Please complete work and get it into the main branch in that time, or move this issue to the 2.51 Milestone: https://github.com/apache/beam/milestone/15
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] riteshghorse commented on issue #25985: [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11
Posted by "riteshghorse (via GitHub)" <gi...@apache.org>.
riteshghorse commented on issue #25985:
URL: https://github.com/apache/beam/issues/25985#issuecomment-1551580182
Are we on track for these for 2.48? @AnandInguva
I'll be cutting the branch today
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [beam] Abacn commented on issue #25985: [Task]: Include python-snappy in Python 3.11 containers once python-snappy supports Python 3.11
Posted by "Abacn (via GitHub)" <gi...@apache.org>.
Abacn commented on issue #25985:
URL: https://github.com/apache/beam/issues/25985#issuecomment-1609971986
no new python-snappy release yet, move onto 2.50.0
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscribe@beam.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org