You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@heron.apache.org by GitBox <gi...@apache.org> on 2020/12/08 15:39:19 UTC
[GitHub] [incubator-heron] Code0x58 opened a new pull request #3646: Upgrade WORKSPACE python packages for python3.9 support
Code0x58 opened a new pull request #3646:
URL: https://github.com/apache/incubator-heron/pull/3646
This upgrades a few python packages. One benefit is that infrastructure/tests work on python3.9 (due to upgrading pex to a version that says it can be installed in python <=3.9) so will fix the issue seen in #3638
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis commented on pull request #3646: Upgrade WORKSPACE python packages for python3.9 support
Posted by GitBox <gi...@apache.org>.
nicknezis commented on pull request #3646:
URL: https://github.com/apache/incubator-heron/pull/3646#issuecomment-741863899
Looks good. What else is needed to remove from draft status?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis commented on pull request #3646: Upgrade WORKSPACE python packages for python3.9 support
Posted by GitBox <gi...@apache.org>.
nicknezis commented on pull request #3646:
URL: https://github.com/apache/incubator-heron/pull/3646#issuecomment-1019661423
#3642 Code was integrated into this PR.
Tracker Behavior changes:
- replace `/` → `/topologies` redirect with (OpenAPI on ReDoc) documentation
- deprecate `executiontime` and always return 0.0 for it in the response envelope - this could probably be put back in, but I don't think it's worth it
- inbound requests aren't terminated if they take over 120s. At the moment, every outbound request has a 5s limit (due to httpx default) so I wouldn't expect the timeout to be needed (even in metrics queries which may transform data). A replacement timeout could be added in if necessary.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@heron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] surahman commented on a change in pull request #3646: Upgrade WORKSPACE python packages for python3.9 support
Posted by GitBox <gi...@apache.org>.
surahman commented on a change in pull request #3646:
URL: https://github.com/apache/incubator-heron/pull/3646#discussion_r786982723
##########
File path: .travis.yml
##########
@@ -29,7 +29,7 @@ addons:
- libgoogle-perftools-dev
env:
- - BAZEL_VERSION=4.1.0 ENABLE_HEAPCHECK=1
+ - BAZEL_VERSION=4.1.0 ENABLE_HEAPCHECK=1 LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 LANGUAGE=en_US.UTF-8
Review comment:
```suggestion
- BAZEL_VERSION=4.1.0 ENABLE_HEAPCHECK=1
```
Should resolve error:
```bash
UnicodeEncodeError: 'ascii' codec can't encode characters in position 73-75: ordinal not in range(128)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@heron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] joshfischer1108 commented on pull request #3646: Upgrade WORKSPACE python packages for python3.9 support
Posted by GitBox <gi...@apache.org>.
joshfischer1108 commented on pull request #3646:
URL: https://github.com/apache/incubator-heron/pull/3646#issuecomment-772937137
@Code0x58 What do we need to do to pick up work on this branch? Anything I can help with?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] surahman commented on a change in pull request #3646: Upgrade WORKSPACE python packages for python3.9 support
Posted by GitBox <gi...@apache.org>.
surahman commented on a change in pull request #3646:
URL: https://github.com/apache/incubator-heron/pull/3646#discussion_r787181978
##########
File path: .travis.yml
##########
@@ -29,7 +29,7 @@ addons:
- libgoogle-perftools-dev
env:
- - BAZEL_VERSION=4.1.0 ENABLE_HEAPCHECK=1
+ - BAZEL_VERSION=4.1.0 ENABLE_HEAPCHECK=1 LC_ALL=en_US.UTF-8 LANG=en_US.UTF-8 LANGUAGE=en_US.UTF-8
Review comment:
Reverted. The build container is still running Python 3.6:
```bash
$ which python
/opt/pyenv/shims/python
The command "which python" exited with 0.
$ python -V
Python 2.7.17
The command "python -V" exited with 0.
$ which python3
/opt/pyenv/shims/python3
The command "which python3" exited with 0.
$ python3 -V
Python 3.6.9
The command "python3 -V" exited with 0.
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@heron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] Code0x58 commented on pull request #3646: Upgrade WORKSPACE python packages for python3.9 support
Posted by GitBox <gi...@apache.org>.
Code0x58 commented on pull request #3646:
URL: https://github.com/apache/incubator-heron/pull/3646#issuecomment-772969808
I think with these changes the python version in Travis blows up, or image (tested via `./scripts/release/build-images --all`) having versions that blow up during the build.
I can't recall what specifically required a change from 3.6 to something higher - probably just the newer libraries that work with 3.9.
One path forward is to upgrade all the python versions installed in the container images+CI (Travis + Ubuntu vagrant manifests) to some minimum version - you could use pyenv in the Vagrant machine to find what the lowest working version is. This would be a bit of a pain due to the multiple runtimes required to use Heron, so can't just use pre-existing base images (I think the JRE may be the first class citizen for that in the images). It's not a particularly appealing change, especially considering it may force python tennants to try upgrading their code+dependencies, but it's probably a reasonable thing to do without reconsidering the whole architecture or trying stick with Python 3.6-3.8 only.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nwangtw commented on pull request #3646: Upgrade WORKSPACE python packages for python3.9 support
Posted by GitBox <gi...@apache.org>.
nwangtw commented on pull request #3646:
URL: https://github.com/apache/incubator-heron/pull/3646#issuecomment-772971299
I am wondering which module uses more libraries? It might be ideal to
minimize the use of extra libraries considering the compatibility issues
with Python.
On Wed, Feb 3, 2021 at 6:08 PM Oliver Bristow <no...@github.com>
wrote:
> I think with these changes the python version in Travis blows up, or image
> (tested via ./scripts/release/build-images --all) having versions that
> blow up during the build.
>
> I can't recall what specifically required a change from 3.6 to something
> higher - probably just the newer libraries that work with 3.9.
>
> One path forward is to upgrade all the python versions installed in the
> container images+CI (Travis + Ubuntu vagrant manifests) to some minimum
> version - you could use pyenv in the Vagrant machine to find what the
> lowest working version is. This would be a bit of a pain due to the
> multiple runtimes required to use Heron, so can't just use pre-existing
> base images (I think the JRE may be the first class citizen for that in the
> images). It's not a particularly appealing change, especially considering
> it may force python tennants to try upgrading their code+dependencies, but
> it's probably a reasonable thing to do without reconsidering the whole
> architecture or trying stick with Python 3.6-3.8 only.
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/apache/incubator-heron/pull/3646#issuecomment-772969808>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/ABQPBH65RSR76W5H75LRT6DS5H6RTANCNFSM4USEOSQQ>
> .
>
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] nicknezis merged pull request #3646: Upgrade WORKSPACE python packages for python3.9 support
Posted by GitBox <gi...@apache.org>.
nicknezis merged pull request #3646:
URL: https://github.com/apache/incubator-heron/pull/3646
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@heron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org
[GitHub] [incubator-heron] surahman commented on pull request #3646: Upgrade WORKSPACE python packages for python3.9 support
Posted by GitBox <gi...@apache.org>.
surahman commented on pull request #3646:
URL: https://github.com/apache/incubator-heron/pull/3646#issuecomment-1015896766
The build containers are still running older versions of Python:
```bash
$ which python
/opt/pyenv/shims/python
The command "which python" exited with 0.
$ python -V
Python 2.7.17
The command "python -V" exited with 0.
$ which python3
/opt/pyenv/shims/python3
The command "which python3" exited with 0.
$ python3 -V
Python 3.6.9
The command "python3 -V" exited with 0.
```
We will probably need to update the base Dockerfiles here: https://github.com/apache/incubator-heron/tree/master/docker/compile
https://github.com/apache/incubator-heron/blob/ee369ef7745df530c765e89d2e8a2eefc8d8b742/docker/compile/Dockerfile.ubuntu18.04#L23-L40
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscribe@heron.apache.org
For queries about this service, please contact Infrastructure at:
users@infra.apache.org