You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by "Łukasz Gajowy (JIRA)" <ji...@apache.org> on 2018/05/08 13:58:00 UTC
[jira] [Updated] (BEAM-4256) Gradle setupVirtualenv task fails on
Jenkins
[ https://issues.apache.org/jira/browse/BEAM-4256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Łukasz Gajowy updated BEAM-4256:
--------------------------------
Description:
AFAIK the setupVirtualenv task should run "pip install" command in the virtual environment. This works on my local machine, yet on Jenkins, when we run:
{code:java}
gradlew clean install -xcheck --stacktrace{code}
from within PerfkitBenchmarker tool, we get the following stacktrace:
{code:java}
> Task :beam-sdks-python:setupVirtualenv
Collecting tox==3.0.0
Using cached https://files.pythonhosted.org/packages/e6/41/4dcfd713282bf3213b0384320fa8841e4db032ddcb80bc08a540159d42a8/tox-3.0.0-py2.py3-none-any.whl
Collecting grpcio-tools==1.3.5
Using cached https://files.pythonhosted.org/packages/05/f6/0296e29b1bac6f85d2a8556d48adf825307f73109a3c2c17fb734292db0a/grpcio_tools-1.3.5-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pluggy<1.0,>=0.3.0 (from tox==3.0.0)
Using cached https://files.pythonhosted.org/packages/82/05/43e3947125a2137cba4746135c75934ceed1863f27e050fc560052104a71/pluggy-0.6.0-py2-none-any.whl
Requirement not upgraded as not directly required: six in /usr/local/lib/python2.7/dist-packages (from tox==3.0.0) (1.11.0)
Requirement not upgraded as not directly required: virtualenv>=1.11.2 in /usr/lib/python2.7/dist-packages (from tox==3.0.0) (15.0.1)
Collecting py>=1.4.17 (from tox==3.0.0)
Using cached https://files.pythonhosted.org/packages/67/a5/f77982214dd4c8fd104b066f249adea2c49e25e8703d284382eb5e9ab35a/py-1.5.3-py2.py3-none-any.whl
Collecting grpcio>=1.3.5 (from grpcio-tools==1.3.5)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting protobuf>=3.2.0 (from grpcio-tools==1.3.5)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Requirement not upgraded as not directly required: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (1.1.6)
Collecting futures>=2.2.0 (from grpcio>=1.3.5->grpcio-tools==1.3.5)
Using cached https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl
Requirement not upgraded as not directly required: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf>=3.2.0->grpcio-tools==1.3.5) (39.0.1)
Installing collected packages: pluggy, py, tox, protobuf, futures, grpcio, grpcio-tools
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/pluggy-0.6.0.dist-info'
Consider using the `--user` option or check the permissions.
> Task :beam-model-job-management:shadowJar
> Task :beam-sdks-python:setupVirtualenv FAILED
> Task :beam-model-job-management:jar
> Task :beam-model-fn-execution:shadowJar
> Task :beam-sdks-java-core:shadowJar
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.7/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 44s
133 actionable tasks: 126 executed, 5 from cache, 2 up-to-date{code}
It looks as if the virtualenv is not used while installing the needed dependencies (why is it trying to install pluggy in /usr/local/lib?). Therefore it throws "PermissionDenied" exception.
This affects all Performance tests on Beam's Jenkins. Currently none of the Jenkins jobs can build the whole repository code before the benchmark is started.
was:
AFAIK the setupVirtualenv task should run "pip install" command in the virtual environment. This works on my local machine, yet on Jenkins, when we run:
gradlew clean install -xcheck --stacktrace
from within PerfkitBenchmarker tool, we get the following stacktrace:
{code:java}
> Task :beam-sdks-python:setupVirtualenv
Collecting tox==3.0.0
Using cached https://files.pythonhosted.org/packages/e6/41/4dcfd713282bf3213b0384320fa8841e4db032ddcb80bc08a540159d42a8/tox-3.0.0-py2.py3-none-any.whl
Collecting grpcio-tools==1.3.5
Using cached https://files.pythonhosted.org/packages/05/f6/0296e29b1bac6f85d2a8556d48adf825307f73109a3c2c17fb734292db0a/grpcio_tools-1.3.5-cp27-cp27mu-manylinux1_x86_64.whl
Collecting pluggy<1.0,>=0.3.0 (from tox==3.0.0)
Using cached https://files.pythonhosted.org/packages/82/05/43e3947125a2137cba4746135c75934ceed1863f27e050fc560052104a71/pluggy-0.6.0-py2-none-any.whl
Requirement not upgraded as not directly required: six in /usr/local/lib/python2.7/dist-packages (from tox==3.0.0) (1.11.0)
Requirement not upgraded as not directly required: virtualenv>=1.11.2 in /usr/lib/python2.7/dist-packages (from tox==3.0.0) (15.0.1)
Collecting py>=1.4.17 (from tox==3.0.0)
Using cached https://files.pythonhosted.org/packages/67/a5/f77982214dd4c8fd104b066f249adea2c49e25e8703d284382eb5e9ab35a/py-1.5.3-py2.py3-none-any.whl
Collecting grpcio>=1.3.5 (from grpcio-tools==1.3.5)
Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
Collecting protobuf>=3.2.0 (from grpcio-tools==1.3.5)
Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
Requirement not upgraded as not directly required: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (1.1.6)
Collecting futures>=2.2.0 (from grpcio>=1.3.5->grpcio-tools==1.3.5)
Using cached https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl
Requirement not upgraded as not directly required: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf>=3.2.0->grpcio-tools==1.3.5) (39.0.1)
Installing collected packages: pluggy, py, tox, protobuf, futures, grpcio, grpcio-tools
Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/pluggy-0.6.0.dist-info'
Consider using the `--user` option or check the permissions.
> Task :beam-model-job-management:shadowJar
> Task :beam-sdks-python:setupVirtualenv FAILED
> Task :beam-model-job-management:jar
> Task :beam-model-fn-execution:shadowJar
> Task :beam-sdks-java-core:shadowJar
Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
See https://docs.gradle.org/4.7/userguide/command_line_interface.html#sec:command_line_warnings
BUILD FAILED in 44s
133 actionable tasks: 126 executed, 5 from cache, 2 up-to-date{code}
It looks as if the virtualenv is not used while installing the needed dependencies (why is it trying to install pluggy in /usr/local/lib?). Therefore it throws "PermissionDenied" exception.
This affects all Performance tests on Beam's Jenkins. Currently none of the Jenkins jobs can build the whole repository code before the benchmark is started.
> Gradle setupVirtualenv task fails on Jenkins
> --------------------------------------------
>
> Key: BEAM-4256
> URL: https://issues.apache.org/jira/browse/BEAM-4256
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Łukasz Gajowy
> Assignee: Ahmet Altay
> Priority: Critical
>
> AFAIK the setupVirtualenv task should run "pip install" command in the virtual environment. This works on my local machine, yet on Jenkins, when we run:
> {code:java}
> gradlew clean install -xcheck --stacktrace{code}
> from within PerfkitBenchmarker tool, we get the following stacktrace:
> {code:java}
> > Task :beam-sdks-python:setupVirtualenv
> Collecting tox==3.0.0
> Using cached https://files.pythonhosted.org/packages/e6/41/4dcfd713282bf3213b0384320fa8841e4db032ddcb80bc08a540159d42a8/tox-3.0.0-py2.py3-none-any.whl
> Collecting grpcio-tools==1.3.5
> Using cached https://files.pythonhosted.org/packages/05/f6/0296e29b1bac6f85d2a8556d48adf825307f73109a3c2c17fb734292db0a/grpcio_tools-1.3.5-cp27-cp27mu-manylinux1_x86_64.whl
> Collecting pluggy<1.0,>=0.3.0 (from tox==3.0.0)
> Using cached https://files.pythonhosted.org/packages/82/05/43e3947125a2137cba4746135c75934ceed1863f27e050fc560052104a71/pluggy-0.6.0-py2-none-any.whl
> Requirement not upgraded as not directly required: six in /usr/local/lib/python2.7/dist-packages (from tox==3.0.0) (1.11.0)
> Requirement not upgraded as not directly required: virtualenv>=1.11.2 in /usr/lib/python2.7/dist-packages (from tox==3.0.0) (15.0.1)
> Collecting py>=1.4.17 (from tox==3.0.0)
> Using cached https://files.pythonhosted.org/packages/67/a5/f77982214dd4c8fd104b066f249adea2c49e25e8703d284382eb5e9ab35a/py-1.5.3-py2.py3-none-any.whl
> Collecting grpcio>=1.3.5 (from grpcio-tools==1.3.5)
> Using cached https://files.pythonhosted.org/packages/0d/54/b647a6323be6526be27b2c90bb042769f1a7a6e59bd1a5f2eeb795bfece4/grpcio-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
> Collecting protobuf>=3.2.0 (from grpcio-tools==1.3.5)
> Using cached https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
> Requirement not upgraded as not directly required: enum34>=1.0.4 in /usr/local/lib/python2.7/dist-packages (from grpcio>=1.3.5->grpcio-tools==1.3.5) (1.1.6)
> Collecting futures>=2.2.0 (from grpcio>=1.3.5->grpcio-tools==1.3.5)
> Using cached https://files.pythonhosted.org/packages/2d/99/b2c4e9d5a30f6471e410a146232b4118e697fa3ffc06d6a65efde84debd0/futures-3.2.0-py2-none-any.whl
> Requirement not upgraded as not directly required: setuptools in /usr/local/lib/python2.7/dist-packages (from protobuf>=3.2.0->grpcio-tools==1.3.5) (39.0.1)
> Installing collected packages: pluggy, py, tox, protobuf, futures, grpcio, grpcio-tools
> Could not install packages due to an EnvironmentError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/pluggy-0.6.0.dist-info'
> Consider using the `--user` option or check the permissions.
> > Task :beam-model-job-management:shadowJar
> > Task :beam-sdks-python:setupVirtualenv FAILED
> > Task :beam-model-job-management:jar
> > Task :beam-model-fn-execution:shadowJar
> > Task :beam-sdks-java-core:shadowJar
> Deprecated Gradle features were used in this build, making it incompatible with Gradle 5.0.
> See https://docs.gradle.org/4.7/userguide/command_line_interface.html#sec:command_line_warnings
> BUILD FAILED in 44s
> 133 actionable tasks: 126 executed, 5 from cache, 2 up-to-date{code}
> It looks as if the virtualenv is not used while installing the needed dependencies (why is it trying to install pluggy in /usr/local/lib?). Therefore it throws "PermissionDenied" exception.
> This affects all Performance tests on Beam's Jenkins. Currently none of the Jenkins jobs can build the whole repository code before the benchmark is started.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)