You are viewing a plain text version of this content. The canonical link for it is here.
Posted to issues@beam.apache.org by "Kamil Wasilewski (Jira)" <ji...@apache.org> on 2019/12/16 16:18:00 UTC
[jira] [Commented] (BEAM-8966) failure in
:sdks:python:test-suites:direct:py37:hdfsIntegrationTest
[ https://issues.apache.org/jira/browse/BEAM-8966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16997429#comment-16997429 ]
Kamil Wasilewski commented on BEAM-8966:
----------------------------------------
Similar error happens in [https://builds.apache.org/job/beam_BiqQueryIO_Write_Performance_Test_Python_Batch/ |https://builds.apache.org/job/beam_BiqQueryIO_Write_Performance_Test_Python_Batch/]and [https://builds.apache.org/job/beam_LoadTests_Python_37_ParDo_Dataflow_Batch_PR/]
No idea why not every Python tests are affected. Also, I'm not able to reproduce this error locally.
{code:java}
16:52:40 > Task :sdks:python:sdist FAILED
16:52:40 setup.py:232: UserWarning: You are using Apache Beam with Python 2. New releases of Apache Beam will soon support Python 3 only.
16:52:40 'You are using Apache Beam with Python 2. '
16:52:40 /home/jenkins/jenkins-slave/workspace/beam_BiqQueryIO_Write_Performance_Test_Python_Batch/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/setuptools/dist.py:476: UserWarning: Normalizing '2.19.0.dev' to '2.19.0.dev0'
16:52:40 normalized_version,
16:52:40 DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
16:52:40 Requirement already satisfied: mypy-protobuf==1.12 in /home/jenkins/jenkins-slave/workspace/beam_BiqQueryIO_Write_Performance_Test_Python_Batch/src/build/gradleenv/1922375555/lib/python2.7/site-packages (1.12)
16:52:40 beam_fn_api.proto: warning: Import google/protobuf/descriptor.proto but not used.
16:52:40 beam_fn_api.proto: warning: Import google/protobuf/wrappers.proto but not used.
16:52:40 protoc-gen-mypy: program not found or is not executable
16:52:40 --mypy_out: protoc-gen-mypy: Plugin failed with status code 1.
16:52:40 Traceback (most recent call last):
16:52:40 File "setup.py", line 295, in <module>
16:52:40 'mypy': generate_protos_first(mypy),
16:52:40 File "/home/jenkins/jenkins-slave/workspace/beam_BiqQueryIO_Write_Performance_Test_Python_Batch/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/setuptools/__init__.py", line 145, in setup
16:52:40 return distutils.core.setup(**attrs)
16:52:40 File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
16:52:40 dist.run_commands()
16:52:40 File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
16:52:40 self.run_command(cmd)
16:52:40 File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
16:52:40 cmd_obj.run()
16:52:40 File "/home/jenkins/jenkins-slave/workspace/beam_BiqQueryIO_Write_Performance_Test_Python_Batch/src/build/gradleenv/1922375555/local/lib/python2.7/site-packages/setuptools/command/sdist.py", line 44, in run
16:52:40 self.run_command('egg_info')
16:52:40 File "/usr/lib/python2.7/distutils/cmd.py", line 326, in run_command
16:52:40 self.distribution.run_command(command)
16:52:40 File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
16:52:40 cmd_obj.run()
16:52:40 File "setup.py", line 220, in run
16:52:40 gen_protos.generate_proto_files(log=log)
16:52:40 File "/home/jenkins/jenkins-slave/workspace/beam_BiqQueryIO_Write_Performance_Test_Python_Batch/src/sdks/python/gen_protos.py", line 144, in generate_proto_files
16:52:40 '%s' % ret_code)
16:52:40 RuntimeError: Protoc returned non-zero status (see logs for details): 1
{code}
> failure in :sdks:python:test-suites:direct:py37:hdfsIntegrationTest
> -------------------------------------------------------------------
>
> Key: BEAM-8966
> URL: https://issues.apache.org/jira/browse/BEAM-8966
> Project: Beam
> Issue Type: Bug
> Components: test-failures
> Reporter: Udi Meiri
> Assignee: Chad Dombrova
> Priority: Major
>
> I believe this is due to https://github.com/apache/beam/pull/9915
> {code}
> [0mCollecting mypy-protobuf==1.12
> Using cached https://files.pythonhosted.org/packages/b6/28/041dea47c93564bfc0ece050362894292ec4f173caa92fa82994a6d061d1/mypy_protobuf-1.12-py3-none-any.whl
> Installing collected packages: mypy-protobuf
> Successfully installed mypy-protobuf-1.12
> [91mbeam_fn_api.proto: warning: Import google/protobuf/descriptor.proto but not used.
> beam_fn_api.proto: warning: Import google/protobuf/wrappers.proto but not used.
> [0m[91mTraceback (most recent call last):
> File "/usr/local/bin/protoc-gen-mypy", line 13, in <module>
> import google.protobuf.descriptor_pb2 as d
> ModuleNotFoundError: No module named 'google'
> [0m[91m--mypy_out: protoc-gen-mypy: Plugin failed with status code 1.
> [0m[91mProcess Process-1:
> Traceback (most recent call last):
> File "/app/sdks/python/gen_protos.py", line 104, in generate_proto_files
> from grpc_tools import protoc
> ModuleNotFoundError: No module named 'grpc_tools'
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "/usr/local/lib/python3.7/multiprocessing/process.py", line 297, in _bootstrap
> self.run()
> File "/usr/local/lib/python3.7/multiprocessing/process.py", line 99, in run
> self._target(*self._args, **self._kwargs)
> File "/app/sdks/python/gen_protos.py", line 189, in _install_grpcio_tools_and_generate_proto_files
> generate_proto_files()
> File "/app/sdks/python/gen_protos.py", line 144, in generate_proto_files
> '%s' % ret_code)
> RuntimeError: Protoc returned non-zero status (see logs for details): 1
> [0m[91mTraceback (most recent call last):
> File "/app/sdks/python/gen_protos.py", line 104, in generate_proto_files
> from grpc_tools import protoc
> ModuleNotFoundError: No module named 'grpc_tools'
> During handling of the above exception, another exception occurred:
> Traceback (most recent call last):
> File "setup.py", line 295, in <module>
> 'mypy': generate_protos_first(mypy),
> File "/usr/local/lib/python3.7/site-packages/setuptools/__init__.py", line 145, in setup
> return distutils.core.setup(**attrs)
> File "/usr/local/lib/python3.7/distutils/core.py", line 148, in setup
> dist.run_commands()
> File "/usr/local/lib/python3.7/distutils/dist.py", line 966, in run_commands
> self.run_command(cmd)
> File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
> cmd_obj.run()
> File "/usr/local/lib/python3.7/site-packages/setuptools/command/sdist.py", line 44, in run
> self.run_command('egg_info')
> File "/usr/local/lib/python3.7/distutils/cmd.py", line 313, in run_command
> self.distribution.run_command(command)
> File "/usr/local/lib/python3.7/distutils/dist.py", line 985, in run_command
> cmd_obj.run()
> File "setup.py", line 220, in run
> gen_protos.generate_proto_files(log=log)
> File "/app/sdks/python/gen_protos.py", line 121, in generate_proto_files
> raise ValueError("Proto generation failed (see log for details).")
> ValueError: Proto generation failed (see log for details).
> [0mService 'test' failed to build: The command '/bin/sh -c cd sdks/python && python setup.py sdist && pip install --no-cache-dir $(ls dist/apache-beam-*.tar.gz | tail -n1)[gcp]' returned a non-zero code: 1
> {code}
> https://builds.apache.org/job/beam_PostCommit_Python37/1114/consoleText
--
This message was sent by Atlassian Jira
(v8.3.4#803005)