You are viewing a plain text version of this content. The canonical link for it is here.
Posted to commits@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2018/05/07 15:41:41 UTC
Build failed in Jenkins: beam_PostCommit_Python_Verify #4875
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/4875/display/redirect>
------------------------------------------
[...truncated 1.13 MB...]
copying apache_beam/runners/worker/opcounters_test.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operation_specs.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operations.pxd -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/operations.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_main.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_main_test.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sdk_worker_test.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sideinputs.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/sideinputs_test.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_fast.pyx -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_slow.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/statesampler_test.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/worker_id_interceptor.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/runners/worker/worker_id_interceptor_test.py -> apache-beam-2.5.0.dev0/apache_beam/runners/worker
copying apache_beam/testing/__init__.py -> apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/pipeline_verifiers.py -> apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/pipeline_verifiers_test.py -> apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/test_pipeline.py -> apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/test_pipeline_test.py -> apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/test_stream.py -> apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/test_stream_test.py -> apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/test_utils.py -> apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/test_utils_test.py -> apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/util.py -> apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/util_test.py -> apache-beam-2.5.0.dev0/apache_beam/testing
copying apache_beam/testing/data/standard_coders.yaml -> apache-beam-2.5.0.dev0/apache_beam/testing/data
copying apache_beam/testing/data/trigger_transcripts.yaml -> apache-beam-2.5.0.dev0/apache_beam/testing/data
copying apache_beam/tools/__init__.py -> apache-beam-2.5.0.dev0/apache_beam/tools
copying apache_beam/tools/distribution_counter_microbenchmark.py -> apache-beam-2.5.0.dev0/apache_beam/tools
copying apache_beam/tools/map_fn_microbenchmark.py -> apache-beam-2.5.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.5.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.5.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.5.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.5.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.5.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.5.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.5.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.5.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.5.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.5.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.5.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.5.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.5.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.5.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.5.0.dev0/apache_beam/utils
Writing apache-beam-2.5.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.5.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run integration tests on the Google Cloud Dataflow service
# and validate that jobs finish successfully.
echo ">>> RUNNING TEST DATAFLOW RUNNER it tests"
>>> RUNNING TEST DATAFLOW RUNNER it tests
python setup.py nosetests \
--attr IT \
--nocapture \
--processes=4 \
--process-timeout=1800 \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--staging_location=$GCS_LOCATION/staging-it \
--temp_location=$GCS_LOCATION/temp-it \
--output=$GCS_LOCATION/py-it-cloud/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--sleep_secs=20"
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/local/lib/python2.7/site-packages/setuptools/dist.py>:388: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
running nosetests
running egg_info
writing requirements to apache_beam.egg-info/requires.txt
writing apache_beam.egg-info/PKG-INFO
writing top-level names to apache_beam.egg-info/top_level.txt
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/io/gcp/gcsio.py>:176: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/io/gcp/gcsio.py>:176: DeprecationWarning: object() takes no parameters
super(GcsIO, cls).__new__(cls, storage_client))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:52: DeprecationWarning: options is deprecated since First stable release.. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:52: DeprecationWarning: options is deprecated since First stable release.. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:52: DeprecationWarning: options is deprecated since First stable release.. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py>:782: DeprecationWarning: options is deprecated since First stable release.. References to <pipeline>.options will not be supported
transform_node.inputs[0].pipeline.options.view_as(StandardOptions))
<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py>:52: DeprecationWarning: options is deprecated since First stable release.. References to <pipeline>.options will not be supported
print('Found: %s.' % self.build_console_url(pipeline.options))
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_streaming_wordcount_it (apache_beam.examples.streaming_wordcount_it_test.StreamingWordCountIT) ... ok
test_bigquery_tornadoes_it (apache_beam.examples.cookbook.bigquery_tornadoes_it_test.BigqueryTornadoesIT) ... ok
test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 4 tests in 530.956s
OK
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-07_08_25_21-11398601758088481729?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-07_08_25_22-7978394308196860132?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-07_08_25_31-3710325258700556432?project=apache-beam-testing.
Found: https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-05-07_08_25_22-16769624375283300161?project=apache-beam-testing.
echo ">>> RUNNING DIRECT RUNNER hdfs_integration_test"
>>> RUNNING DIRECT RUNNER hdfs_integration_test
apache_beam/io/hdfs_integration_test/hdfs_integration_test.sh
++ dirname apache_beam/io/hdfs_integration_test/hdfs_integration_test.sh
+ TEST_DIR=apache_beam/io/hdfs_integration_test
+ ROOT_DIR=apache_beam/io/hdfs_integration_test/../../../../..
+ CONTEXT_DIR=apache_beam/io/hdfs_integration_test/../../../../../build/hdfs_integration
+ rm -r apache_beam/io/hdfs_integration_test/../../../../../build/hdfs_integration
rm: cannot remove 'apache_beam/io/hdfs_integration_test/../../../../../build/hdfs_integration': No such file or directory
+ true
+ mkdir -p apache_beam/io/hdfs_integration_test/../../../../../build/hdfs_integration/sdks
+ cp apache_beam/io/hdfs_integration_test/docker-compose.yml apache_beam/io/hdfs_integration_test/Dockerfile apache_beam/io/hdfs_integration_test/hdfscli.cfg apache_beam/io/hdfs_integration_test/hdfs_integration_test.sh apache_beam/io/hdfs_integration_test/../../../../../build/hdfs_integration/
+ cp -r apache_beam/io/hdfs_integration_test/../../../../../sdks/python apache_beam/io/hdfs_integration_test/../../../../../build/hdfs_integration/sdks/
+ cp -r apache_beam/io/hdfs_integration_test/../../../../../model apache_beam/io/hdfs_integration_test/../../../../../build/hdfs_integration/
++ echo hdfs_IT-jenkins-beam_PostCommit_Python_Verify-4875
+ PROJECT_NAME=hdfs_IT-jenkins-beam_PostCommit_Python_Verify-4875
+ cd apache_beam/io/hdfs_integration_test/../../../../../build/hdfs_integration
+ docker network prune --force
Deleted Networks:
hdfs_it-jenkins-beam_postcommit_python_verify-4874_test_net
+ docker-compose -p hdfs_IT-jenkins-beam_PostCommit_Python_Verify-4875 build
namenode uses an image, skipping
datanode uses an image, skipping
Building test
Step 1/9 : FROM python:2
---> cb627aaf5f26
Step 2/9 : WORKDIR /app
---> Using cache
---> e686439ed3f3
Step 3/9 : ENV HDFSCLI_CONFIG /app/sdks/python/apache_beam/io/hdfs_integration_test/hdfscli.cfg
---> Using cache
---> f89c80605d19
Step 4/9 : RUN pip install --no-cache-dir holdup gsutil
---> Using cache
---> 27ba1766845e
Step 5/9 : RUN gsutil cp gs://dataflow-samples/shakespeare/kinglear.txt .
---> Using cache
---> 0f4cf8bb4b88
Step 6/9 : ADD sdks/python /app/sdks/python
---> 1fed82286086
Removing intermediate container dd3bf2601147
Step 7/9 : ADD model /app/model
---> 23f5b72023ce
Removing intermediate container 85fac9f7edf2
Step 8/9 : RUN cd sdks/python && python setup.py sdist && pip install --no-cache-dir $(ls dist/apache-beam-*.tar.gz | tail -n1)[gcp]
---> Running in 8635dde37e0c
[91m/usr/local/lib/python2.7/site-packages/setuptools/dist.py:388: UserWarning: Normalizing '2.5.0.dev' to '2.5.0.dev0'
normalized_version,
[0mrunning sdist
[91m/app/sdks/python/gen_protos.py:49: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
[0m[91mWARNING:root:Installing grpcio-tools into /app/sdks/python/.eggs/grpcio-wheels
[0mCollecting grpcio-tools<2,>=1.3.5
[91m Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f314ff0a6d0>, 'Connection to files.pythonhosted.org timed out. (connect timeout=15)')': /packages/8e/cb/f8334f96f8fa8340375928d30b781301b8f9d4dc2a46c3647f3267209594/grpcio_tools-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
[0m[91m Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f314fef4210>, 'Connection to files.pythonhosted.org timed out. (connect timeout=15)')': /packages/8e/cb/f8334f96f8fa8340375928d30b781301b8f9d4dc2a46c3647f3267209594/grpcio_tools-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
[0m[91m Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f314fef4850>, 'Connection to files.pythonhosted.org timed out. (connect timeout=15)')': /packages/8e/cb/f8334f96f8fa8340375928d30b781301b8f9d4dc2a46c3647f3267209594/grpcio_tools-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
[0m[91m Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f314fef4990>, 'Connection to files.pythonhosted.org timed out. (connect timeout=15)')': /packages/8e/cb/f8334f96f8fa8340375928d30b781301b8f9d4dc2a46c3647f3267209594/grpcio_tools-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
[0m[91m Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f314fef4a10>, 'Connection to files.pythonhosted.org timed out. (connect timeout=15)')': /packages/8e/cb/f8334f96f8fa8340375928d30b781301b8f9d4dc2a46c3647f3267209594/grpcio_tools-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl
[0m[91mCould not install packages due to an EnvironmentError: HTTPSConnectionPool(host='files.pythonhosted.org', port=443): Max retries exceeded with url: /packages/8e/cb/f8334f96f8fa8340375928d30b781301b8f9d4dc2a46c3647f3267209594/grpcio_tools-1.11.0-cp27-cp27mu-manylinux1_x86_64.whl (Caused by ConnectTimeoutError(<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f314fef4b10>, 'Connection to files.pythonhosted.org timed out. (connect timeout=15)'))
[0m[91mProcess Process-1:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/multiprocessing/process.py", line 267, in _bootstrap
[0m[91m self.run()
File "/usr/local/lib/python2.7/multiprocessing/process.py", line 114, in run
self._target(*self._args, **self._kwargs)
File "/app/sdks/python/gen_protos.py", line 141, in _install_grpcio_tools_and_generate_proto_files
'--upgrade', GRPC_TOOLS])
File "/usr/local/lib/python2.7/subprocess.py", line 186, in check_call
[0m[91m raise CalledProcessError(retcode, cmd)
[0m[91mCalledProcessError: Command '['/usr/local/bin/python', '-m', 'pip', 'install', '--target', '/app/sdks/python/.eggs/grpcio-wheels', '--build', '/app/sdks/python/.eggs/grpcio-wheels-build', '--upgrade', 'grpcio-tools>=1.3.5,<2']' returned non-zero exit status 1
[0m[91mTraceback (most recent call last):
File "setup.py", line 204, in <module>
'test': generate_protos_first(test),
File "/usr/local/lib/python2.7/site-packages/setuptools/__init__.py", line 129, in setup
[0m[91m return distutils.core.setup(**attrs)
File "/usr/local/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/usr/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
[0m[91m self.run_command(cmd)
File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
[0m[91m cmd_obj.run()
File "setup.py", line 140, in run
gen_protos.generate_proto_files()
File "/app/sdks/python/gen_protos.py", line 95, in generate_proto_files
raise ValueError("Proto generation failed (see log for details).")
[0m[91mValueError: [0m[91mProto generation failed (see log for details).
[0mService 'test' failed to build: The command '/bin/sh -c cd sdks/python && python setup.py sdist && pip install --no-cache-dir $(ls dist/apache-beam-*.tar.gz | tail -n1)[gcp]' returned a non-zero code: 1
Build step 'Execute shell' marked build as failure
Jenkins build is back to normal : beam_PostCommit_Python_Verify
#4878
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/4878/display/redirect>
Build failed in Jenkins: beam_PostCommit_Python_Verify #4877
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/4877/display/redirect>
------------------------------------------
[...truncated 1.07 MB...]
test_no_partial_writeouts (apache_beam.pipeline_test.DirectRunnerRetryTests) ... ok
test_retry_fork_graph (apache_beam.pipeline_test.DirectRunnerRetryTests) ... ok
test_element (apache_beam.pipeline_test.DoFnTest) ... ok
test_side_input_no_tag (apache_beam.pipeline_test.DoFnTest) ... ok
test_side_input_tagged (apache_beam.pipeline_test.DoFnTest) ... ok
test_timestamp_param (apache_beam.pipeline_test.DoFnTest) ... ok
test_timestamp_param_map (apache_beam.pipeline_test.DoFnTest) ... ok
test_window_param (apache_beam.pipeline_test.DoFnTest) ... ok
test_attribute_setting (apache_beam.pipeline_test.PipelineOptionsTest) ... ok
test_defaults (apache_beam.pipeline_test.PipelineOptionsTest) ... ok
test_dir (apache_beam.pipeline_test.PipelineOptionsTest) ... ok
test_flag_parsing (apache_beam.pipeline_test.PipelineOptionsTest) ... ok
test_keyword_parsing (apache_beam.pipeline_test.PipelineOptionsTest) ... ok
test_view_as (apache_beam.pipeline_test.PipelineOptionsTest) ... ok
test_aggregator_empty_input (apache_beam.pipeline_test.PipelineTest) ... ok
test_apply_custom_transform (apache_beam.pipeline_test.PipelineTest) ... ok
test_create (apache_beam.pipeline_test.PipelineTest) ... ok
test_create_singleton_pcollection (apache_beam.pipeline_test.PipelineTest) ... ok
test_fake_read (apache_beam.pipeline_test.PipelineTest) ... ok
test_flatmap_builtin (apache_beam.pipeline_test.PipelineTest) ... ok
test_memory_usage (apache_beam.pipeline_test.PipelineTest) ... ok
test_metrics_in_fake_source (apache_beam.pipeline_test.PipelineTest) ... ok
test_pipeline_as_context (apache_beam.pipeline_test.PipelineTest) ... ok
test_ptransform_override_type_hints (apache_beam.pipeline_test.PipelineTest) ... ok
test_ptransform_overrides (apache_beam.pipeline_test.PipelineTest) ... ok
test_reuse_cloned_custom_transform_instance (apache_beam.pipeline_test.PipelineTest) ... ok
test_reuse_custom_transform_instance (apache_beam.pipeline_test.PipelineTest) ... ok
test_transform_no_super_init (apache_beam.pipeline_test.PipelineTest) ... ok
test_visit_entire_graph (apache_beam.pipeline_test.PipelineTest) ... ok
test_parent_pointer (apache_beam.pipeline_test.RunnerApiTest) ... ok
test_assingleton_multi_element (apache_beam.pvalue_test.PValueTest) ... ok
test_pvalue_expected_arguments (apache_beam.pvalue_test.PValueTest) ... ok
test_file_checksum_matchcer_invalid_sleep_time (apache_beam.testing.pipeline_verifiers_test.PipelineVerifiersTest) ... ok
test_file_checksum_matcher_read_failed (apache_beam.testing.pipeline_verifiers_test.PipelineVerifiersTest) ... ok
test_file_checksum_matcher_service_error (apache_beam.testing.pipeline_verifiers_test.PipelineVerifiersTest) ... ok
test_file_checksum_matcher_sleep_before_verify (apache_beam.testing.pipeline_verifiers_test.PipelineVerifiersTest) ... ok
test_file_checksum_matcher_success (apache_beam.testing.pipeline_verifiers_test.PipelineVerifiersTest) ... ok
Test PipelineStateMatcher fails when using default expected state ... ok
Test PipelineStateMatcher successes when matches given state ... ok
Test PipelineStateMatcher successes when using default expected state ... ok
test_append_extra_options (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... ok
test_append_verifier_in_extra_opt (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... ok
test_create_test_pipeline_options (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... ok
test_empty_option_args_parsing (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... ok
test_get_option (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... ok
test_option_args_parsing (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... ok
test_skip_IT (apache_beam.testing.test_pipeline_test.TestPipelineTest) ... SKIP: IT is skipped because --test-pipeline-options is not specified
test_basic_execution (apache_beam.testing.test_stream_test.TestStreamTest) ... ok
test_basic_execution_batch_sideinputs (apache_beam.testing.test_stream_test.TestStreamTest) ... ok
test_basic_execution_batch_sideinputs_fixed_windows (apache_beam.testing.test_stream_test.TestStreamTest) ... ok
test_basic_execution_sideinputs (apache_beam.testing.test_stream_test.TestStreamTest) ... ok
test_basic_execution_sideinputs_fixed_windows (apache_beam.testing.test_stream_test.TestStreamTest) ... ok
test_basic_test_stream (apache_beam.testing.test_stream_test.TestStreamTest) ... ok
Advance TestClock to (X + delta) and see the pipeline does finish. ... ok
test_gbk_execution_after_watermark_trigger (apache_beam.testing.test_stream_test.TestStreamTest) ... ok
test_gbk_execution_no_triggers (apache_beam.testing.test_stream_test.TestStreamTest) ... ok
test_test_stream_errors (apache_beam.testing.test_stream_test.TestStreamTest) ... ok
test_cleanup_subscriptions (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok
test_cleanup_topics (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok
test_delete_files_fails_with_invalid_arg (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok
test_delete_files_fails_with_io_error (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok
test_delete_files_succeeds (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok
test_temp_dir_removes_files (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok
test_temp_file_field_correct (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok
test_wait_for_subscriptions_created_fails (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok
test_wait_for_subscriptions_created_succeeds (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok
test_wait_for_topics_created_fails (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok
test_wait_for_topics_created_succeeds (apache_beam.testing.test_utils_test.TestUtilsTest) ... ok
test_assert_that_fails (apache_beam.testing.util_test.UtilTest) ... ok
test_assert_that_fails_on_empty_expected (apache_beam.testing.util_test.UtilTest) ... ok
test_assert_that_fails_on_empty_input (apache_beam.testing.util_test.UtilTest) ... ok
test_assert_that_passes (apache_beam.testing.util_test.UtilTest) ... ok
test_assert_that_passes_empty_equal_to (apache_beam.testing.util_test.UtilTest) ... ok
test_assert_that_passes_empty_is_empty (apache_beam.testing.util_test.UtilTest) ... ok
test_windowed_value_assert_fail_unmatched_timestamp (apache_beam.testing.util_test.UtilTest) ... ok
test_windowed_value_assert_fail_unmatched_value (apache_beam.testing.util_test.UtilTest) ... ok
test_windowed_value_assert_fail_unmatched_window (apache_beam.testing.util_test.UtilTest) ... ok
test_windowed_value_passes (apache_beam.testing.util_test.UtilTest) ... ok
----------------------------------------------------------------------
XML: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 1571 tests in 176.043s
OK (SKIP=47)
py27-gcp runtests: commands[5] | <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/run_tox_cleanup.sh>
py27-lint create: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/target/.tox/py27-lint>
py27-lint installdeps: pycodestyle==2.3.1, pylint==1.7.2, future==0.16.0, isort==4.2.15, flake8==3.5.0
py27-lint inst: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/target/.tox/dist/apache-beam-2.5.0.dev0.zip>
py27-lint installed: apache-beam==2.5.0.dev0,astroid==1.6.3,avro==1.8.2,backports.functools-lru-cache==1.5,certifi==2018.4.16,chardet==3.0.4,configparser==3.5.0,crcmod==1.7,dill==0.2.6,docopt==0.6.2,enum34==1.1.6,flake8==3.5.0,funcsigs==1.0.2,future==0.16.0,futures==3.2.0,grpcio==1.11.0,hdfs==2.1.0,httplib2==0.9.2,idna==2.6,isort==4.2.15,lazy-object-proxy==1.3.1,mccabe==0.6.1,mock==2.0.0,nose==1.3.7,oauth2client==4.1.2,pbr==4.0.2,protobuf==3.5.2.post1,pyasn1==0.4.2,pyasn1-modules==0.2.1,pycodestyle==2.3.1,pyflakes==1.6.0,PyHamcrest==1.9.0,pylint==1.7.2,pytz==2018.4,PyVCF==0.6.8,PyYAML==3.12,requests==2.18.4,rsa==3.4.2,singledispatch==3.4.0.3,six==1.11.0,typing==3.6.4,urllib3==1.22,wrapt==1.10.11
py27-lint runtests: PYTHONHASHSEED='4234001943'
py27-lint runtests: commands[0] | python --version
Python 2.7.12
py27-lint runtests: commands[1] | pip --version
pip 10.0.1 from <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/target/.tox/py27-lint/local/lib/python2.7/site-packages/pip> (python 2.7)
py27-lint runtests: commands[2] | time <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/run_pylint.sh>
Skipping lint for generated files: bigquery_v2_client.py, bigquery_v2_messages.py, dataflow_v1b3_client.py, dataflow_v1b3_messages.py, storage_v1_client.py, storage_v1_messages.py, proto2_coder_test_messages_pb2.py, beam_artifact_api_pb2_grpc.py, beam_artifact_api_pb2.py, beam_fn_api_pb2_grpc.py, beam_fn_api_pb2.py, beam_job_api_pb2_grpc.py, beam_job_api_pb2.py, beam_provision_api_pb2_grpc.py, beam_provision_api_pb2.py, beam_runner_api_pb2_grpc.py, beam_runner_api_pb2.py, endpoints_pb2_grpc.py, endpoints_pb2.py, standard_window_fns_pb2_grpc.py, standard_window_fns_pb2.py
Running pylint for module apache_beam gen_protos.py setup.py test_config.py:
--------------------------------------------------------------------
Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)
Running pycodestyle for module apache_beam gen_protos.py setup.py test_config.py:
Running flake8 for module apache_beam gen_protos.py setup.py test_config.py:
0
Running isort for module apache_beam gen_protos.py setup.py test_config.py:
Skipped 72 files
Checking for files requiring stage 1 refactoring from futurize
No future changes needed
Checking unittest.main for module apache_beam gen_protos.py setup.py test_config.py:
224.96user 6.39system 0:36.74elapsed 629%CPU (0avgtext+0avgdata 259048maxresident)k
0inputs+176outputs (0major+810623minor)pagefaults 0swaps
py3-lint create: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/target/.tox/py3-lint>
py3-lint installdeps: pycodestyle==2.3.1, pylint==1.7.2, future==0.16.0, isort==4.2.15, flake8==3.5.0
py3-lint inst: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/target/.tox/dist/apache-beam-2.5.0.dev0.zip>
py3-lint installed: apache-beam==2.5.0.dev0,astroid==1.6.3,avro==1.8.2,certifi==2018.4.16,chardet==3.0.4,crcmod==1.7,dill==0.2.6,docopt==0.6.2,flake8==3.5.0,future==0.16.0,futures==3.1.1,grpcio==1.11.0,hdfs==2.1.0,httplib2==0.9.2,idna==2.6,isort==4.2.15,lazy-object-proxy==1.3.1,mccabe==0.6.1,mock==2.0.0,nose==1.3.7,oauth2client==4.1.2,pbr==4.0.2,protobuf==3.5.2.post1,pyasn1==0.4.2,pyasn1-modules==0.2.1,pycodestyle==2.3.1,pyflakes==1.6.0,PyHamcrest==1.9.0,pylint==1.7.2,pytz==2018.4,PyVCF==0.6.8,PyYAML==3.12,requests==2.18.4,rsa==3.4.2,six==1.11.0,typing==3.6.4,urllib3==1.22,wrapt==1.10.11
py3-lint runtests: PYTHONHASHSEED='4234001943'
py3-lint runtests: commands[0] | python --version
Python 3.5.2
py3-lint runtests: commands[1] | pip --version
pip 10.0.1 from <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/target/.tox/py3-lint/lib/python3.5/site-packages/pip> (python 3.5)
py3-lint runtests: commands[2] | time <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/run_mini_py3lint.sh>
Running flake8 for module apache_beam:
0
23.67user 0.49system 0:03.35elapsed 720%CPU (0avgtext+0avgdata 46832maxresident)k
0inputs+0outputs (0major+106817minor)pagefaults 0swaps
___________________________________ summary ____________________________________
cover: commands succeeded
docs: commands succeeded
py27: commands succeeded
py27-cython: commands succeeded
py27-gcp: commands succeeded
py27-lint: commands succeeded
py3-lint: commands succeeded
congratulations :)
# Virtualenv for the rest of the script to run setup & e2e tests
${LOCAL_PATH}/virtualenv sdks/python
New python executable in <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/sdks/python/bin/python>
Installing setuptools, pip, wheel...done.
. sdks/python/bin/activate
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly
deactivate () {
unset -f pydoc >/dev/null 2>&1
# reset old environment variables
# ! [ -z ${VAR+_} ] returns true if VAR is declared at all
if ! [ -z "${_OLD_VIRTUAL_PATH+_}" ] ; then
PATH="$_OLD_VIRTUAL_PATH"
export PATH
unset _OLD_VIRTUAL_PATH
fi
if ! [ -z "${_OLD_VIRTUAL_PYTHONHOME+_}" ] ; then
PYTHONHOME="$_OLD_VIRTUAL_PYTHONHOME"
export PYTHONHOME
unset _OLD_VIRTUAL_PYTHONHOME
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
if ! [ -z "${_OLD_VIRTUAL_PS1+_}" ] ; then
PS1="$_OLD_VIRTUAL_PS1"
export PS1
unset _OLD_VIRTUAL_PS1
fi
unset VIRTUAL_ENV
if [ ! "${1-}" = "nondestructive" ] ; then
# Self destruct!
unset -f deactivate
fi
}
# unset irrelevant variables
deactivate nondestructive
VIRTUAL_ENV="<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/sdks/python">
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/bin:$PATH"
export PATH
# unset PYTHONHOME if set
if ! [ -z "${PYTHONHOME+_}" ] ; then
_OLD_VIRTUAL_PYTHONHOME="$PYTHONHOME"
unset PYTHONHOME
fi
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT-}" ] ; then
_OLD_VIRTUAL_PS1="$PS1"
if [ "x" != x ] ; then
PS1="$PS1"
else
PS1="(`basename \"$VIRTUAL_ENV\"`) $PS1"
fi
export PS1
fi
basename "$VIRTUAL_ENV"
# Make sure to unalias pydoc if it's already there
alias pydoc 2>/dev/null >/dev/null && unalias pydoc
pydoc () {
python -m pydoc "$@"
}
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
cd sdks/python
pip install -e .[gcp,test]
Directory '.' is not installable. File 'setup.py' not found.
> Task :beam-sdks-python:postCommit FAILED
:beam-sdks-python:postCommit (Thread[Task worker for ':' Thread 9,5,main]) completed. Took 25 mins 35.287 secs.
FAILURE: Build failed with an exception.
* Where:
Build file '<https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/src/sdks/python/build.gradle'> line: 165
* What went wrong:
Execution failed for task ':beam-sdks-python:postCommit'.
> Process 'command 'sh'' finished with non-zero exit value 1
* Try:
Run with --stacktrace option to get the stack trace. Run with --debug option to get more log output. Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 25m 40s
1 actionable task: 1 executed
Publishing build scan...
https://gradle.com/s/i65ojphle5vl4
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Build failed in Jenkins: beam_PostCommit_Python_Verify #4876
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://builds.apache.org/job/beam_PostCommit_Python_Verify/4876/display/redirect>
------------------------------------------
GitHub pull request #5289 of commit 6db09912b536ba49d5f36e565d4de9bed636e0cb, no merge conflicts.
Setting status of 6db09912b536ba49d5f36e565d4de9bed636e0cb to PENDING with url https://builds.apache.org/job/beam_PostCommit_Python_Verify/4876/ and message: 'Build started sha1 is merged.'
Using context: Jenkins: Python SDK PostCommit Tests
[EnvInject] - Loading node environment variables.
Building remotely on beam11 (beam) in workspace <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/>
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
> git --version # timeout=10
> git fetch --tags --progress https://github.com/apache/beam.git +refs/heads/*:refs/remotes/origin/* +refs/pull/5289/*:refs/remotes/origin/pr/5289/*
> git rev-parse refs/remotes/origin/pr/5289/merge^{commit} # timeout=10
> git rev-parse refs/remotes/origin/origin/pr/5289/merge^{commit} # timeout=10
Checking out Revision 68a3cc8172f28aac9f1f6d039070f59050f46f3e (refs/remotes/origin/pr/5289/merge)
> git config core.sparsecheckout # timeout=10
> git checkout -f 68a3cc8172f28aac9f1f6d039070f59050f46f3e
Commit message: "Merge 6db09912b536ba49d5f36e565d4de9bed636e0cb into 0e1dcccf2fbc3737c72cd52fd3eb2018b586c85c"
First time build. Skipping changelog.
Cleaning workspace
> git rev-parse --verify HEAD # timeout=10
Resetting working tree
> git reset --hard # timeout=10
> git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the SCM step.
[EnvInject] - Injecting as environment variables the properties content
SPARK_LOCAL_IP=127.0.0.1
[EnvInject] - Variables injected successfully.
[Gradle] - Launching build.
FATAL: The Gradle wrapper has not been found in these directories: <https://builds.apache.org/job/beam_PostCommit_Python_Verify/ws/>
Build step 'Invoke Gradle script' marked build as failure