You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/02/19 00:25:12 UTC
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7457
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7457/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-11834] Enable arrays literals to have null values.
[Pablo Estrada] Enable SideInput metrics for DF worker. These have been launched on
[noreply] [BEAM-10708] Support SqlTransform in container (#13944)
------------------------------------------
[...truncated 897.67 KB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.734MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-000025
---> bd67d6dc49a9
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 87cddb0d2722
Removing intermediate container 87cddb0d2722
---> e0c86ad39bfa
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> bd02c060d982
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in e5c2cbd8db96
[91m2021/02/19 00:20:23 Installing setup packages ...
[0m[91m2021/02/19 00:20:23 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:42f87f1d-d2fc-4901-bba0-4839693d10ee']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 655.210s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-000025
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:08bcf142920797db7f2ae127990dcd90adebab9a0a2bba7aad149d5906492fef
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:08bcf142920797db7f2ae127990dcd90adebab9a0a2bba7aad149d5906492fef
Associated tags:
- 20210219-000025
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-000025
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-000025].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:08bcf142920797db7f2ae127990dcd90adebab9a0a2bba7aad149d5906492fef].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Py_ValCont #7493
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7493/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Py_ValCont - Build # 7492 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7492 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7492/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7491 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7491 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7491/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7490 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7490 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7490/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7489 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7489 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7489/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7488 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7488 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7488/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7487 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7487 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7487/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7486 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7486 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7486/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7485 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7485 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7485/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7484 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7484 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7484/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7483 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7483 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7483/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7482 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7482 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7482/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7481 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7481 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7481/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7480 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7480 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7480/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7479 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7479 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7479/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7478 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7478 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7478/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7477 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7477 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7477/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7476 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7476 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7476/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7475 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7475 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7475/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7474 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7474 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7474/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7473 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7473 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7473/ to view the results.
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7472
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7472/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-180021
---> 6bcdf607ab3b
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in abbb7c1ef51c
Removing intermediate container abbb7c1ef51c
---> a8417693dfcc
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> f39f2154e891
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in c9e2a1871e83
[91m2021/02/22 18:21:39 Installing setup packages ...
[0m[91m2021/02/22 18:21:39 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:4eb8948c-6bc1-4bc8-9811-df971cd2cd15']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 623.196s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-180021
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:79f6db7ce83b98aea42ba4ebd1e50ed7c6316c07a83a9613f50319c9e24163a2
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:79f6db7ce83b98aea42ba4ebd1e50ed7c6316c07a83a9613f50319c9e24163a2
Associated tags:
- 20210222-180021
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-180021
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-180021].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:79f6db7ce83b98aea42ba4ebd1e50ed7c6316c07a83a9613f50319c9e24163a2].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7471
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7471/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-120023
---> e14b37c6178c
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 55b32cc73c85
Removing intermediate container 55b32cc73c85
---> f396044f9519
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> c2861246a091
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in d6d7193bcbb9
[91m2021/02/22 12:20:59 Installing setup packages ...
[0m[91m2021/02/22 12:20:59 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:e89fda4b-c735-498d-81a2-f8e2d554aab6']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 582.961s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-120023
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:cff33ff8412534597a0e9740c717052043d23904dc099a4aa08e89ca6a496f10
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:cff33ff8412534597a0e9740c717052043d23904dc099a4aa08e89ca6a496f10
Associated tags:
- 20210222-120023
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-120023
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-120023].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:cff33ff8412534597a0e9740c717052043d23904dc099a4aa08e89ca6a496f10].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7470
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7470/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-060023
---> d587fa066690
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in bf06c0970fa8
Removing intermediate container bf06c0970fa8
---> 0dd4ab684681
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 188ac27614b1
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in bab9d40b857f
[91m2021/02/22 06:20:06 Installing setup packages ...
[0m[91m2021/02/22 06:20:06 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:e01f59f0-1769-426a-9fde-49e2773028a5']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 588.353s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-060023
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f6eb69d7a40ff31b5ba423146d6e8d6351bb03a4a7b9de3fad843cedde643af3
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f6eb69d7a40ff31b5ba423146d6e8d6351bb03a4a7b9de3fad843cedde643af3
Associated tags:
- 20210222-060023
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-060023
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-060023].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f6eb69d7a40ff31b5ba423146d6e8d6351bb03a4a7b9de3fad843cedde643af3].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7469
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7469/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-000022
---> 327b22cce35d
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 168623e98ba5
Removing intermediate container 168623e98ba5
---> b8418d4e7320
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 493ef9473246
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in eb789ede8acc
[91m2021/02/22 00:20:18 Installing setup packages ...
[0m[91m2021/02/22 00:20:18 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:9d67ec8f-beab-4216-b137-b05f0b4ceeec']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 594.780s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-000022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:c9e4c409a6733054fbf19c3e65d813510d4e9e1d16a5fc225299e151121fd0ea
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:c9e4c409a6733054fbf19c3e65d813510d4e9e1d16a5fc225299e151121fd0ea
Associated tags:
- 20210222-000022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-000022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-000022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:c9e4c409a6733054fbf19c3e65d813510d4e9e1d16a5fc225299e151121fd0ea].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7468
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7468/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-180021
---> e96bd71e0402
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in da84510aca97
Removing intermediate container da84510aca97
---> a1f3ad98e3f6
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 1624e9ba5d2d
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in ee5cdac2c346
[91m2021/02/21 18:20:14 Installing setup packages ...
[0m[91m2021/02/21 18:20:14 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:61e0bfa5-d815-487e-97d9-c8b2e246d6d0']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 634.644s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-180021
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5a8248e6084b6564ed01369fa0fa6d1cc8ae4704fc87787567219b0515c5aeaf
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5a8248e6084b6564ed01369fa0fa6d1cc8ae4704fc87787567219b0515c5aeaf
Associated tags:
- 20210221-180021
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-180021
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-180021].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5a8248e6084b6564ed01369fa0fa6d1cc8ae4704fc87787567219b0515c5aeaf].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7467
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7467/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-120022
---> cdd414a29b54
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 17f5bf018653
Removing intermediate container 17f5bf018653
---> 5a8bee104db9
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 2eb98a6bbc0f
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in cd518c2a55aa
[91m2021/02/21 12:20:49 Installing setup packages ...
[0m[91m2021/02/21 12:20:49 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:8f329f1b-0deb-4ae1-b72c-407009d3dc1c']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 592.728s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-120022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5043e2d9c5a7ba1e3b6dfec50f08f1bcd17cef16acb9b2563541fbfb6539ddb1
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5043e2d9c5a7ba1e3b6dfec50f08f1bcd17cef16acb9b2563541fbfb6539ddb1
Associated tags:
- 20210221-120022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-120022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-120022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5043e2d9c5a7ba1e3b6dfec50f08f1bcd17cef16acb9b2563541fbfb6539ddb1].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7466
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7466/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-060023
---> c09cde5d8b97
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 4ac3f87f7c86
Removing intermediate container 4ac3f87f7c86
---> a00afb786f67
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 4a5d31dbcc71
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in d5d1e01d4748
[91m2021/02/21 06:20:11 Installing setup packages ...
[0m[91m2021/02/21 06:20:11 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:e089dad8-561a-449d-8ce7-7210a1edd37d']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 570.684s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-060023
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:be6ccf05acafe34d6fe61432e7d7c732f99c9f8ce45d2f58661938c21bea0781
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:be6ccf05acafe34d6fe61432e7d7c732f99c9f8ce45d2f58661938c21bea0781
Associated tags:
- 20210221-060023
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-060023
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-060023].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:be6ccf05acafe34d6fe61432e7d7c732f99c9f8ce45d2f58661938c21bea0781].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7465
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7465/display/redirect?page=changes>
Changes:
[noreply] [BEAM-7372] remove aliases created for python 2 (#14012)
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-000022
---> 3ef515a6e53c
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 5fecb341531a
Removing intermediate container 5fecb341531a
---> 8f719a156b71
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 03a64137fa0a
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 5743c82265e4
[91m2021/02/21 00:21:38 Installing setup packages ...
[0m[91m2021/02/21 00:21:38 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:8dfcf132-bd4c-4d41-9fed-3a04af72cb5d']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 672.349s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-000022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:7882d815e67726f9a24b223d9c9bf95910d0503ac0b4d43849858459596f5c1f
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:7882d815e67726f9a24b223d9c9bf95910d0503ac0b4d43849858459596f5c1f
Associated tags:
- 20210221-000022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-000022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-000022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:7882d815e67726f9a24b223d9c9bf95910d0503ac0b4d43849858459596f5c1f].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7464
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7464/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-180021
---> c7f12bf0a538
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 8490c9aeef26
Removing intermediate container 8490c9aeef26
---> 88138f1de33e
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 70cadba93c11
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 6528dccfd9cd
[91m2021/02/20 18:20:12 Installing setup packages ...
[0m[91m2021/02/20 18:20:12 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:f9bf1960-fc32-4ff1-9458-d7fc01e33afe']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 572.715s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-180021
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:336e82668d45fdd7a7567a8980bb860eea01ddfbdf05bb57b8d7f3a47ccc8d59
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:336e82668d45fdd7a7567a8980bb860eea01ddfbdf05bb57b8d7f3a47ccc8d59
Associated tags:
- 20210220-180021
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-180021
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-180021].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:336e82668d45fdd7a7567a8980bb860eea01ddfbdf05bb57b8d7f3a47ccc8d59].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7463
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7463/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-120021
---> 977a1629ed1b
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 6c668ad6a982
Removing intermediate container 6c668ad6a982
---> d682eb229677
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 1734116c3cf6
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 84e4f3d9430d
[91m2021/02/20 12:20:46 Installing setup packages ...
[0m[91m2021/02/20 12:20:46 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:0ffdc7ee-93da-4731-97b4-d79bd64848b2']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 620.991s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-120021
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:009373f8e174d1dd06bacdda9fa82f6ecd8f5e20a3dfb9d4673c55fcc1bdeda0
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:009373f8e174d1dd06bacdda9fa82f6ecd8f5e20a3dfb9d4673c55fcc1bdeda0
Associated tags:
- 20210220-120021
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-120021
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-120021].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:009373f8e174d1dd06bacdda9fa82f6ecd8f5e20a3dfb9d4673c55fcc1bdeda0].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7462
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7462/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Remove side input combine examples that don't work on most runners.
[Robert Bradshaw] [BEAM-11842] Add resource hints to model proto.
[noreply] Add null checks to FhirIO.Search for when the search results are empty.
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-060022
---> 8200c020b078
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in be33ed830dd9
Removing intermediate container be33ed830dd9
---> f58d5e4490da
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 3b1b1f6548a6
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in e02d1a8480c7
[91m2021/02/20 06:19:51 Installing setup packages ...
[0m[91m2021/02/20 06:19:51 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:5dcf87c8-a210-4070-ac7a-82e6579e90ec']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 637.954s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-060022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:bf35a081fa308f9a371db50d5a513a3fc8fda1b93efa3bc5cb7172e6ee3e22af
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:bf35a081fa308f9a371db50d5a513a3fc8fda1b93efa3bc5cb7172e6ee3e22af
Associated tags:
- 20210220-060022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-060022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-060022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:bf35a081fa308f9a371db50d5a513a3fc8fda1b93efa3bc5cb7172e6ee3e22af].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7461
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7461/display/redirect?page=changes>
Changes:
[randomstep] [BEAM-11120] bump Gradle License Report plugin to 1.16
[zyichi] [BEAM-11779] Fix python pipeline options for CrossLanguage tests
[noreply] [BEAM-11800] Support ARRAY_AGG fn for Zetasql dialect (#13483)
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-000038
---> 7b1787d12632
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in cbc40a45af5b
Removing intermediate container cbc40a45af5b
---> b15b0c77f845
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 91d9ecd093d5
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 086e28f12efc
[91m2021/02/20 00:31:44 Installing setup packages ...
[0m[91m2021/02/20 00:31:44 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:9e08d986-5eb2-4a9f-9cb7-765b7f3d5413']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 623.873s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-000038
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f885b4c32fce5bce3d7e8ee8454e9a0720bf04d9e339e3a4c30d88a4e5ad68f7
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f885b4c32fce5bce3d7e8ee8454e9a0720bf04d9e339e3a4c30d88a4e5ad68f7
Associated tags:
- 20210220-000038
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-000038
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-000038].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f885b4c32fce5bce3d7e8ee8454e9a0720bf04d9e339e3a4c30d88a4e5ad68f7].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7460
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7460/display/redirect?page=changes>
Changes:
[noreply] [BEAM-9547] Fill in some trivial compliance gaps (#13932)
[noreply] [BEAM-11589] Migrated vendor/bytebuddy build.gradle file to Kotlin
[noreply] Merge pull request #14019 from [BEAM-11791] Fixing FnApiRunner
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-180022
---> d905f581884f
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 338baf72216c
Removing intermediate container 338baf72216c
---> 72e19ea8592a
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 23deaa43fe4a
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 87df32d46a1f
[91m2021/02/19 18:21:27 Installing setup packages ...
[0m[91m2021/02/19 18:21:27 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:0156f93a-820f-474d-8afd-c4ab932c82b0']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 642.759s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-180022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:530e90a5da51bd1551cc513f1a2e4bdebe15c5ef779306118027bc60670742b6
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:530e90a5da51bd1551cc513f1a2e4bdebe15c5ef779306118027bc60670742b6
Associated tags:
- 20210219-180022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-180022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-180022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:530e90a5da51bd1551cc513f1a2e4bdebe15c5ef779306118027bc60670742b6].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7459
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7459/display/redirect?page=changes>
Changes:
[noreply] Adding performance improvements to ApproximateQuantiles. (#13175)
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.737MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-120026
---> 8ac58c6a2f93
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 67c5291302ef
Removing intermediate container 67c5291302ef
---> 14ecf5dd1743
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 8dd0d9ecad14
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 7668b4d5700f
[91m2021/02/19 12:20:31 Installing setup packages ...
[0m[91m2021/02/19 12:20:31 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:7977983a-4b9c-448b-abc4-a7db743bd86c']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 598.487s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-120026
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:10aee2449b193569fedf08f76721efdcc72eea0eb5adc766ef6abb7e8e6f89aa
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:10aee2449b193569fedf08f76721efdcc72eea0eb5adc766ef6abb7e8e6f89aa
Associated tags:
- 20210219-120026
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-120026
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-120026].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:10aee2449b193569fedf08f76721efdcc72eea0eb5adc766ef6abb7e8e6f89aa].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7458
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7458/display/redirect?page=changes>
Changes:
[dmytrokozhevin] [BEAM-11760] Cleanup interactive cache just once.
[dmytrokozhevin] [BEAM-11760] Cleanup interactive cache just once.
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.734MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-060022
---> 0e6aa86e66da
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in dd3e0bfd3804
Removing intermediate container dd3e0bfd3804
---> 57eb166c58a6
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 30d5ed998786
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in a54ce76a4b51
[91m2021/02/19 06:20:55 Installing setup packages ...
[0m[91m2021/02/19 06:20:55 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:1ce86ac6-5cc3-4ce9-8e8a-66cf0b7352da']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 632.005s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-060022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:53e727a7ddb480b6fbfd7325e9c5e5db5a7835292582e38516a84fffa9279efe
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:53e727a7ddb480b6fbfd7325e9c5e5db5a7835292582e38516a84fffa9279efe
Associated tags:
- 20210219-060022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-060022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-060022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:53e727a7ddb480b6fbfd7325e9c5e5db5a7835292582e38516a84fffa9279efe].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org