You are viewing a plain text version of this content. The canonical link for it is here.
Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/02/13 18:25:51 UTC
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7436
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7436/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210213-180023
---> ab51f73d54d6
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 69be75aba563
Removing intermediate container 69be75aba563
---> e34f1b173af3
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 214adce804d1
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 4bfaaf65347e
[91m2021/02/13 18:21:18 Installing setup packages ...
[0m[91m2021/02/13 18:21:18 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:31c1d503-0aca-427e-b635-49a9491ee5c0']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 607.728s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210213-180023
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:db56de3a2ce1e74a272441534486f522b317dbe7dda5084fd61a100c613279ee
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:db56de3a2ce1e74a272441534486f522b317dbe7dda5084fd61a100c613279ee
Associated tags:
- 20210213-180023
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210213-180023
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210213-180023].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:db56de3a2ce1e74a272441534486f522b317dbe7dda5084fd61a100c613279ee].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Jenkins build is back to normal : beam_PostCommit_Py_ValCont #7493
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7493/display/redirect?page=changes>
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
beam_PostCommit_Py_ValCont - Build # 7492 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7492 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7492/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7491 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7491 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7491/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7490 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7490 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7490/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7489 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7489 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7489/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7488 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7488 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7488/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7487 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7487 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7487/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7486 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7486 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7486/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7485 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7485 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7485/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7484 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7484 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7484/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7483 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7483 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7483/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7482 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7482 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7482/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7481 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7481 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7481/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7480 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7480 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7480/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7479 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7479 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7479/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7478 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7478 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7478/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7477 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7477 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7477/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7476 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7476 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7476/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7475 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7475 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7475/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7474 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7474 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7474/ to view the results.
beam_PostCommit_Py_ValCont - Build # 7473 - Aborted!
Posted by Apache Jenkins Server <je...@builds.apache.org>.
beam_PostCommit_Py_ValCont - Build # 7473 - Aborted:
Check console output at https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7473/ to view the results.
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7472
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7472/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-180021
---> 6bcdf607ab3b
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in abbb7c1ef51c
Removing intermediate container abbb7c1ef51c
---> a8417693dfcc
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> f39f2154e891
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in c9e2a1871e83
[91m2021/02/22 18:21:39 Installing setup packages ...
[0m[91m2021/02/22 18:21:39 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:4eb8948c-6bc1-4bc8-9811-df971cd2cd15']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 623.196s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-180021
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:79f6db7ce83b98aea42ba4ebd1e50ed7c6316c07a83a9613f50319c9e24163a2
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:79f6db7ce83b98aea42ba4ebd1e50ed7c6316c07a83a9613f50319c9e24163a2
Associated tags:
- 20210222-180021
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-180021
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-180021].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:79f6db7ce83b98aea42ba4ebd1e50ed7c6316c07a83a9613f50319c9e24163a2].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7471
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7471/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-120023
---> e14b37c6178c
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 55b32cc73c85
Removing intermediate container 55b32cc73c85
---> f396044f9519
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> c2861246a091
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in d6d7193bcbb9
[91m2021/02/22 12:20:59 Installing setup packages ...
[0m[91m2021/02/22 12:20:59 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:e89fda4b-c735-498d-81a2-f8e2d554aab6']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 582.961s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-120023
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:cff33ff8412534597a0e9740c717052043d23904dc099a4aa08e89ca6a496f10
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:cff33ff8412534597a0e9740c717052043d23904dc099a4aa08e89ca6a496f10
Associated tags:
- 20210222-120023
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-120023
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-120023].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:cff33ff8412534597a0e9740c717052043d23904dc099a4aa08e89ca6a496f10].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7470
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7470/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-060023
---> d587fa066690
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in bf06c0970fa8
Removing intermediate container bf06c0970fa8
---> 0dd4ab684681
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 188ac27614b1
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in bab9d40b857f
[91m2021/02/22 06:20:06 Installing setup packages ...
[0m[91m2021/02/22 06:20:06 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:e01f59f0-1769-426a-9fde-49e2773028a5']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 588.353s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-060023
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f6eb69d7a40ff31b5ba423146d6e8d6351bb03a4a7b9de3fad843cedde643af3
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f6eb69d7a40ff31b5ba423146d6e8d6351bb03a4a7b9de3fad843cedde643af3
Associated tags:
- 20210222-060023
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-060023
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-060023].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f6eb69d7a40ff31b5ba423146d6e8d6351bb03a4a7b9de3fad843cedde643af3].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7469
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7469/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-000022
---> 327b22cce35d
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 168623e98ba5
Removing intermediate container 168623e98ba5
---> b8418d4e7320
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 493ef9473246
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in eb789ede8acc
[91m2021/02/22 00:20:18 Installing setup packages ...
[0m[91m2021/02/22 00:20:18 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:9d67ec8f-beab-4216-b137-b05f0b4ceeec']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 594.780s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-000022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:c9e4c409a6733054fbf19c3e65d813510d4e9e1d16a5fc225299e151121fd0ea
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:c9e4c409a6733054fbf19c3e65d813510d4e9e1d16a5fc225299e151121fd0ea
Associated tags:
- 20210222-000022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-000022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-000022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:c9e4c409a6733054fbf19c3e65d813510d4e9e1d16a5fc225299e151121fd0ea].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7468
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7468/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-180021
---> e96bd71e0402
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in da84510aca97
Removing intermediate container da84510aca97
---> a1f3ad98e3f6
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 1624e9ba5d2d
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in ee5cdac2c346
[91m2021/02/21 18:20:14 Installing setup packages ...
[0m[91m2021/02/21 18:20:14 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:61e0bfa5-d815-487e-97d9-c8b2e246d6d0']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 634.644s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-180021
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5a8248e6084b6564ed01369fa0fa6d1cc8ae4704fc87787567219b0515c5aeaf
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5a8248e6084b6564ed01369fa0fa6d1cc8ae4704fc87787567219b0515c5aeaf
Associated tags:
- 20210221-180021
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-180021
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-180021].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5a8248e6084b6564ed01369fa0fa6d1cc8ae4704fc87787567219b0515c5aeaf].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7467
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7467/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-120022
---> cdd414a29b54
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 17f5bf018653
Removing intermediate container 17f5bf018653
---> 5a8bee104db9
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 2eb98a6bbc0f
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in cd518c2a55aa
[91m2021/02/21 12:20:49 Installing setup packages ...
[0m[91m2021/02/21 12:20:49 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:8f329f1b-0deb-4ae1-b72c-407009d3dc1c']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 592.728s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-120022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5043e2d9c5a7ba1e3b6dfec50f08f1bcd17cef16acb9b2563541fbfb6539ddb1
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5043e2d9c5a7ba1e3b6dfec50f08f1bcd17cef16acb9b2563541fbfb6539ddb1
Associated tags:
- 20210221-120022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-120022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-120022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:5043e2d9c5a7ba1e3b6dfec50f08f1bcd17cef16acb9b2563541fbfb6539ddb1].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7466
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7466/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-060023
---> c09cde5d8b97
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 4ac3f87f7c86
Removing intermediate container 4ac3f87f7c86
---> a00afb786f67
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 4a5d31dbcc71
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in d5d1e01d4748
[91m2021/02/21 06:20:11 Installing setup packages ...
[0m[91m2021/02/21 06:20:11 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:e089dad8-561a-449d-8ce7-7210a1edd37d']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 570.684s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-060023
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:be6ccf05acafe34d6fe61432e7d7c732f99c9f8ce45d2f58661938c21bea0781
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:be6ccf05acafe34d6fe61432e7d7c732f99c9f8ce45d2f58661938c21bea0781
Associated tags:
- 20210221-060023
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-060023
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-060023].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:be6ccf05acafe34d6fe61432e7d7c732f99c9f8ce45d2f58661938c21bea0781].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7465
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7465/display/redirect?page=changes>
Changes:
[noreply] [BEAM-7372] remove aliases created for python 2 (#14012)
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-000022
---> 3ef515a6e53c
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 5fecb341531a
Removing intermediate container 5fecb341531a
---> 8f719a156b71
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 03a64137fa0a
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 5743c82265e4
[91m2021/02/21 00:21:38 Installing setup packages ...
[0m[91m2021/02/21 00:21:38 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:8dfcf132-bd4c-4d41-9fed-3a04af72cb5d']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 672.349s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-000022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:7882d815e67726f9a24b223d9c9bf95910d0503ac0b4d43849858459596f5c1f
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:7882d815e67726f9a24b223d9c9bf95910d0503ac0b4d43849858459596f5c1f
Associated tags:
- 20210221-000022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-000022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210221-000022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:7882d815e67726f9a24b223d9c9bf95910d0503ac0b4d43849858459596f5c1f].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7464
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7464/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-180021
---> c7f12bf0a538
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 8490c9aeef26
Removing intermediate container 8490c9aeef26
---> 88138f1de33e
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 70cadba93c11
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 6528dccfd9cd
[91m2021/02/20 18:20:12 Installing setup packages ...
[0m[91m2021/02/20 18:20:12 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:f9bf1960-fc32-4ff1-9458-d7fc01e33afe']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 572.715s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-180021
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:336e82668d45fdd7a7567a8980bb860eea01ddfbdf05bb57b8d7f3a47ccc8d59
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:336e82668d45fdd7a7567a8980bb860eea01ddfbdf05bb57b8d7f3a47ccc8d59
Associated tags:
- 20210220-180021
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-180021
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-180021].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:336e82668d45fdd7a7567a8980bb860eea01ddfbdf05bb57b8d7f3a47ccc8d59].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7463
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7463/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-120021
---> 977a1629ed1b
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 6c668ad6a982
Removing intermediate container 6c668ad6a982
---> d682eb229677
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 1734116c3cf6
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 84e4f3d9430d
[91m2021/02/20 12:20:46 Installing setup packages ...
[0m[91m2021/02/20 12:20:46 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:0ffdc7ee-93da-4731-97b4-d79bd64848b2']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 620.991s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-120021
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:009373f8e174d1dd06bacdda9fa82f6ecd8f5e20a3dfb9d4673c55fcc1bdeda0
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:009373f8e174d1dd06bacdda9fa82f6ecd8f5e20a3dfb9d4673c55fcc1bdeda0
Associated tags:
- 20210220-120021
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-120021
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-120021].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:009373f8e174d1dd06bacdda9fa82f6ecd8f5e20a3dfb9d4673c55fcc1bdeda0].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7462
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7462/display/redirect?page=changes>
Changes:
[Robert Bradshaw] Remove side input combine examples that don't work on most runners.
[Robert Bradshaw] [BEAM-11842] Add resource hints to model proto.
[noreply] Add null checks to FhirIO.Search for when the search results are empty.
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-060022
---> 8200c020b078
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in be33ed830dd9
Removing intermediate container be33ed830dd9
---> f58d5e4490da
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 3b1b1f6548a6
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in e02d1a8480c7
[91m2021/02/20 06:19:51 Installing setup packages ...
[0m[91m2021/02/20 06:19:51 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:5dcf87c8-a210-4070-ac7a-82e6579e90ec']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 637.954s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-060022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:bf35a081fa308f9a371db50d5a513a3fc8fda1b93efa3bc5cb7172e6ee3e22af
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:bf35a081fa308f9a371db50d5a513a3fc8fda1b93efa3bc5cb7172e6ee3e22af
Associated tags:
- 20210220-060022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-060022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-060022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:bf35a081fa308f9a371db50d5a513a3fc8fda1b93efa3bc5cb7172e6ee3e22af].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7461
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7461/display/redirect?page=changes>
Changes:
[randomstep] [BEAM-11120] bump Gradle License Report plugin to 1.16
[zyichi] [BEAM-11779] Fix python pipeline options for CrossLanguage tests
[noreply] [BEAM-11800] Support ARRAY_AGG fn for Zetasql dialect (#13483)
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-000038
---> 7b1787d12632
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in cbc40a45af5b
Removing intermediate container cbc40a45af5b
---> b15b0c77f845
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 91d9ecd093d5
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 086e28f12efc
[91m2021/02/20 00:31:44 Installing setup packages ...
[0m[91m2021/02/20 00:31:44 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:9e08d986-5eb2-4a9f-9cb7-765b7f3d5413']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 623.873s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-000038
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f885b4c32fce5bce3d7e8ee8454e9a0720bf04d9e339e3a4c30d88a4e5ad68f7
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f885b4c32fce5bce3d7e8ee8454e9a0720bf04d9e339e3a4c30d88a4e5ad68f7
Associated tags:
- 20210220-000038
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-000038
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210220-000038].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f885b4c32fce5bce3d7e8ee8454e9a0720bf04d9e339e3a4c30d88a4e5ad68f7].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7460
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7460/display/redirect?page=changes>
Changes:
[noreply] [BEAM-9547] Fill in some trivial compliance gaps (#13932)
[noreply] [BEAM-11589] Migrated vendor/bytebuddy build.gradle file to Kotlin
[noreply] Merge pull request #14019 from [BEAM-11791] Fixing FnApiRunner
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-180022
---> d905f581884f
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 338baf72216c
Removing intermediate container 338baf72216c
---> 72e19ea8592a
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 23deaa43fe4a
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 87df32d46a1f
[91m2021/02/19 18:21:27 Installing setup packages ...
[0m[91m2021/02/19 18:21:27 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:0156f93a-820f-474d-8afd-c4ab932c82b0']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 642.759s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-180022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:530e90a5da51bd1551cc513f1a2e4bdebe15c5ef779306118027bc60670742b6
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:530e90a5da51bd1551cc513f1a2e4bdebe15c5ef779306118027bc60670742b6
Associated tags:
- 20210219-180022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-180022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-180022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:530e90a5da51bd1551cc513f1a2e4bdebe15c5ef779306118027bc60670742b6].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7459
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7459/display/redirect?page=changes>
Changes:
[noreply] Adding performance improvements to ApproximateQuantiles. (#13175)
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.737MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-120026
---> 8ac58c6a2f93
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 67c5291302ef
Removing intermediate container 67c5291302ef
---> 14ecf5dd1743
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 8dd0d9ecad14
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 7668b4d5700f
[91m2021/02/19 12:20:31 Installing setup packages ...
[0m[91m2021/02/19 12:20:31 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:7977983a-4b9c-448b-abc4-a7db743bd86c']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 598.487s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-120026
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:10aee2449b193569fedf08f76721efdcc72eea0eb5adc766ef6abb7e8e6f89aa
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:10aee2449b193569fedf08f76721efdcc72eea0eb5adc766ef6abb7e8e6f89aa
Associated tags:
- 20210219-120026
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-120026
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-120026].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:10aee2449b193569fedf08f76721efdcc72eea0eb5adc766ef6abb7e8e6f89aa].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7458
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7458/display/redirect?page=changes>
Changes:
[dmytrokozhevin] [BEAM-11760] Cleanup interactive cache just once.
[dmytrokozhevin] [BEAM-11760] Cleanup interactive cache just once.
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.734MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-060022
---> 0e6aa86e66da
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in dd3e0bfd3804
Removing intermediate container dd3e0bfd3804
---> 57eb166c58a6
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 30d5ed998786
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in a54ce76a4b51
[91m2021/02/19 06:20:55 Installing setup packages ...
[0m[91m2021/02/19 06:20:55 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:1ce86ac6-5cc3-4ce9-8e8a-66cf0b7352da']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 632.005s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-060022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:53e727a7ddb480b6fbfd7325e9c5e5db5a7835292582e38516a84fffa9279efe
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:53e727a7ddb480b6fbfd7325e9c5e5db5a7835292582e38516a84fffa9279efe
Associated tags:
- 20210219-060022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-060022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-060022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:53e727a7ddb480b6fbfd7325e9c5e5db5a7835292582e38516a84fffa9279efe].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7457
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7457/display/redirect?page=changes>
Changes:
[Kyle Weaver] [BEAM-11834] Enable arrays literals to have null values.
[Pablo Estrada] Enable SideInput metrics for DF worker. These have been launched on
[noreply] [BEAM-10708] Support SqlTransform in container (#13944)
------------------------------------------
[...truncated 897.67 KB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.734MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-000025
---> bd67d6dc49a9
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 87cddb0d2722
Removing intermediate container 87cddb0d2722
---> e0c86ad39bfa
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> bd02c060d982
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in e5c2cbd8db96
[91m2021/02/19 00:20:23 Installing setup packages ...
[0m[91m2021/02/19 00:20:23 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:42f87f1d-d2fc-4901-bba0-4839693d10ee']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 655.210s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-000025
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:08bcf142920797db7f2ae127990dcd90adebab9a0a2bba7aad149d5906492fef
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:08bcf142920797db7f2ae127990dcd90adebab9a0a2bba7aad149d5906492fef
Associated tags:
- 20210219-000025
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-000025
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210219-000025].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:08bcf142920797db7f2ae127990dcd90adebab9a0a2bba7aad149d5906492fef].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7456
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7456/display/redirect?page=changes>
Changes:
[Steve Niemitz] [BEAM-11801] Don't set useCachedDataPool if an emulator host is set
[baeminbo] Fix IllegalArgumentException in Interval
[Andrew Pilloud] [BEAM-11747] Better filters for BeamCalc
[Kyle Weaver] [BEAM-11747] Disable BeamJavaUdfCalcRule by default.
[Kyle Weaver] spotless
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.734MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-180022
---> 1c6a53980122
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 2891a71297d0
Removing intermediate container 2891a71297d0
---> 120678f913f8
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 7eaa53abf8b5
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 58533a01f818
[91m2021/02/18 18:20:42 Installing setup packages ...
[0m[91m2021/02/18 18:20:42 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:86f246e2-3ce0-4cbf-8174-b3ad17cfcef7']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 652.088s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-180022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:e5d70fe8b4dbe08c503d6099adfcc9f2ce89695e5018db3ed46f2e209f82d502
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:e5d70fe8b4dbe08c503d6099adfcc9f2ce89695e5018db3ed46f2e209f82d502
Associated tags:
- 20210218-180022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-180022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-180022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:e5d70fe8b4dbe08c503d6099adfcc9f2ce89695e5018db3ed46f2e209f82d502].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7455
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7455/display/redirect?page=changes>
Changes:
[randomstep] [BEAM-11125] bump checkerframework to 3.10.0
[noreply] Merge #13558. [BEAM-11494][BEAM-11821] FileIO stops overwriting files on
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.734MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-120022
---> 465f1e6a2cd3
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 488acb00f1cf
Removing intermediate container 488acb00f1cf
---> a1b47521a636
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 21d9c2695755
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in f572c01ca735
[91m2021/02/18 12:19:27 Installing setup packages ...
[0m[91m2021/02/18 12:19:27 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:77263518-d097-4d5f-90b5-3d388621ac92']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 603.006s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-120022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:ee76a498efd116a94519ce270bdad1029a26ceae99d4e2f8980d27efd98d2eaf
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:ee76a498efd116a94519ce270bdad1029a26ceae99d4e2f8980d27efd98d2eaf
Associated tags:
- 20210218-120022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-120022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-120022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:ee76a498efd116a94519ce270bdad1029a26ceae99d4e2f8980d27efd98d2eaf].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7454
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7454/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11211] Update pandas and pyarrow in python container (#13987)
[noreply] Update go version to 1.12.7 (#13996)
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.734MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-060024
---> 46aee9cd9d41
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 53515c3ce873
Removing intermediate container 53515c3ce873
---> 6d1af0c61d20
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 46c90a9f7793
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 169c7c01c297
[91m2021/02/18 06:20:36 Installing setup packages ...
[0m[91m2021/02/18 06:20:36 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:758efb39-69e1-4694-a9a4-3b6e48c7c872']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 606.279s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-060024
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:79fd321a807b5d76a05fce4364634f46826c9ca821a7ef351ea9a077ee071fd3
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:79fd321a807b5d76a05fce4364634f46826c9ca821a7ef351ea9a077ee071fd3
Associated tags:
- 20210218-060024
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-060024
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-060024].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:79fd321a807b5d76a05fce4364634f46826c9ca821a7ef351ea9a077ee071fd3].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7453
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7453/display/redirect?page=changes>
Changes:
[zyichi] [BEAM-11779] Use shuffle_mode=appliance when using dataflow runner v2
[zyichi] Add comment to remove the patch
[zyichi] minor fix to run_integration_test.sh in python sdk
[Pablo Estrada] Fixing dashboard
[Andrew Pilloud] [BEAM-10925] Support selecting Calc for ZetaSQL testing
[ajamato] Revert "[BEAM-2914] Add portable merging window support to Python.
[noreply] [BEANM-11827] checkJavaLinkage task requires a spread operator for args
[noreply] [BEAM-11805] Replace user-agent for spanner (#13990)
------------------------------------------
[...truncated 764.61 KB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.734MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-000022
---> 170fa7a2e24a
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in f1e74a009d50
Removing intermediate container f1e74a009d50
---> 1cf48939ab56
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 8cb7b6af4819
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 4393901944a5
[91m2021/02/18 00:28:19 Installing setup packages ...
[0m[91m2021/02/18 00:28:19 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:46304dce-a0b7-4b61-a3ac-3e112efb2cba']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 646.592s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-000022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:b33a3ac1385da55027f29dc5be7afac728efaf86a34c771e89c47a73dbbe1563
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:b33a3ac1385da55027f29dc5be7afac728efaf86a34c771e89c47a73dbbe1563
Associated tags:
- 20210218-000022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-000022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210218-000022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:b33a3ac1385da55027f29dc5be7afac728efaf86a34c771e89c47a73dbbe1563].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7452
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7452/display/redirect?page=changes>
Changes:
[juanitodread] Beam transform that uses DebeziumIO connector to support CDC
[noreply] [BEAM-11626] Upgrading Guava to 30.1-jre while keeping 25.1-jre for
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-180031
---> 8a80020b9f34
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 755da1b39db0
Removing intermediate container 755da1b39db0
---> 1b7eaf46ea25
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 3122dc41e40f
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 6e4ffc42924a
[91m2021/02/17 18:47:19 Installing setup packages ...
[0m[91m2021/02/17 18:47:19 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:a7a6c761-ef2c-4d3f-a981-3031f039db2d']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 741.686s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-180031
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f9e32ca2afe33e96a325b6f1ffa842c6c2211ee7b84c7367edc2d3feba86640e
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f9e32ca2afe33e96a325b6f1ffa842c6c2211ee7b84c7367edc2d3feba86640e
Associated tags:
- 20210217-180031
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-180031
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-180031].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f9e32ca2afe33e96a325b6f1ffa842c6c2211ee7b84c7367edc2d3feba86640e].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7451
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7451/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-120021
---> 06fdde07a6d0
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 2ae5bbfd0be5
Removing intermediate container 2ae5bbfd0be5
---> cff57b062234
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> ea0076728bd1
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 17218a98ae7c
[91m2021/02/17 12:21:20 Installing setup packages ...
[0m[91m2021/02/17 12:21:20 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:7708196e-1ab3-4b76-8144-ebf0063c2bd3']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 612.928s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-120021
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:e2a9fcbb8200a0d8f8a7813adc8c8d9cb165ae733f83618a4d6c1d766234dbe5
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:e2a9fcbb8200a0d8f8a7813adc8c8d9cb165ae733f83618a4d6c1d766234dbe5
Associated tags:
- 20210217-120021
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-120021
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-120021].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:e2a9fcbb8200a0d8f8a7813adc8c8d9cb165ae733f83618a4d6c1d766234dbe5].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7450
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7450/display/redirect?page=changes>
Changes:
[daniel.o.programmer] [BEAM-11415] Fix misc. bugs affecting Go XLang tests.
[noreply] [BEAM-11779] Fix dataflow URL printed by Go. (#13997)
[noreply] [BEAM-9615] Register Schemas on Beam Init (#13940)
------------------------------------------
[...truncated 765.49 KB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-060027
---> 1ab338071522
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in b6f7f73266a8
Removing intermediate container b6f7f73266a8
---> 0fe85071d2a2
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 54c145855ce9
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 29efd4c36af5
[91m2021/02/17 06:26:28 Installing setup packages ...
[0m[91m2021/02/17 06:26:28 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:e45fe2d2-d337-403d-b828-5df435c12cc1']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 605.209s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-060027
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:0df63eacbc504bc497ab43c4c8b57420c1372cec66f83ce9ad7805fa763bd383
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:0df63eacbc504bc497ab43c4c8b57420c1372cec66f83ce9ad7805fa763bd383
Associated tags:
- 20210217-060027
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-060027
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-060027].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:0df63eacbc504bc497ab43c4c8b57420c1372cec66f83ce9ad7805fa763bd383].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7449
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7449/display/redirect?page=changes>
Changes:
[zyichi] Minor fixes for python nexmark benchmarks
[Kyle Weaver] [BEAM-11811] [Dataflow Java] Don't allow numWorkers > maxNumWorkers.
[Kyle Weaver] Don't create unassigned variable.
[noreply] Removed conscrypt from a shaded dataflow-worker jar (#13846)
[noreply] [BEAM-11747] Reject the mixed Java UDF and ZetaSQL builtin operator
[noreply] [BEAM-11589] Migrated release build.gradle file to Kotlin script
[noreply] Bump dataflow container version (#13988)
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-000020
---> 8de14456a53f
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 4a5f0d0d0a98
Removing intermediate container 4a5f0d0d0a98
---> 775c1dac8d0b
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 47160ed6409d
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 05db584bf771
[91m2021/02/17 00:20:50 Installing setup packages ...
[0m[91m2021/02/17 00:20:50 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:0e1134c4-0f19-4390-8763-c9e03481120e']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 627.720s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-000020
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f81f8a89d45acbda02777f471a343c63acad53a6e083e127364594ce27dd1a9b
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f81f8a89d45acbda02777f471a343c63acad53a6e083e127364594ce27dd1a9b
Associated tags:
- 20210217-000020
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-000020
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210217-000020].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:f81f8a89d45acbda02777f471a343c63acad53a6e083e127364594ce27dd1a9b].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7448
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7448/display/redirect?page=changes>
Changes:
[noreply] Move payload providers out into their own module. (#13963)
------------------------------------------
[...truncated 884.62 KB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-180033
---> af09834e7a30
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 43d0b412b93b
Removing intermediate container 43d0b412b93b
---> 038d5f329148
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 92e0c8453c61
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 270bd8ea247b
[91m2021/02/16 18:21:22 Installing setup packages ...
[0m[91m2021/02/16 18:21:22 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:6f6014c3-a229-4de5-80b9-0e0f974663ca']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 628.004s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-180033
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:17f39a920dba42f3faa47770eaf047226eea53d20ea67b05f330f98a8d8f2e2f
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:17f39a920dba42f3faa47770eaf047226eea53d20ea67b05f330f98a8d8f2e2f
Associated tags:
- 20210216-180033
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-180033
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-180033].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:17f39a920dba42f3faa47770eaf047226eea53d20ea67b05f330f98a8d8f2e2f].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7447
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7447/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-120023
---> 1ad8912211be
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 5a9cc84d754d
Removing intermediate container 5a9cc84d754d
---> 5e443ad900f8
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> a58092aedf69
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 2cb64817b3db
[91m2021/02/16 12:21:43 Installing setup packages ...
[0m[91m2021/02/16 12:21:43 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:8426114f-3765-4cd6-a600-8c27dd1084c8']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 639.594s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-120023
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:cd7d847e06a24934d955b1d1f05b14491f55da355015c4a96342cf350a374a3e
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:cd7d847e06a24934d955b1d1f05b14491f55da355015c4a96342cf350a374a3e
Associated tags:
- 20210216-120023
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-120023
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-120023].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:cd7d847e06a24934d955b1d1f05b14491f55da355015c4a96342cf350a374a3e].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7446
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7446/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-060023
---> 0730d7a89025
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 56d3f737cf5f
Removing intermediate container 56d3f737cf5f
---> bd1165a6991e
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> fba1c54e5633
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 18c19fcc2bc2
[91m2021/02/16 06:20:44 Installing setup packages ...
[0m[91m2021/02/16 06:20:44 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:af5561fd-c16b-49c9-ab62-d4d64549b8c8']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 651.520s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-060023
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:bc808508cd926247596c0c7dc2fd59fb1b380baf74b2999ed8d0ab0442291f39
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:bc808508cd926247596c0c7dc2fd59fb1b380baf74b2999ed8d0ab0442291f39
Associated tags:
- 20210216-060023
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-060023
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-060023].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:bc808508cd926247596c0c7dc2fd59fb1b380baf74b2999ed8d0ab0442291f39].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7445
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7445/display/redirect?page=changes>
Changes:
[randomstep] [BEAM-11124] bump joda-time to 2.10.10
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-000028
---> a4036d519058
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in e20ab08dda3c
Removing intermediate container e20ab08dda3c
---> 748a961d9c6d
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> fe917bd549c7
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in e640aea9130b
[91m2021/02/16 00:20:28 Installing setup packages ...
[0m[91m2021/02/16 00:20:28 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:fc9521ca-00e4-4c4d-a411-076e03c71b2e']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 651.894s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-000028
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:4cdb45ac3878b1593e8463ca62b420fa1dc45646c0d8c4f97d4845269e03aead
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:4cdb45ac3878b1593e8463ca62b420fa1dc45646c0d8c4f97d4845269e03aead
Associated tags:
- 20210216-000028
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-000028
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210216-000028].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:4cdb45ac3878b1593e8463ca62b420fa1dc45646c0d8c4f97d4845269e03aead].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7444
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7444/display/redirect?page=changes>
Changes:
[noreply] [BEAM-11806] Explicit Partition Support for KafkaIO.WriteRecords
------------------------------------------
[...truncated 763.37 KB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-180028
---> 1a0f340a31a9
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in a66a3ca80120
Removing intermediate container a66a3ca80120
---> 71c50b5a5bb7
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> aa7a07a278ce
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 9ac4d09b884c
[91m2021/02/15 18:26:07 Installing setup packages ...
[0m[91m2021/02/15 18:26:07 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:1f3c860f-481d-4942-8230-5bf18293b166']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 609.614s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-180028
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:3e510755c040a56756dd73cef0f34bf30418cfa2e9e268cdc0f197fe52d71ddb
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:3e510755c040a56756dd73cef0f34bf30418cfa2e9e268cdc0f197fe52d71ddb
Associated tags:
- 20210215-180028
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-180028
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-180028].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:3e510755c040a56756dd73cef0f34bf30418cfa2e9e268cdc0f197fe52d71ddb].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7443
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7443/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-120023
---> c6b45deeb68e
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 1327eb7f4b30
Removing intermediate container 1327eb7f4b30
---> e0d18f4ecab6
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> c2317f802cca
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 7cb193e8f50c
[91m2021/02/15 12:21:41 Installing setup packages ...
[0m[91m2021/02/15 12:21:41 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:c0bfbead-c7fa-4f2b-911d-753fa914ead7']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 648.197s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-120023
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:411d07eeeb0b79163b63316eda93e912090b9241b3b0fc1847e5dcba4587fae3
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:411d07eeeb0b79163b63316eda93e912090b9241b3b0fc1847e5dcba4587fae3
Associated tags:
- 20210215-120023
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-120023
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-120023].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:411d07eeeb0b79163b63316eda93e912090b9241b3b0fc1847e5dcba4587fae3].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7442
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7442/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-060022
---> 3cf4268f049d
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in c83602f21af4
Removing intermediate container c83602f21af4
---> 15147dc2a10a
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 02f51204d07f
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 5f2f783e4e02
[91m2021/02/15 06:20:49 Installing setup packages ...
[0m[91m2021/02/15 06:20:49 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:a268ae20-3564-45b9-97f2-12bd1a925338']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 608.212s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-060022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:b9f2e3d411619208400b3cae4c61845bf1aedd27f2a9591b145b8af3ceb8af06
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:b9f2e3d411619208400b3cae4c61845bf1aedd27f2a9591b145b8af3ceb8af06
Associated tags:
- 20210215-060022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-060022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-060022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:b9f2e3d411619208400b3cae4c61845bf1aedd27f2a9591b145b8af3ceb8af06].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7441
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7441/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-000022
---> 65d9978691c5
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in c7e7404a2123
Removing intermediate container c7e7404a2123
---> 61dc187a3e83
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 0a1265f3bf8c
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 373bad369314
[91m2021/02/15 00:21:27 Installing setup packages ...
[0m[91m2021/02/15 00:21:27 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:fa848246-1179-4684-925c-1abd63f99f84']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 637.661s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-000022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:0e8e336030b2cd861b79cf647b58c484b8cac6247abf0785d2f7f47a292d0a2d
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:0e8e336030b2cd861b79cf647b58c484b8cac6247abf0785d2f7f47a292d0a2d
Associated tags:
- 20210215-000022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-000022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210215-000022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:0e8e336030b2cd861b79cf647b58c484b8cac6247abf0785d2f7f47a292d0a2d].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7440
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7440/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-180022
---> 3f36b814e82f
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in da27d01d6a6a
Removing intermediate container da27d01d6a6a
---> f0993660b991
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> f8d76df109f0
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in b68122139f3b
[91m2021/02/14 18:21:01 Installing setup packages ...
[0m[91m2021/02/14 18:21:01 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:3d59b5fa-5315-4e87-b474-315086b4e5bb']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 637.051s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-180022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:7f02bc2864e450d8afe06c1482125eb9c165c290f968d0d8526c2468be49403c
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:7f02bc2864e450d8afe06c1482125eb9c165c290f968d0d8526c2468be49403c
Associated tags:
- 20210214-180022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-180022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-180022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:7f02bc2864e450d8afe06c1482125eb9c165c290f968d0d8526c2468be49403c].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7439
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7439/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-120022
---> c5d41fbf769e
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in e4ee085f2217
Removing intermediate container e4ee085f2217
---> 80cbef42248b
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> a260ac86d47f
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in b3d4b32f710c
[91m2021/02/14 12:20:51 Installing setup packages ...
[0m[91m2021/02/14 12:20:51 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:a0af6870-f029-4e35-a314-674888ec8709']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 632.329s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-120022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:64dea93900b5829145d13ce421e2dba3831290a21f8930f6aac1da8a8e2cc215
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:64dea93900b5829145d13ce421e2dba3831290a21f8930f6aac1da8a8e2cc215
Associated tags:
- 20210214-120022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-120022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-120022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:64dea93900b5829145d13ce421e2dba3831290a21f8930f6aac1da8a8e2cc215].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7438
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7438/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-060022
---> 8c324f972afc
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 298e8eea4417
Removing intermediate container 298e8eea4417
---> b06e7b5f5044
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 2143da91bec9
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in abae52b7a494
[91m2021/02/14 06:20:33 Installing setup packages ...
[0m[91m2021/02/14 06:20:33 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:924ae99b-5f9f-4a65-b305-ba9481a05bc9']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 634.337s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-060022
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:8271d281140b06769d2e367d66d00c8c52aa8fbecc974a818f2480130093960a
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:8271d281140b06769d2e367d66d00c8c52aa8fbecc974a818f2480130093960a
Associated tags:
- 20210214-060022
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-060022
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-060022].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:8271d281140b06769d2e367d66d00c8c52aa8fbecc974a818f2480130093960a].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org
Build failed in Jenkins: beam_PostCommit_Py_ValCont #7437
Posted by Apache Jenkins Server <je...@builds.apache.org>.
See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7437/display/redirect>
Changes:
------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/sideinput_microbenchmark.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz
# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
--attr ValidatesContainer \
--nologcapture \
--processes=1 \
--process-timeout=900 \
--with-xunitmp \
--xunitmp-file=$XUNIT_FILE \
--ignore-files '.*py3\d?\.py$' \
--test-pipeline-options=" \
--runner=TestDataflowRunner \
--project=$PROJECT \
--region=$REGION \
--worker_harness_container_image=$CONTAINER:$TAG \
--staging_location=$GCS_LOCATION/staging-validatesrunner-test \
--temp_location=$GCS_LOCATION/temp-validatesrunner-test \
--output=$GCS_LOCATION/output \
--sdk_location=$SDK_LOCATION \
--num_workers=1 \
--prebuild_sdk_container_base_image=$CONTAINER:$TAG \
--docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
"VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon 1.736MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-000024
---> 0fd64b57270e
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
---> Running in 2043f5455a9f
Removing intermediate container 2043f5455a9f
---> 21c8e1682b65
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
---> 9d69e5a131b3
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
---> Running in 4fe36fd8f49f
[91m2021/02/14 00:21:41 Installing setup packages ...
[0m[91m2021/02/14 00:21:41 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok
======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
cwd=self._temp_src_dir)
File "/usr/lib/python3.6/subprocess.py", line 438, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:48aefee0-9ecd-40b9-85f8-8013f35ac450']' returned non-zero exit status 1.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
prebuild_sdk_container_engine='local_docker')
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
save_main_session=False,
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
output | 'Write' >> WriteToText(known_args.output)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
self.result = self.run()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
self._options).run(False)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
return self.runner.run_pipeline(self, self._options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
self).run_pipeline(pipeline, options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
environments.DockerEnvironment.from_options(options))
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
options)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
return builder._build()
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
self._invoke_docker_build_and_push(container_image_name)
File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
None.
----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 652.352s
FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-000024
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:a612aac6cddbf29441699d6893aac6fa11e547a1429a42aabdf585aeb2da41ba
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.
Usage: docker rmi [OPTIONS] IMAGE [IMAGE...]
Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:a612aac6cddbf29441699d6893aac6fa11e547a1429a42aabdf585aeb2da41ba
Associated tags:
- 20210214-000024
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-000024
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210214-000024].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:a612aac6cddbf29441699d6893aac6fa11e547a1429a42aabdf585aeb2da41ba].
Removed the container
Build step 'Execute shell' marked build as failure
---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org