You are viewing a plain text version of this content. The canonical link for it is here.

Posted to builds@beam.apache.org by Apache Jenkins Server <je...@builds.apache.org> on 2021/02/22 18:26:03 UTC

Build failed in Jenkins: beam_PostCommit_Py_ValCont #7472

See <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/7472/display/redirect>

Changes:


------------------------------------------
[...truncated 1.09 MB...]
copying apache_beam/tools/utils.py -> apache-beam-2.29.0.dev0/apache_beam/tools
copying apache_beam/transforms/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_pipeline.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combinefn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/combiners_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/core.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_source.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/create_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_combiners.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/cy_dataflow_distribution_counter.pyx -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dataflow_distribution_counter_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/deduplicate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/display_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/dofn_lifecycle_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/environments_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_it_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_java.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/external_test_py37.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/periodicsequence_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/py_dataflow_distribution_counter.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sideinputs_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/sql_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.pxd -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/stats_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/timeutil.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/transforms_keyword_only_args_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/trigger_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/userstate_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/util_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/validate_runner_xlang_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/window_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/transforms/write_ptransform_test.py -> apache-beam-2.29.0.dev0/apache_beam/transforms
copying apache_beam/typehints/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/decorators_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/native_type_compatibility_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/opcodes.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/row_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/schemas_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/sharded_key_type_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/trivial_inference_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typecheck_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typed_pipeline_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/typehints/typehints_test_py3.py -> apache-beam-2.29.0.dev0/apache_beam/typehints
copying apache_beam/utils/__init__.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/annotations_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/counters_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/histogram_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/interactive_utils_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/plugin.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/processes_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/profiler_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/proto_utils.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/retry_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sentinel.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/sharded_key.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/shared_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/subprocess_server_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/thread_pool_executor_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/timestamp_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/urns.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.pxd -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value.py -> apache-beam-2.29.0.dev0/apache_beam/utils
copying apache_beam/utils/windowed_value_test.py -> apache-beam-2.29.0.dev0/apache_beam/utils
Writing apache-beam-2.29.0.dev0/setup.cfg
creating dist
Creating tar archive
removing 'apache-beam-2.29.0.dev0' (and everything under it)
SDK_LOCATION=$(find dist/apache-beam-*.tar.gz)
find dist/apache-beam-*.tar.gz

# Run ValidatesRunner tests on Google Cloud Dataflow service
echo ">>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST"
>>> RUNNING DATAFLOW RUNNER VALIDATESCONTAINER TEST
python setup.py nosetests \
  --attr ValidatesContainer \
  --nologcapture \
  --processes=1 \
  --process-timeout=900 \
  --with-xunitmp \
  --xunitmp-file=$XUNIT_FILE \
  --ignore-files '.*py3\d?\.py$' \
  --test-pipeline-options=" \
    --runner=TestDataflowRunner \
    --project=$PROJECT \
    --region=$REGION \
    --worker_harness_container_image=$CONTAINER:$TAG \
    --staging_location=$GCS_LOCATION/staging-validatesrunner-test \
    --temp_location=$GCS_LOCATION/temp-validatesrunner-test \
    --output=$GCS_LOCATION/output \
    --sdk_location=$SDK_LOCATION \
    --num_workers=1 \
    --prebuild_sdk_container_base_image=$CONTAINER:$TAG \
    --docker_registry_push_url=us.gcr.io/$PROJECT/$USER"
running nosetests
running egg_info
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/container/venv/python3.6/lib/python3.6/site-packages/setuptools/dist.py>:461: UserWarning: Normalizing '2.29.0.dev' to '2.29.0.dev0'
  warnings.warn(tmpl.format(**locals()))
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/gen_protos.py>:243: UserWarning: Installing grpcio-tools is recommended for development.
  warnings.warn('Installing grpcio-tools is recommended for development.')
INFO:gen_protos:Skipping proto regeneration: all files up to date
writing apache_beam.egg-info/PKG-INFO
writing dependency_links to apache_beam.egg-info/dependency_links.txt
writing entry points to apache_beam.egg-info/entry_points.txt
writing requirements to apache_beam.egg-info/requires.txt
writing top-level names to apache_beam.egg-info/top_level.txt
reading manifest file 'apache_beam.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'README.md'
warning: no files found matching 'NOTICE'
warning: no files found matching 'LICENSE'
writing manifest file 'apache_beam.egg-info/SOURCES.txt'
WARNING:root:python-snappy is not installed; some tests will be skipped.
WARNING:root:Tensorflow is not installed, so skipping some tests.
<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/io/vcfio.py>:50: UserWarning: VCF IO will support Python 3 after migration to Nucleus, see: BEAM-5628.
  "VCF IO will support Python 3 after migration to Nucleus, "
WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.
WARNING:apache_beam.runners.interactive.interactive_environment:You cannot use Interactive Beam features when you are not in an interactive environment such as a Jupyter notebook or ipython terminal.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
Sending build context to Docker daemon  1.738MB
Step 1/4 : FROM us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-180021
 ---> 6bcdf607ab3b
Step 2/4 : RUN mkdir -p /opt/apache/beam/artifacts
 ---> Running in abbb7c1ef51c
Removing intermediate container abbb7c1ef51c
 ---> a8417693dfcc
Step 3/4 : COPY ./* /opt/apache/beam/artifacts/
 ---> f39f2154e891
Step 4/4 : RUN /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json
 ---> Running in c9e2a1871e83
[91m2021/02/22 18:21:39 Installing setup packages ...
[0m[91m2021/02/22 18:21:39 Failed to install required packages: failed to install SDK: package 'dataflow_python_sdk.tar' not found
[0mThe command '/bin/sh -c /opt/apache/beam/boot --setup_only --artifacts /opt/apache/beam/artifacts/artifacts_info.json' returned a non-zero code: 1
WARNING:root:Make sure that locally built Python SDK docker image has Python 3.6 interpreter.
test_wordcount_fnapi_it (apache_beam.examples.wordcount_it_test.WordCountIT) ... ok
test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT) ... ERROR
test_metrics_fnapi_it (apache_beam.runners.dataflow.dataflow_exercise_metrics_pipeline_test.ExerciseMetricsPipelineTest) ... ok

======================================================================
ERROR: test_wordcount_it_with_prebuilt_sdk_container (apache_beam.examples.wordcount_it_test.WordCountIT)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 173, in _invoke_docker_build_and_push
    cwd=self._temp_src_dir)
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['docker', 'build', '.', '-t', 'us.gcr.io/apache-beam-testing/jenkins/beam_python_prebuilt_sdk:4eb8948c-6bc1-4bc8-9811-df971cd2cd15']' returned non-zero exit status 1.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 64, in test_wordcount_it_with_prebuilt_sdk_container
    prebuild_sdk_container_engine='local_docker')
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount_it_test.py",> line 107, in _run_wordcount_it
    save_main_session=False,
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/examples/wordcount.py",> line 94, in run
    output | 'Write' >> WriteToText(known_args.output)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 580, in __exit__
    self.result = self.run()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 530, in run
    self._options).run(False)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/pipeline.py",> line 559, in run
    return self.runner.run_pipeline(self, self._options)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",> line 57, in run_pipeline
    self).run_pipeline(pipeline, options)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",> line 497, in run_pipeline
    environments.DockerEnvironment.from_options(options))
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/transforms/environments.py",> line 285, in from_options
    options)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 140, in build_container_image
    return builder._build()
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 93, in _build
    self._invoke_docker_build_and_push(container_image_name)
  File "<https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/apache_beam/runners/portability/sdk_container_builder.py",> line 177, in _invoke_docker_build_and_push
    'stderr:\n %s.' % err.stderr)
RuntimeError: Failed to build sdk container with local docker, stderr:
 None.

----------------------------------------------------------------------
XML: nosetests-beam_python3.6_sdk.xml
----------------------------------------------------------------------
XML: <https://ci-beam.apache.org/job/beam_PostCommit_Py_ValCont/ws/src/sdks/python/nosetests.xml>
----------------------------------------------------------------------
Ran 3 tests in 623.196s

FAILED (errors=1)
cleanup_container
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-180021
Untagged: us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:79f6db7ce83b98aea42ba4ebd1e50ed7c6316c07a83a9613f50319c9e24163a2
docker images --format '{{.Repository}}:{{.Tag}}' | grep 'prebuilt_sdk'
"docker rmi" requires at least 1 argument.
See 'docker rmi --help'.

Usage:  docker rmi [OPTIONS] IMAGE [IMAGE...]

Remove one or more images
Failed to remove prebuilt sdk container
Digests:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:79f6db7ce83b98aea42ba4ebd1e50ed7c6316c07a83a9613f50319c9e24163a2
  Associated tags:
 - 20210222-180021
Tags:
- us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-180021
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk:20210222-180021].
Deleted [us.gcr.io/apache-beam-testing/jenkins/beam_python3.6_sdk@sha256:79f6db7ce83b98aea42ba4ebd1e50ed7c6316c07a83a9613f50319c9e24163a2].
Removed the container
Build step 'Execute shell' marked build as failure

---------------------------------------------------------------------
To unsubscribe, e-mail: builds-unsubscribe@beam.apache.org
For additional commands, e-mail: builds-help@beam.apache.org